MegalaniaDocs
Sign inApply for the beta
ProductThe climbPricingMCP for agentsPrivacyDevelopersQuickstartDocsBlogSign in
Docs menu
  1. Docs
  2. Reference
  3. REST API
Reference

REST API

llms.txt

One endpoint, three ways in. POST /v1 with a JSON object is the canonical one and the one the SDKs generate from; GET /v1 with query parameters is the paste-a-URL one; QUERY /v1 takes the POST envelope unchanged and is the RFC 10008 standards path, described under POST because OpenAPI 3.1 has no query path item. Every answer carries Cache-Control: no-store. Megalania fetches the target behind a geo-targeted proxy and streams the result back. Parameter names are the proto field names, except url and country_code. The envelope adds method, body and content_type, which exist only there. The token is the Authorization header, on every route. A request that carries api_key in the query is a 401 and is never charged: a key in a URL is written down by every log the URL passes.

The same contract as an OpenAPI 3 file: openapi.yaml.

GET /v1

Scrape a URL and stream the result.

Synchronous. The connection stays open until the result streams back or the timeout hits. Nothing is buffered; the size cap is 50 MB. A target that answers with a status under 500 is a delivered result: HTTP 200 from us, the target status in X-Target-Status. Blocks we detect are free: a blocked result (401, 403, 407, 429, or a detected challenge) costs nothing and is a 422 blocked with X-Billing: blocked and no page content. A target 5xx is free too: a 422 target_error with the target status in X-Target-Status and no page content.

Error codes action_failed, bad_extract, bad_parameter, bad_token, blocked, busy, capacity_spent, convert_cap, daily_cost_cap, document_unreadable, fetch_error, forbidden_target, free_failures_spent, free_tier_cap, idempotency_mismatch, insufficient_credits, internal, llm_global_stop, llm_token_cap, llm_unavailable, max_credits_too_low, over_concurrency, period_cost_cap, replayed, spend_cap, target_error, target_rate_limited, timeout, unavailable, unknown_host, unsupported_content, upstream_cap, wrong_scope.

Parameters

name in required type or values what it is
url query yes string Absolute http(s) URL to scrape. Paste it as it is with url as the last parameter: everything after url= that is not one of our parameter names belongs to the target. Percent-encode it if the target’s own query uses one of our names or you want our parameters after it.
engine query no http, browser; default http Which engine fetches the page. Leave it out and we pick. http sends one plain request; a page that builds itself in JavaScript comes back as a shell, marked with X-Hint. browser renders the page in Chrome on a datacenter exit, GET only.
country_code query no AD, AE, AF, AG, AI, AL, AM, AO, AQ, AR, AS, AT, AU, AW, AX, AZ, BA, BB, BD, BE, BF, BG, BH, BI, BJ, BL, BM, BN, BO, BQ, BR, BS, BT, BV, BW, BY, BZ, CA, CC, CD, CF, CG, CH, CI, CK, CL, CM, CN, CO, CR, CU, CV, CW, CX, CY, CZ, DE, DJ, DK, DM, DO, DZ, EC, EE, EG, EH, ER, ES, ET, FI, FJ, FK, FM, FO, FR, GA, GB, GD, GE, GF, GG, GH, GI, GL, GM, GN, GP, GQ, GR, GS, GT, GU, GW, GY, HK, HM, HN, HR, HT, HU, ID, IE, IL, IM, IN, IO, IQ, IR, IS, IT, JE, JM, JO, JP, KE, KG, KH, KI, KM, KN, KP, KR, KW, KY, KZ, LA, LB, LC, LI, LK, LR, LS, LT, LU, LV, LY, MA, MC, MD, ME, MF, MG, MH, MK, ML, MM, MN, MO, MP, MQ, MR, MS, MT, MU, MV, MW, MX, MY, MZ, NA, NC, NE, NF, NG, NI, NL, NO, NP, NR, NU, NZ, OM, PA, PE, PF, PG, PH, PK, PL, PM, PN, PR, PS, PT, PW, PY, QA, RE, RO, RS, RU, RW, SA, SB, SC, SD, SE, SG, SH, SI, SJ, SK, SL, SM, SN, SO, SR, SS, ST, SV, SX, SY, SZ, TC, TD, TF, TG, TH, TJ, TK, TL, TM, TN, TO, TR, TT, TV, TW, TZ, UA, UG, UM, US, UY, UZ, VA, VC, VE, VG, VI, VN, VU, WF, WS, YE, YT, ZA, ZM, ZW Exit country, ISO 3166-1 alpha-2. Default is the platform’s configured country.
proxy_city query no string Exit city in the proxy provider’s spelling, for example new_york.
proxy_tier query no datacenter, residential, mobile; default residential Which proxy pool the request leaves from. Cheaper pools are blocked more often.
identity query no firefox, chrome Which browser the fetch looks like on the wire: its TLS hello, its HTTP/2 frames and its headers. Leave it out and we pick the identity that does best on this site. Both cost the same.
session_id query no string Send the same value again to leave from the same IP, so a login and the pages behind it look like one visitor. 1 to 64 letters, digits, - or _. Your sessions are yours alone; the same value from another customer is a different exit. We keep the cookies the session collects on our side for at most 10 minutes and send them with the next request that names it; you never see them. A session_id with no proxy_tier is pinned to residential, which can raise the price. Changing country_code, proxy_city or proxy_tier gives you a new exit and a new, empty cookie jar.
wait_until query no string; default load When the page counts as loaded. load (default), domcontentloaded, networkidle, or selector: of at most 1024 bytes, which waits for that element. Any value but load uses the browser and its price. Example: selector:#price
response_types query no array of html, json, markdown, text, links, metadata, page_data, compact_html, inspect; default html Comma-separated. One type means the body is that type. Several means multipart/mixed, one part per type, in request order.
actions query no string Steps the browser runs after the page loads and wait_until is met, in order, then it returns the page. kind:argument entries joined by |, at most 20: wait: (1 to 10000), wait_for: (up to 10 s for a visible match), click: (the first visible match), type:= (1 to 1024 bytes), press: (Enter, Tab, Escape, Backspace, Space, ArrowUp, ArrowDown, ArrowLeft, ArrowRight, PageUp, PageDown, Home, End), scroll:<bottom, top or pixels>. A selector is at most 1024 bytes and cannot hold |. All actions together get at most 30 s inside timeout. It needs the browser, so leaving engine out sends the request to the browser, and engine=http is a 400. Adds 3 credits to the page. If an action cannot run, the answer is a free 422 action_failed whose message names the entry’s index. Send type by POST, so the text is not in a URL. Batches take actions too. Example: click:.more|wait_for:.item-2
screenshot query no viewport, full A JPEG of the page as the browser rendered it: viewport (the visible window) or full (the whole page, at most 16,384 px tall). It needs the browser, so leaving engine out sends the request to the browser, and engine=http is a 400. The answer is multipart/mixed with a part named screenshot: {url, expires_at, bytes, credits}, a link that works for 1 hour and downloads as screenshot--.jpg; the image is deleted after 7 days. X-Credits-Charged is the page alone; the part credits is what the screenshot adds (1 viewport, 2 full), charged only when the link is delivered. If it could not be made, the part is {error: screenshot_failed, credits: 0}. A blocked page charges nothing. Batches take screenshot too: the stored result keeps bytes and credits, and each item read signs a fresh 1-hour screenshot_url.
network query no true, false true also returns the JSON and text the page itself loaded with XHR or fetch, as a part named network in application/x-ndjson, one JSON object per response: {url, method, status, content_type, bytes, truncated, body}. A body that is not UTF-8 text is base64, with encoding: base64. bytes is the body’s size; when the body was too large to keep (over 2 MiB) the body is empty, truncated is true, and bytes is its encoded size on the wire. At most 200 responses, 1 MiB per body (then truncated is true) and 8 MiB in all; X-Network-Truncated says when the list was cut. Only the page’s own XHR and fetch are captured, not workers or service workers. No request or response headers, no cookies and no request bodies. It needs the browser, so leaving engine out sends the request to the browser, and engine=http is a 400. Adds no credits; the page’s bytes are already metered. Batches take network too.
network_url_globs query no string With network=true, capture only the responses whose full URL matches one of these patterns, of any content type. A comma list of at most 10 patterns, 1024 bytes in all; * matches anything, ? one character. Without it, network captures JSON, NDJSON and text responses. Example: /api/
only_main_content query no true, false; default true true returns only the page’s main content in markdown, text and compact_html. false returns the whole page and needs markdown, text or compact_html in response_types. Any other value is a 400.
extract query no string Selector rules, required with response_types=json and only valid with it. : joined by |, at most 32 rules and 4096 bytes. Add @attr to read an attribute instead of the text, and a trailing * to get a list. The answer is bounded: 8 MB in total, 1000 matches for one list rule, 1 MB for one value. A rule that reaches a bound stops there and the object gains truncated: true. That is a delivered result and is charged, so truncated is the one name a rule cannot have. If the cut leaves every field empty and json is the only output, the answer is a free 422 convert_cap on field extract. Example: title:h1|links:a@href*
schema query no string A JSON Schema object (at most 8192 bytes) with type object at the top and additionalProperties false on every object; it may use type, properties, required, items, enum, const, anyOf, format, description and title. With response_types=json and no extract, a language model fills the schema from the page’s main content. The json part is then {data, credits, llm_credits, tokens}: data matches the schema; credits equals X-Credits-Charged, the whole request (page plus extraction); llm_credits is the extraction’s part (2 plus a rate per started 1,000 weighted tokens, input + 5 x output); tokens is that weighted count. The page’s main content, the schema and the prompt go to the model provider: Anthropic in the US. An answer that does not match the schema is a 422 bad_extract: the extraction is free, the page is still charged, and the body carries the page as markdown.
prompt query no string Extra instructions for the model, at most 2048 bytes. Needs schema.
extract_model query no fast, default, non_china; default default fast, default or non_china. An alias, never a vendor: we never use a model of China origin, whatever its host. Today all three aliases run claude-haiku-4-5. Needs schema.
headers query no array of string Extra request header as name:value. Repeat the parameter for more than one. Host, Connection, Content-Length, Transfer-Encoding, TE, Trailer, Upgrade, Expect, Keep-Alive, Via, HTTP2-Settings and any Proxy- header are set for you; sending one is a 400.
timeout query no integer, 1 to 90; default 60 Seconds to wait for the result.
max_credits query no integer, 1 to 32767 The most credits one page may cost, 1 to 32767. Routes priced above it are left out of the climb and the page’s byte cap shrinks to fit it. If no route is left, the answer is a free 400 max_credits_too_low naming the lowest price. A page whose bytes cost more than it is a 200 cut at the bytes it pays for, with X-Truncated max_credits.
max_age query no integer, 0 to 900; default 300 Serve a copy of this page fetched at most this many seconds ago, 0 to 900, 300 when you leave it out. A copy costs the cache price from the price list and says X-Cache hit. With no copy that new, we fetch the page and keep a copy for a short time. 0 always fetches fresh. A request with method, body, headers, actions, screenshot, network, session_id or a user name in the URL never uses the cache; sending max_age with one of them is a 400.
Idempotency-Key header no string Your own id for this call, 1 to 128 printable ASCII characters, scoped to your account. The first call with it runs and is charged. Any later call with the same key, inside 24 hours, answers 409 replayed, charges nothing, and carries the first call’s X-Request-Id. Anything we refund releases the key - a 402, a 429, and every 5xx of ours - so retrying the same request with the same key is safe. A delivered result keeps it, blocked pages included. Send the header once: two Idempotency-Key headers are a 400 naming it. After 24 hours a key is forgotten and the same key is a new, charged request. Without the header nothing changes.

Answers

status what it means
200 Result delivered. The body is the requested type, or multipart/mixed for several.
400 Bad parameter. error.field names it and error.message lists the allowed values.
401 Bad or missing token, or a token in the query. Nothing is charged.
402 Not enough credits.
403 wrong_scope, an account key (meg_ak_). Scraping needs a scrape key (meg_sk_). Nothing is charged.
409 This Idempotency-Key was already used. Nothing is charged, and X-Request-Id names the first call.
422 convert_cap. The page is too large or too complex to convert into the response_types you asked for (markdown, text, links, metadata, page_data, compact_html, inspect or json). Nothing is charged. Ask for html, or for fewer response_types. upstream_cap. The page pulled more data than one request may use. Credits refunded. Fetch a smaller page. action_failed. An entry in actions could not run; error.field is actions and the message names the entry’s index. Nothing is charged, unless X-Billing is failure_cap; then X-Credits-Charged and its three parts say what the answer cost. Or blocked. The site blocked us; a free answer carries no page content, only this error. X-Billing is blocked and X-Untried names a route left to try. Nothing is charged, unless X-Billing is failure_cap. target_error. The target answered 5xx; X-Target-Status has its status. Nothing is charged.
429 Over the plan’s concurrency limit. Nothing is charged. Retry after the Retry-After seconds.
500 Our bug. Credits refunded.
503 Credits refunded. error.code says why. busy, all engines busy; retry after the Retry-After seconds. fetch_error, the page could not be fetched; check the URL, then retry. timeout, the fetch did not finish in time; raise timeout. unavailable or capacity_spent; error.message says what to change.

POST /v1

Scrape a URL from a JSON envelope and stream the result.

The canonical call. One JSON object, at most 1 MB, whose keys are the parameter names of the GET plus method, body and content_type. Nothing of ours travels in the query string here, so a target URL is sent as it is with no encoding. The token is the Authorization header, and the body is not read until the token is verified. Every value is a string, except headers (a list of strings) and timeout, max_credits and max_age (a whole JSON number or a string). An unknown key is a 400, exactly as on the GET, and so is any query string on this route: a parameter left there would not be read. QUERY /v1 is accepted with this exact request body, on the same code path: same parameters, same errors, same Idempotency-Key, same price, only the method word changes. It is the RFC 10008 standards path (Proposed Standard, June 2026) for a request that carries a body and is safe and idempotent. It is described here rather than as its own operation because the query path item field arrived in OpenAPI 3.2 and this document is 3.1. POST stays the example either way: browsers are still evaluating QUERY and an older WAF or proxy may refuse a method it does not know.

Error codes action_failed, bad_extract, bad_parameter, bad_token, blocked, body_too_large, busy, capacity_spent, convert_cap, daily_cost_cap, document_unreadable, fetch_error, forbidden_target, free_failures_spent, free_tier_cap, idempotency_mismatch, insufficient_credits, internal, llm_global_stop, llm_token_cap, llm_unavailable, max_credits_too_low, over_concurrency, period_cost_cap, replayed, spend_cap, target_error, target_rate_limited, timeout, unavailable, unknown_host, unsupported_content, unsupported_media_type, upstream_cap, wrong_scope.

Parameters

name in required type or values what it is
Idempotency-Key header no string Your own id for this call, 1 to 128 printable ASCII characters, scoped to your account. The first call with it runs and is charged. Any later call with the same key, inside 24 hours, answers 409 replayed, charges nothing, and carries the first call’s X-Request-Id. Anything we refund releases the key - a 402, a 429, and every 5xx of ours - so retrying the same request with the same key is safe. A delivered result keeps it, blocked pages included. Send the header once: two Idempotency-Key headers are a 400 naming it. After 24 hours a key is forgotten and the same key is a new, charged request. Without the header nothing changes.

Body

field required type or values what it is
url yes string Absolute http(s) URL to scrape, sent as it is. Nothing of ours is in the query string on this route, so the target’s own query needs no encoding.
method no GET, POST; default GET The method sent to the target. POST is the one that may carry body and content_type.
body no string The target request body, base64. Only with method POST.
content_type no string Media type of body, for example application/json. Only with method POST.
engine no http, browser; default http Which engine fetches the page. Leave it out and we pick. http sends one plain request. browser renders the page in Chrome on a datacenter exit, GET only.
country_code no AD, AE, AF, AG, AI, AL, AM, AO, AQ, AR, AS, AT, AU, AW, AX, AZ, BA, BB, BD, BE, BF, BG, BH, BI, BJ, BL, BM, BN, BO, BQ, BR, BS, BT, BV, BW, BY, BZ, CA, CC, CD, CF, CG, CH, CI, CK, CL, CM, CN, CO, CR, CU, CV, CW, CX, CY, CZ, DE, DJ, DK, DM, DO, DZ, EC, EE, EG, EH, ER, ES, ET, FI, FJ, FK, FM, FO, FR, GA, GB, GD, GE, GF, GG, GH, GI, GL, GM, GN, GP, GQ, GR, GS, GT, GU, GW, GY, HK, HM, HN, HR, HT, HU, ID, IE, IL, IM, IN, IO, IQ, IR, IS, IT, JE, JM, JO, JP, KE, KG, KH, KI, KM, KN, KP, KR, KW, KY, KZ, LA, LB, LC, LI, LK, LR, LS, LT, LU, LV, LY, MA, MC, MD, ME, MF, MG, MH, MK, ML, MM, MN, MO, MP, MQ, MR, MS, MT, MU, MV, MW, MX, MY, MZ, NA, NC, NE, NF, NG, NI, NL, NO, NP, NR, NU, NZ, OM, PA, PE, PF, PG, PH, PK, PL, PM, PN, PR, PS, PT, PW, PY, QA, RE, RO, RS, RU, RW, SA, SB, SC, SD, SE, SG, SH, SI, SJ, SK, SL, SM, SN, SO, SR, SS, ST, SV, SX, SY, SZ, TC, TD, TF, TG, TH, TJ, TK, TL, TM, TN, TO, TR, TT, TV, TW, TZ, UA, UG, UM, US, UY, UZ, VA, VC, VE, VG, VI, VN, VU, WF, WS, YE, YT, ZA, ZM, ZW Exit country, ISO 3166-1 alpha-2. Default is the platform’s configured country.
proxy_city no string Exit city in the proxy provider’s spelling, for example new_york.
proxy_tier no datacenter, residential, mobile; default residential Which proxy pool the request leaves from. Cheaper pools are blocked more often.
identity no firefox, chrome Which browser the fetch looks like on the wire. Leave it out and we pick.
session_id no string Send the same value again to leave from the same IP. 1 to 64 letters, digits, - or _. Your sessions are yours alone.
wait_until no string; default load Same rule as the query parameter.
response_types no string; default html Comma-separated list of html, json, markdown, text, links, metadata, page_data, compact_html, inspect. Several means multipart/mixed, one part per type, in request order.
actions no string Same rule as the query parameter. The recommended way to send type, so the text is not in a URL.
screenshot no viewport, full Same rule as the query parameter. A JPEG link in a part named screenshot.
network no boolean Same rule as the query parameter. The page’s XHR and fetch responses in a part named network.
network_url_globs no string Same rule as the query parameter.
only_main_content no true, false; default true Same rule as the query parameter. false returns the whole page and needs markdown, text or compact_html in response_types.
extract no string Selector rules, required with response_types=json and only valid with it. : joined by |, at most 32 rules and 4096 bytes. Add @attr to read an attribute instead of the text, and a trailing * to get a list. The answer is bounded: 8 MB in total, 1000 matches for one list rule, 1 MB for one value. A rule that reaches a bound stops there and the object gains truncated: true. That is a delivered result and is charged, so truncated is the one name a rule cannot have. If the cut leaves every field empty and json is the only output, the answer is a free 422 convert_cap on field extract. Example: title:h1|links:a@href*
schema no string Same rule as the query parameter, as a JSON string.
prompt no string Same rule as the query parameter.
extract_model no fast, default, non_china Same rule as the query parameter.
headers no array of string Extra request headers, each as name:value. Host, Connection, Content-Length, Transfer-Encoding, TE, Trailer, Upgrade, Expect, Keep-Alive, Via, HTTP2-Settings and any Proxy- header are set for you; sending one is a 400.
timeout no one of several; default 60 Seconds to wait for the result, 1 to 90.
max_credits no one of several The most credits one page may cost, 1 to 32767. Routes priced above it are left out; if none is left, a free 400 max_credits_too_low. A page whose bytes cost more is a 200 cut to fit, with X-Truncated max_credits.
max_age no one of several Serve a copy of the page fetched at most this many seconds ago, 0 to 900, 300 when left out (a hit costs the cache price). 0 always fetches fresh.

Answers

status what it means
200 Result delivered. The body is the requested type, or multipart/mixed for several.
400 Bad parameter. error.field names it and error.message lists the allowed values.
401 Bad or missing token, or a token in the query. Nothing is charged.
402 Not enough credits.
403 wrong_scope, an account key (meg_ak_). Scraping needs a scrape key (meg_sk_). Nothing is charged.
409 This Idempotency-Key was already used. Nothing is charged, and X-Request-Id names the first call.
413 The JSON envelope was larger than 1 MB. Nothing is charged.
415 The request body was not application/json. Nothing is charged.
422 convert_cap. The page is too large or too complex to convert into the response_types you asked for (markdown, text, links, metadata, page_data, compact_html, inspect or json). Nothing is charged. Ask for html, or for fewer response_types. upstream_cap. The page pulled more data than one request may use. Credits refunded. Fetch a smaller page. action_failed. An entry in actions could not run; error.field is actions and the message names the entry’s index. Nothing is charged, unless X-Billing is failure_cap; then X-Credits-Charged and its three parts say what the answer cost. Or blocked. The site blocked us; a free answer carries no page content, only this error. X-Billing is blocked and X-Untried names a route left to try. Nothing is charged, unless X-Billing is failure_cap. target_error. The target answered 5xx; X-Target-Status has its status. Nothing is charged.
429 Over the plan’s concurrency limit. Nothing is charged. Retry after the Retry-After seconds.
500 Our bug. Credits refunded.
503 Credits refunded. error.code says why. busy, all engines busy; retry after the Retry-After seconds. fetch_error, the page could not be fetched; check the URL, then retry. timeout, the fetch did not finish in time; raise timeout. unavailable or capacity_spent; error.message says what to change.

GET /v1/tier

The price tier of one host, and what each route to it costs.

Free, and writes no audit row. It shares your plan’s concurrency limit with /v1. A host with no tier in effect yet answers tier, temporary, price and pending as null.

Error codes bad_parameter, bad_token, busy, internal, over_concurrency, wrong_scope.

Parameters

name in required type or values what it is
host query yes string One hostname, like shop.example.com. Letters, digits, hyphens and dots, each label 1 to 63 characters. No wildcard, underscore, IPv6 literal, scheme, port or path. A trailing dot is dropped. www. is dropped too.
country_code query no AD, AE, AF, AG, AI, AL, AM, AO, AQ, AR, AS, AT, AU, AW, AX, AZ, BA, BB, BD, BE, BF, BG, BH, BI, BJ, BL, BM, BN, BO, BQ, BR, BS, BT, BV, BW, BY, BZ, CA, CC, CD, CF, CG, CH, CI, CK, CL, CM, CN, CO, CR, CU, CV, CW, CX, CY, CZ, DE, DJ, DK, DM, DO, DZ, EC, EE, EG, EH, ER, ES, ET, FI, FJ, FK, FM, FO, FR, GA, GB, GD, GE, GF, GG, GH, GI, GL, GM, GN, GP, GQ, GR, GS, GT, GU, GW, GY, HK, HM, HN, HR, HT, HU, ID, IE, IL, IM, IN, IO, IQ, IR, IS, IT, JE, JM, JO, JP, KE, KG, KH, KI, KM, KN, KP, KR, KW, KY, KZ, LA, LB, LC, LI, LK, LR, LS, LT, LU, LV, LY, MA, MC, MD, ME, MF, MG, MH, MK, ML, MM, MN, MO, MP, MQ, MR, MS, MT, MU, MV, MW, MX, MY, MZ, NA, NC, NE, NF, NG, NI, NL, NO, NP, NR, NU, NZ, OM, PA, PE, PF, PG, PH, PK, PL, PM, PN, PR, PS, PT, PW, PY, QA, RE, RO, RS, RU, RW, SA, SB, SC, SD, SE, SG, SH, SI, SJ, SK, SL, SM, SN, SO, SR, SS, ST, SV, SX, SY, SZ, TC, TD, TF, TG, TH, TJ, TK, TL, TM, TN, TO, TR, TT, TV, TW, TZ, UA, UG, UM, US, UY, UZ, VA, VC, VE, VG, VI, VN, VU, WF, WS, YE, YT, ZA, ZM, ZW Optional. Quote a request that exits from this country, read the way /v1 reads it. A premium country adds its price.
response_types query no array of html, json, markdown, text, links, metadata, page_data, compact_html, inspect Optional. Quote a request that asks for these types, read the way /v1 reads them. A paid type adds its price.

Answers

status what it means
200 The host’s tier.
400 Bad parameter. error.field names it.
401 Bad or missing token, or a token in the query.
403 wrong_scope, an account key (meg_ak_). Scraping needs a scrape key (meg_sk_).
429 Over the plan’s concurrency limit. Retry after the Retry-After seconds.
500 Our bug.
503 busy. Our credit store did not answer. Retry after the Retry-After seconds.

GET /v1/geos

The countries each route can exit from.

Public and free: no key, no audit row. Each route lists the countries its supplier serves, from our configuration, so a country_code it does not serve answers 400 on country_code when that route is forced with engine or proxy_tier. With engine and proxy_tier left out, we pick only routes that serve the country, and a country no route serves answers 400 on country_code. A premium country adds its price.

No error codes.

Answers

status what it means
200 The served countries.

POST /v1/map

List a site’s URLs without fetching their content.

Reads the site’s robots.txt, the sitemaps it names (or /sitemap.xml), at most 10 sitemap files, gzipped ones included, and the same-host links on the start page: at most 12 fetches. The start page uses the route the router picks; robots.txt and the sitemaps use HTTP on that route’s proxy tier. Each URL says where it came from. truncated is true when the limit, the route’s byte cap or your balance stopped the map early. A map costs 4 credits with 384 KiB of wire on HTTP datacenter; above that, and on a residential or browser route, the route’s normal byte and rung prices apply to every byte of every fetch. A map where we found no URL and the start page was blocked, failed or an error page (4xx or 5xx) is free; a search that matches nothing is still charged. The URLs go to you only and are never stored.

Error codes bad_parameter, bad_token, blocked, body_too_large, busy, capacity_spent, daily_cost_cap, fetch_error, forbidden_target, free_failures_spent, free_tier_cap, idempotency_mismatch, insufficient_credits, internal, over_concurrency, period_cost_cap, replayed, spend_cap, target_rate_limited, timeout, unavailable, unknown_host, unsupported_media_type, upstream_cap, wrong_scope.

Parameters

name in required type or values what it is
Idempotency-Key header no string Your own id for this call, 1 to 128 printable ASCII characters, scoped to your account. The first call with it runs and is charged. Any later call with the same key, inside 24 hours, answers 409 replayed, charges nothing, and carries the first call’s X-Request-Id. Anything we refund releases the key - a 402, a 429, and every 5xx of ours - so retrying the same request with the same key is safe. A delivered result keeps it, blocked pages included. Send the header once: two Idempotency-Key headers are a 400 naming it. After 24 hours a key is forgotten and the same key is a new, charged request. Without the header nothing changes.

Body

field required type or values what it is
url yes string The page to map from, http or https. Its host is the site.
limit no integer, 1 to 5000; default 500 The most URLs to return.
search no string Keep only URLs that contain this text.
include_subdomains no boolean; default false Also keep URLs on subdomains of the site’s host.

Answers

status what it means
200 The site’s URLs.
400 Bad parameter. error.field names it.
401 Bad or missing token, or a token in the query. Nothing is charged.
402 Not enough credits for the cheapest route’s map. Nothing is charged.
403 wrong_scope (an account key), or a target we never fetch. Nothing is charged.
409 This Idempotency-Key was already used. Nothing is charged, and X-Request-Id names the first call.
413 The JSON body was larger than 1 MB. Nothing is charged.
429 over_concurrency, over the plan’s concurrency limit, or target_rate_limited, the site asked us to slow down. Nothing is charged. Retry after the Retry-After seconds.
500 Our bug. Nothing is charged.
503 Nothing is charged. unavailable or capacity_spent, no route can reach this site right now; error.message says what to change. busy, our credit store did not answer; retry after the Retry-After seconds.

GET /v1/stats/domain

How the whole platform does on one host.

Free. The numbers are the whole platform’s, per host: shop.example.com and example.com are two answers, and www. is dropped. They are rolled up every hour, at 20 minutes past, so a fetch shows within about 80 minutes, today included. Only fetches that reached the site count: refusals before the fetch (balance, limits, capacity) are left out. A breakdown line is shown only when at least 3 customers and at least 20 requests are behind it in the window, and requests counts only the lines shown; with no line left the answer is requests 0 and an empty breakdown. Customers are counted per day, so a line whose customers came on different days can stay hidden. The answer is cached for five minutes.

Error codes bad_parameter, bad_token, internal, wrong_scope.

Parameters

name in required type or values what it is
host query yes string One hostname, like shop.example.com. Letters, digits, hyphens and dots, each label 1 to 63 characters. No wildcard, underscore, IPv6 literal, scheme, port or path. A trailing dot is dropped. The answer is for that host; www. is dropped.
days query no 1, 7, 30; default 7 Whole days back from yesterday. 1, 7 or 30.

Answers

status what it means
200 The host’s numbers.
400 Bad parameter. error.field names it.
401 Bad or missing token, or a token in the query.
403 wrong_scope, an account key (meg_ak_). Scraping needs a scrape key (meg_sk_).
500 Our bug.