openapi: 3.1.0 info: title: Megalania API version: "1" description: > One endpoint, three ways in. POST /v1 with a JSON object is the canonical one and the one the SDKs generate from; GET /v1 with query parameters is the paste-a-URL one; QUERY /v1 takes the POST envelope unchanged and is the RFC 10008 standards path, described under POST because OpenAPI 3.1 has no query path item. Every answer carries Cache-Control: no-store. Megalania fetches the target behind a geo-targeted proxy and streams the result back. Parameter names are the proto field names, except url and country_code. The envelope adds method, body and content_type, which exist only there. The token is the Authorization header, on every route. A request that carries api_key in the query is a 401 and is never charged: a key in a URL is written down by every log the URL passes. security: - bearer: [] paths: /v1: get: operationId: scrape summary: Scrape a URL and stream the result description: > Synchronous. The connection stays open until the result streams back or the timeout hits. Nothing is buffered; the size cap is 50 MB. A target that answers with a status under 500 is a delivered result: HTTP 200 from us, the target status in X-Target-Status. Blocks we detect are free: a blocked result (401, 403, 407, 429, or a detected challenge) costs nothing and is a 422 blocked with X-Billing: blocked and no page content. A target 5xx is free too: a 422 target_error with the target status in X-Target-Status and no page content. Error codes action_failed, bad_extract, bad_parameter, bad_token, blocked, busy, capacity_spent, convert_cap, daily_cost_cap, document_unreadable, fetch_error, forbidden_target, free_failures_spent, free_tier_cap, idempotency_mismatch, insufficient_credits, internal, llm_global_stop, llm_token_cap, llm_unavailable, max_credits_too_low, over_concurrency, period_cost_cap, replayed, spend_cap, target_error, target_rate_limited, timeout, unavailable, unknown_host, unsupported_content, upstream_cap, wrong_scope. parameters: - name: url in: query required: true description: | Absolute http(s) URL to scrape. Paste it as it is with url as the last parameter: everything after url= that is not one of our parameter names belongs to the target. Percent-encode it if the target's own query uses one of our names or you want our parameters after it. schema: type: string format: uri - name: engine in: query description: Which engine fetches the page. Leave it out and we pick. http sends one plain request; a page that builds itself in JavaScript comes back as a shell, marked with X-Hint. browser renders the page in Chrome on a datacenter exit, GET only. schema: type: string enum: - http - browser default: http - name: country_code in: query description: Exit country, ISO 3166-1 alpha-2. Default is the platform's configured country. schema: type: string enum: - AD - AE - AF - AG - AI - AL - AM - AO - AQ - AR - AS - AT - AU - AW - AX - AZ - BA - BB - BD - BE - BF - BG - BH - BI - BJ - BL - BM - BN - BO - BQ - BR - BS - BT - BV - BW - BY - BZ - CA - CC - CD - CF - CG - CH - CI - CK - CL - CM - CN - CO - CR - CU - CV - CW - CX - CY - CZ - DE - DJ - DK - DM - DO - DZ - EC - EE - EG - EH - ER - ES - ET - FI - FJ - FK - FM - FO - FR - GA - GB - GD - GE - GF - GG - GH - GI - GL - GM - GN - GP - GQ - GR - GS - GT - GU - GW - GY - HK - HM - HN - HR - HT - HU - ID - IE - IL - IM - IN - IO - IQ - IR - IS - IT - JE - JM - JO - JP - KE - KG - KH - KI - KM - KN - KP - KR - KW - KY - KZ - LA - LB - LC - LI - LK - LR - LS - LT - LU - LV - LY - MA - MC - MD - ME - MF - MG - MH - MK - ML - MM - MN - MO - MP - MQ - MR - MS - MT - MU - MV - MW - MX - MY - MZ - NA - NC - NE - NF - NG - NI - NL - NO - NP - NR - NU - NZ - OM - PA - PE - PF - PG - PH - PK - PL - PM - PN - PR - PS - PT - PW - PY - QA - RE - RO - RS - RU - RW - SA - SB - SC - SD - SE - SG - SH - SI - SJ - SK - SL - SM - SN - SO - SR - SS - ST - SV - SX - SY - SZ - TC - TD - TF - TG - TH - TJ - TK - TL - TM - TN - TO - TR - TT - TV - TW - TZ - UA - UG - UM - US - UY - UZ - VA - VC - VE - VG - VI - VN - VU - WF - WS - YE - YT - ZA - ZM - ZW - name: proxy_city in: query description: Exit city in the proxy provider's spelling, for example new_york. schema: type: string maxLength: 128 - name: proxy_tier in: query description: Which proxy pool the request leaves from. Cheaper pools are blocked more often. schema: type: string enum: - datacenter - residential - mobile default: residential - name: identity in: query description: "Which browser the fetch looks like on the wire: its TLS hello, its HTTP/2 frames and its headers. Leave it out and we pick the identity that does best on this site. Both cost the same." schema: type: string enum: - firefox - chrome - name: session_id in: query description: Send the same value again to leave from the same IP, so a login and the pages behind it look like one visitor. 1 to 64 letters, digits, - or _. Your sessions are yours alone; the same value from another customer is a different exit. We keep the cookies the session collects on our side for at most 10 minutes and send them with the next request that names it; you never see them. A session_id with no proxy_tier is pinned to residential, which can raise the price. Changing country_code, proxy_city or proxy_tier gives you a new exit and a new, empty cookie jar. schema: type: string pattern: ^[A-Za-z0-9_-]{1,64}$ - name: wait_until in: query description: "When the page counts as loaded. load (default), domcontentloaded, networkidle, or selector: of at most 1024 bytes, which waits for that element. Any value but load uses the browser and its price. Example: selector:#price" schema: type: string default: load maxLength: 1033 - name: response_types in: query description: Comma-separated. One type means the body is that type. Several means multipart/mixed, one part per type, in request order. style: form explode: false schema: type: array items: type: string enum: - html - json - markdown - text - links - metadata - page_data - compact_html - inspect default: - html - name: actions in: query description: "Steps the browser runs after the page loads and wait_until is met, in order, then it returns the page. kind:argument entries joined by |, at most 20: wait: (1 to 10000), wait_for: (up to 10 s for a visible match), click: (the first visible match), type:= (1 to 1024 bytes), press: (Enter, Tab, Escape, Backspace, Space, ArrowUp, ArrowDown, ArrowLeft, ArrowRight, PageUp, PageDown, Home, End), scroll:. A selector is at most 1024 bytes and cannot hold |. All actions together get at most 30 s inside timeout. It needs the browser, so leaving engine out sends the request to the browser, and engine=http is a 400. Adds 3 credits to the page. If an action cannot run, the answer is a free 422 action_failed whose message names the entry's index. Send type by POST, so the text is not in a URL. Batches take actions too. Example: click:.more|wait_for:.item-2" schema: type: string - name: screenshot in: query description: "A JPEG of the page as the browser rendered it: viewport (the visible window) or full (the whole page, at most 16,384 px tall). It needs the browser, so leaving engine out sends the request to the browser, and engine=http is a 400. The answer is multipart/mixed with a part named screenshot: {url, expires_at, bytes, credits}, a link that works for 1 hour and downloads as screenshot--.jpg; the image is deleted after 7 days. X-Credits-Charged is the page alone; the part credits is what the screenshot adds (1 viewport, 2 full), charged only when the link is delivered. If it could not be made, the part is {error: screenshot_failed, credits: 0}. A blocked page charges nothing. Batches take screenshot too: the stored result keeps bytes and credits, and each item read signs a fresh 1-hour screenshot_url." schema: type: string enum: - viewport - full - name: network in: query description: "true also returns the JSON and text the page itself loaded with XHR or fetch, as a part named network in application/x-ndjson, one JSON object per response: {url, method, status, content_type, bytes, truncated, body}. A body that is not UTF-8 text is base64, with encoding: base64. bytes is the body's size; when the body was too large to keep (over 2 MiB) the body is empty, truncated is true, and bytes is its encoded size on the wire. At most 200 responses, 1 MiB per body (then truncated is true) and 8 MiB in all; X-Network-Truncated says when the list was cut. Only the page's own XHR and fetch are captured, not workers or service workers. No request or response headers, no cookies and no request bodies. It needs the browser, so leaving engine out sends the request to the browser, and engine=http is a 400. Adds no credits; the page's bytes are already metered. Batches take network too." schema: type: string enum: - "true" - "false" - name: network_url_globs in: query description: "With network=true, capture only the responses whose full URL matches one of these patterns, of any content type. A comma list of at most 10 patterns, 1024 bytes in all; * matches anything, ? one character. Without it, network captures JSON, NDJSON and text responses. Example: */api/*" schema: type: string - name: only_main_content in: query description: true returns only the page's main content in markdown, text and compact_html. false returns the whole page and needs markdown, text or compact_html in response_types. Any other value is a 400. schema: type: string enum: - "true" - "false" default: "true" - name: extract in: query description: "Selector rules, required with response_types=json and only valid with it. : joined by |, at most 32 rules and 4096 bytes. Add @attr to read an attribute instead of the text, and a trailing * to get a list. The answer is bounded: 8 MB in total, 1000 matches for one list rule, 1 MB for one value. A rule that reaches a bound stops there and the object gains truncated: true. That is a delivered result and is charged, so truncated is the one name a rule cannot have. If the cut leaves every field empty and json is the only output, the answer is a free 422 convert_cap on field extract. Example: title:h1|links:a@href*" schema: type: string - name: schema in: query description: "A JSON Schema object (at most 8192 bytes) with type object at the top and additionalProperties false on every object; it may use type, properties, required, items, enum, const, anyOf, format, description and title. With response_types=json and no extract, a language model fills the schema from the page's main content. The json part is then {data, credits, llm_credits, tokens}: data matches the schema; credits equals X-Credits-Charged, the whole request (page plus extraction); llm_credits is the extraction's part (2 plus a rate per started 1,000 weighted tokens, input + 5 x output); tokens is that weighted count. The page's main content, the schema and the prompt go to the model provider: Anthropic in the US. An answer that does not match the schema is a 422 bad_extract: the extraction is free, the page is still charged, and the body carries the page as markdown." schema: type: string - name: prompt in: query description: Extra instructions for the model, at most 2048 bytes. Needs schema. schema: type: string - name: extract_model in: query description: "fast, default or non_china. An alias, never a vendor: we never use a model of China origin, whatever its host. Today all three aliases run claude-haiku-4-5. Needs schema." schema: type: string enum: - fast - default - non_china default: default - name: headers in: query description: Extra request header as name:value. Repeat the parameter for more than one. Host, Connection, Content-Length, Transfer-Encoding, TE, Trailer, Upgrade, Expect, Keep-Alive, Via, HTTP2-Settings and any Proxy- header are set for you; sending one is a 400. style: form explode: true schema: type: array items: type: string pattern: ^[^:]+:.+$ - name: timeout in: query description: Seconds to wait for the result. schema: type: integer minimum: 1 maximum: 90 default: 60 - name: max_credits in: query description: The most credits one page may cost, 1 to 32767. Routes priced above it are left out of the climb and the page's byte cap shrinks to fit it. If no route is left, the answer is a free 400 max_credits_too_low naming the lowest price. A page whose bytes cost more than it is a 200 cut at the bytes it pays for, with X-Truncated max_credits. schema: type: integer minimum: 1 maximum: 32767 - name: max_age in: query description: Serve a copy of this page fetched at most this many seconds ago, 0 to 900, 300 when you leave it out. A copy costs the cache price from the price list and says X-Cache hit. With no copy that new, we fetch the page and keep a copy for a short time. 0 always fetches fresh. A request with method, body, headers, actions, screenshot, network, session_id or a user name in the URL never uses the cache; sending max_age with one of them is a 400. schema: type: integer minimum: 0 maximum: 900 default: 300 - $ref: "#/components/parameters/IdempotencyKey" responses: "200": $ref: "#/components/responses/Scraped" "400": description: Bad parameter. error.field names it and error.message lists the allowed values. $ref: "#/components/responses/Error" "401": description: Bad or missing token, or a token in the query. Nothing is charged. $ref: "#/components/responses/Error" "402": description: Not enough credits. $ref: "#/components/responses/Error" "403": description: wrong_scope, an account key (meg_ak_). Scraping needs a scrape key (meg_sk_). Nothing is charged. $ref: "#/components/responses/Error" "409": description: This Idempotency-Key was already used. Nothing is charged, and X-Request-Id names the first call. $ref: "#/components/responses/Error" "422": description: convert_cap. The page is too large or too complex to convert into the response_types you asked for (markdown, text, links, metadata, page_data, compact_html, inspect or json). Nothing is charged. Ask for html, or for fewer response_types. upstream_cap. The page pulled more data than one request may use. Credits refunded. Fetch a smaller page. action_failed. An entry in actions could not run; error.field is actions and the message names the entry's index. Nothing is charged, unless X-Billing is failure_cap; then X-Credits-Charged and its three parts say what the answer cost. Or blocked. The site blocked us; a free answer carries no page content, only this error. X-Billing is blocked and X-Untried names a route left to try. Nothing is charged, unless X-Billing is failure_cap. target_error. The target answered 5xx; X-Target-Status has its status. Nothing is charged. $ref: "#/components/responses/Refused" "429": description: Over the plan's concurrency limit. Nothing is charged. Retry after the Retry-After seconds. headers: Retry-After: schema: type: integer X-Request-Id: $ref: "#/components/headers/X-Request-Id" content: application/json: schema: $ref: "#/components/schemas/Error" "500": description: Our bug. Credits refunded. $ref: "#/components/responses/Error" "503": description: Credits refunded. error.code says why. busy, all engines busy; retry after the Retry-After seconds. fetch_error, the page could not be fetched; check the URL, then retry. timeout, the fetch did not finish in time; raise timeout. unavailable or capacity_spent; error.message says what to change. headers: Retry-After: schema: type: integer X-Request-Id: $ref: "#/components/headers/X-Request-Id" content: application/json: schema: $ref: "#/components/schemas/Error" post: operationId: scrapeJson summary: Scrape a URL from a JSON envelope and stream the result description: > The canonical call. One JSON object, at most 1 MB, whose keys are the parameter names of the GET plus method, body and content_type. Nothing of ours travels in the query string here, so a target URL is sent as it is with no encoding. The token is the Authorization header, and the body is not read until the token is verified. Every value is a string, except headers (a list of strings) and timeout, max_credits and max_age (a whole JSON number or a string). An unknown key is a 400, exactly as on the GET, and so is any query string on this route: a parameter left there would not be read. QUERY /v1 is accepted with this exact request body, on the same code path: same parameters, same errors, same Idempotency-Key, same price, only the method word changes. It is the RFC 10008 standards path (Proposed Standard, June 2026) for a request that carries a body and is safe and idempotent. It is described here rather than as its own operation because the query path item field arrived in OpenAPI 3.2 and this document is 3.1. POST stays the example either way: browsers are still evaluating QUERY and an older WAF or proxy may refuse a method it does not know. Error codes action_failed, bad_extract, bad_parameter, bad_token, blocked, body_too_large, busy, capacity_spent, convert_cap, daily_cost_cap, document_unreadable, fetch_error, forbidden_target, free_failures_spent, free_tier_cap, idempotency_mismatch, insufficient_credits, internal, llm_global_stop, llm_token_cap, llm_unavailable, max_credits_too_low, over_concurrency, period_cost_cap, replayed, spend_cap, target_error, target_rate_limited, timeout, unavailable, unknown_host, unsupported_content, unsupported_media_type, upstream_cap, wrong_scope. parameters: - $ref: "#/components/parameters/IdempotencyKey" requestBody: required: true content: application/json: schema: $ref: "#/components/schemas/ScrapeEnvelope" responses: "200": $ref: "#/components/responses/Scraped" "400": description: Bad parameter. error.field names it and error.message lists the allowed values. $ref: "#/components/responses/Error" "401": description: Bad or missing token, or a token in the query. Nothing is charged. $ref: "#/components/responses/Error" "402": description: Not enough credits. $ref: "#/components/responses/Error" "403": description: wrong_scope, an account key (meg_ak_). Scraping needs a scrape key (meg_sk_). Nothing is charged. $ref: "#/components/responses/Error" "409": description: This Idempotency-Key was already used. Nothing is charged, and X-Request-Id names the first call. $ref: "#/components/responses/Error" "413": description: The JSON envelope was larger than 1 MB. Nothing is charged. $ref: "#/components/responses/Error" "415": description: The request body was not application/json. Nothing is charged. $ref: "#/components/responses/Error" "422": description: convert_cap. The page is too large or too complex to convert into the response_types you asked for (markdown, text, links, metadata, page_data, compact_html, inspect or json). Nothing is charged. Ask for html, or for fewer response_types. upstream_cap. The page pulled more data than one request may use. Credits refunded. Fetch a smaller page. action_failed. An entry in actions could not run; error.field is actions and the message names the entry's index. Nothing is charged, unless X-Billing is failure_cap; then X-Credits-Charged and its three parts say what the answer cost. Or blocked. The site blocked us; a free answer carries no page content, only this error. X-Billing is blocked and X-Untried names a route left to try. Nothing is charged, unless X-Billing is failure_cap. target_error. The target answered 5xx; X-Target-Status has its status. Nothing is charged. $ref: "#/components/responses/Refused" "429": description: Over the plan's concurrency limit. Nothing is charged. Retry after the Retry-After seconds. headers: Retry-After: schema: type: integer X-Request-Id: $ref: "#/components/headers/X-Request-Id" content: application/json: schema: $ref: "#/components/schemas/Error" "500": description: Our bug. Credits refunded. $ref: "#/components/responses/Error" "503": description: Credits refunded. error.code says why. busy, all engines busy; retry after the Retry-After seconds. fetch_error, the page could not be fetched; check the URL, then retry. timeout, the fetch did not finish in time; raise timeout. unavailable or capacity_spent; error.message says what to change. headers: Retry-After: schema: type: integer X-Request-Id: $ref: "#/components/headers/X-Request-Id" content: application/json: schema: $ref: "#/components/schemas/Error" /v1/tier: get: operationId: getTier summary: The price tier of one host, and what each route to it costs description: > Free, and writes no audit row. It shares your plan's concurrency limit with /v1. A host with no tier in effect yet answers tier, temporary, price and pending as null. Error codes bad_parameter, bad_token, busy, internal, over_concurrency, wrong_scope. parameters: - $ref: "#/components/parameters/TierHost" - $ref: "#/components/parameters/TierCountry" - $ref: "#/components/parameters/TierResponseTypes" responses: "200": description: The host's tier. headers: X-Request-Id: $ref: "#/components/headers/X-Request-Id" Cache-Control: $ref: "#/components/headers/Cache-Control" content: application/json: schema: $ref: "#/components/schemas/Tier" "400": description: Bad parameter. error.field names it. $ref: "#/components/responses/Error" "401": description: Bad or missing token, or a token in the query. $ref: "#/components/responses/Error" "403": description: wrong_scope, an account key (meg_ak_). Scraping needs a scrape key (meg_sk_). $ref: "#/components/responses/Error" "429": description: Over the plan's concurrency limit. Retry after the Retry-After seconds. headers: Retry-After: schema: type: integer X-Request-Id: $ref: "#/components/headers/X-Request-Id" content: application/json: schema: $ref: "#/components/schemas/Error" "500": description: Our bug. $ref: "#/components/responses/Error" "503": description: busy. Our credit store did not answer. Retry after the Retry-After seconds. headers: Retry-After: schema: type: integer X-Request-Id: $ref: "#/components/headers/X-Request-Id" content: application/json: schema: $ref: "#/components/schemas/Error" /v1/geos: get: operationId: getGeos summary: The countries each route can exit from security: [] description: | Public and free: no key, no audit row. Each route lists the countries its supplier serves, from our configuration, so a country_code it does not serve answers 400 on country_code when that route is forced with engine or proxy_tier. With engine and proxy_tier left out, we pick only routes that serve the country, and a country no route serves answers 400 on country_code. A premium country adds its price. No error codes. responses: "200": description: The served countries. headers: Cache-Control: schema: type: string X-Request-Id: $ref: "#/components/headers/X-Request-Id" content: application/json: schema: $ref: "#/components/schemas/Geos" /v1/map: post: operationId: mapSite summary: List a site's URLs without fetching their content description: > Reads the site's robots.txt, the sitemaps it names (or /sitemap.xml), at most 10 sitemap files, gzipped ones included, and the same-host links on the start page: at most 12 fetches. The start page uses the route the router picks; robots.txt and the sitemaps use HTTP on that route's proxy tier. Each URL says where it came from. truncated is true when the limit, the route's byte cap or your balance stopped the map early. A map costs 4 credits with 384 KiB of wire on HTTP datacenter; above that, and on a residential or browser route, the route's normal byte and rung prices apply to every byte of every fetch. A map where we found no URL and the start page was blocked, failed or an error page (4xx or 5xx) is free; a search that matches nothing is still charged. The URLs go to you only and are never stored. Error codes bad_parameter, bad_token, blocked, body_too_large, busy, capacity_spent, daily_cost_cap, fetch_error, forbidden_target, free_failures_spent, free_tier_cap, idempotency_mismatch, insufficient_credits, internal, over_concurrency, period_cost_cap, replayed, spend_cap, target_rate_limited, timeout, unavailable, unknown_host, unsupported_media_type, upstream_cap, wrong_scope. parameters: - $ref: "#/components/parameters/IdempotencyKey" requestBody: required: true content: application/json: schema: $ref: "#/components/schemas/MapRequest" responses: "200": description: The site's URLs. headers: X-Request-Id: $ref: "#/components/headers/X-Request-Id" X-Credits-Charged: description: Credits this map cost, 0 when the map found no URL and its start page was blocked, failed or an error page (4xx or 5xx). schema: type: integer X-Credits-Route: description: The part of X-Credits-Charged that paid to reach the page, the engine, the proxy tier, a premium country and any data over the included amount. Sent with X-Credits-Charged; the three X-Credits parts always add up to it. schema: type: integer minimum: 0 X-Credits-Output: description: The part of X-Credits-Charged that paid for what we made of the page, the response types and a schema extraction. The screenshot is not in it, as it is not in X-Credits-Charged. schema: type: integer minimum: 0 X-Credits-Other: description: The part of X-Credits-Charged that paid for the rest, actions, and on a monitor's page the monitor check. schema: type: integer minimum: 0 content: application/json: schema: $ref: "#/components/schemas/MapResult" "400": description: Bad parameter. error.field names it. $ref: "#/components/responses/Error" "401": description: Bad or missing token, or a token in the query. Nothing is charged. $ref: "#/components/responses/Error" "402": description: Not enough credits for the cheapest route's map. Nothing is charged. $ref: "#/components/responses/Error" "403": description: wrong_scope (an account key), or a target we never fetch. Nothing is charged. $ref: "#/components/responses/Error" "409": description: This Idempotency-Key was already used. Nothing is charged, and X-Request-Id names the first call. $ref: "#/components/responses/Error" "413": description: The JSON body was larger than 1 MB. Nothing is charged. $ref: "#/components/responses/Error" "429": description: over_concurrency, over the plan's concurrency limit, or target_rate_limited, the site asked us to slow down. Nothing is charged. Retry after the Retry-After seconds. headers: Retry-After: schema: type: integer X-Request-Id: $ref: "#/components/headers/X-Request-Id" content: application/json: schema: $ref: "#/components/schemas/Error" "500": description: Our bug. Nothing is charged. $ref: "#/components/responses/Error" "503": description: Nothing is charged. unavailable or capacity_spent, no route can reach this site right now; error.message says what to change. busy, our credit store did not answer; retry after the Retry-After seconds. headers: Retry-After: schema: type: integer X-Request-Id: $ref: "#/components/headers/X-Request-Id" content: application/json: schema: $ref: "#/components/schemas/Error" /v1/stats/domain: get: operationId: getDomainStats summary: How the whole platform does on one host description: | Free. The numbers are the whole platform's, per host: shop.example.com and example.com are two answers, and www. is dropped. They are rolled up every hour, at 20 minutes past, so a fetch shows within about 80 minutes, today included. Only fetches that reached the site count: refusals before the fetch (balance, limits, capacity) are left out. A breakdown line is shown only when at least 3 customers and at least 20 requests are behind it in the window, and requests counts only the lines shown; with no line left the answer is requests 0 and an empty breakdown. Customers are counted per day, so a line whose customers came on different days can stay hidden. The answer is cached for five minutes. Error codes bad_parameter, bad_token, internal, wrong_scope. parameters: - $ref: "#/components/parameters/StatsHost" - $ref: "#/components/parameters/StatsDays" responses: "200": description: The host's numbers. headers: X-Request-Id: $ref: "#/components/headers/X-Request-Id" content: application/json: schema: $ref: "#/components/schemas/DomainStats" "400": description: Bad parameter. error.field names it. $ref: "#/components/responses/Error" "401": description: Bad or missing token, or a token in the query. $ref: "#/components/responses/Error" "403": description: wrong_scope, an account key (meg_ak_). Scraping needs a scrape key (meg_sk_). $ref: "#/components/responses/Error" "500": description: Our bug. $ref: "#/components/responses/Error" components: securitySchemes: bearer: type: http scheme: bearer description: A scrape key, meg_sk_ then 43 characters, shown once when it is made. headers: X-Request-Id: description: UUIDv7 of this request. Quote it in support requests. schema: type: string format: uuid Cache-Control: description: Always no-store. A scrape is live data and must never be served from a cache, on any method. schema: type: string enum: - no-store Content-Security-Policy: description: Always the same sandbox policy. The page is the target's, not ours, so a browser that opens it runs no script, keeps no origin, and loads nothing from the network; only inline styles and data images show. schema: type: string enum: - sandbox; default-src 'none'; img-src data:; style-src 'unsafe-inline' X-Content-Type-Options: description: Always nosniff. A browser reads the body only as its Content-Type says. schema: type: string enum: - nosniff Referrer-Policy: description: Always no-referrer. A link in the page tells the next site nothing about where it was opened. schema: type: string enum: - no-referrer parameters: TierHost: name: host in: query required: true description: One hostname, like shop.example.com. Letters, digits, hyphens and dots, each label 1 to 63 characters. No wildcard, underscore, IPv6 literal, scheme, port or path. A trailing dot is dropped. www. is dropped too. schema: type: string maxLength: 253 TierCountry: name: country_code in: query description: Optional. Quote a request that exits from this country, read the way /v1 reads it. A premium country adds its price. schema: type: string enum: - AD - AE - AF - AG - AI - AL - AM - AO - AQ - AR - AS - AT - AU - AW - AX - AZ - BA - BB - BD - BE - BF - BG - BH - BI - BJ - BL - BM - BN - BO - BQ - BR - BS - BT - BV - BW - BY - BZ - CA - CC - CD - CF - CG - CH - CI - CK - CL - CM - CN - CO - CR - CU - CV - CW - CX - CY - CZ - DE - DJ - DK - DM - DO - DZ - EC - EE - EG - EH - ER - ES - ET - FI - FJ - FK - FM - FO - FR - GA - GB - GD - GE - GF - GG - GH - GI - GL - GM - GN - GP - GQ - GR - GS - GT - GU - GW - GY - HK - HM - HN - HR - HT - HU - ID - IE - IL - IM - IN - IO - IQ - IR - IS - IT - JE - JM - JO - JP - KE - KG - KH - KI - KM - KN - KP - KR - KW - KY - KZ - LA - LB - LC - LI - LK - LR - LS - LT - LU - LV - LY - MA - MC - MD - ME - MF - MG - MH - MK - ML - MM - MN - MO - MP - MQ - MR - MS - MT - MU - MV - MW - MX - MY - MZ - NA - NC - NE - NF - NG - NI - NL - NO - NP - NR - NU - NZ - OM - PA - PE - PF - PG - PH - PK - PL - PM - PN - PR - PS - PT - PW - PY - QA - RE - RO - RS - RU - RW - SA - SB - SC - SD - SE - SG - SH - SI - SJ - SK - SL - SM - SN - SO - SR - SS - ST - SV - SX - SY - SZ - TC - TD - TF - TG - TH - TJ - TK - TL - TM - TN - TO - TR - TT - TV - TW - TZ - UA - UG - UM - US - UY - UZ - VA - VC - VE - VG - VI - VN - VU - WF - WS - YE - YT - ZA - ZM - ZW TierResponseTypes: name: response_types in: query description: Optional. Quote a request that asks for these types, read the way /v1 reads them. A paid type adds its price. style: form explode: false schema: type: array items: type: string enum: - html - json - markdown - text - links - metadata - page_data - compact_html - inspect StatsHost: name: host in: query required: true description: One hostname, like shop.example.com. Letters, digits, hyphens and dots, each label 1 to 63 characters. No wildcard, underscore, IPv6 literal, scheme, port or path. A trailing dot is dropped. The answer is for that host; www. is dropped. schema: type: string maxLength: 253 StatsDays: name: days in: query description: Whole days back from yesterday. 1, 7 or 30. schema: type: integer enum: - 1 - 7 - 30 default: 7 IdempotencyKey: name: Idempotency-Key in: header description: | Your own id for this call, 1 to 128 printable ASCII characters, scoped to your account. The first call with it runs and is charged. Any later call with the same key, inside 24 hours, answers 409 replayed, charges nothing, and carries the first call's X-Request-Id. Anything we refund releases the key - a 402, a 429, and every 5xx of ours - so retrying the same request with the same key is safe. A delivered result keeps it, blocked pages included. Send the header once: two Idempotency-Key headers are a 400 naming it. After 24 hours a key is forgotten and the same key is a new, charged request. Without the header nothing changes. schema: type: string minLength: 1 maxLength: 128 responses: Scraped: description: Result delivered. The body is the requested type, or multipart/mixed for several. headers: X-Request-Id: $ref: "#/components/headers/X-Request-Id" Cache-Control: $ref: "#/components/headers/Cache-Control" Content-Security-Policy: $ref: "#/components/headers/Content-Security-Policy" X-Content-Type-Options: $ref: "#/components/headers/X-Content-Type-Options" Referrer-Policy: $ref: "#/components/headers/Referrer-Policy" X-Credits-Charged: description: The charge if the body completes. A body we cut (an engine error mid-stream, so the multipart closing boundary is missing or the stream ends in an error) is refunded in full. Closing the connection after the headers keeps the charge. Once body bytes have gone out, reading so slowly that the body outlasts your timeout keeps the charge too. Usage is the final record. schema: type: integer X-Credits-Route: description: The part of X-Credits-Charged that paid to reach the page, the engine, the proxy tier, a premium country and any data over the included amount. Sent with X-Credits-Charged; the three X-Credits parts always add up to it. schema: type: integer minimum: 0 X-Credits-Output: description: The part of X-Credits-Charged that paid for what we made of the page, the response types and a schema extraction. The screenshot is not in it, as it is not in X-Credits-Charged. schema: type: integer minimum: 0 X-Credits-Other: description: The part of X-Credits-Charged that paid for the rest, actions, and on a monitor's page the monitor check. schema: type: integer minimum: 0 X-Engine: description: The engine that produced the result. schema: type: string enum: - http - browser X-Target-Status: description: HTTP status the target answered with. schema: type: integer X-Timing-Target-Ms: description: How long the target took to answer, in milliseconds. schema: type: integer X-Route: description: The engine and proxy tier that delivered the result. schema: type: string enum: - http/datacenter - http/residential - http/mobile - browser/datacenter - browser/residential - browser/mobile X-Geo: description: The country the request exited from, uppercase ISO 3166-1 alpha-2. country_code when sent, else the default country. schema: type: string X-Identity: description: The browser identity that fetched the page. schema: type: string enum: - firefox - chrome X-Billing: description: blocked when we detected a block. Blocks we detect are free and carry no page content. failure_cap on a blocked answer to a request with actions, when more than 30% of at least 20 of your requests with actions failed or were blocked in the last hour; that answer is charged like a delivered page. schema: type: string enum: - blocked - failure_cap X-Block: description: Present on a blocked result. The kind of block we detected. schema: type: string enum: - challenge - deny - rate_limited X-Challenge: description: Present when an anti-bot challenge was detected. The vendor name. schema: type: string enum: - cloudflare - datadome - perimeterx - akamai - kasada - other X-Vendor: description: The anti-bot vendor present on the page, even when it let us through. X-Challenge names the one that blocked us. schema: type: string enum: - cloudflare - datadome - perimeterx - akamai - kasada - other X-Hint: description: Present when it applies. needs_js, the page is a JavaScript shell; send engine=browser. browser_blocked, the site blocks our browser too, so we did not try it. schema: type: string enum: - needs_js - browser_blocked X-Untried: description: On a blocked or needs_js result, when a route is left. The rungs your key may use that can serve now and this request did not try (X-Hops lists the ones it tried), comma-separated, such as c:dc,c:res. Absent means every route we have was tried. schema: type: string X-Untried-Reason: description: Why X-Untried names routes we did not try. budget means residential routes are unavailable right now, so the climb skipped every residential route and tried the next open one. A skipped route is never charged. Absent when the climb simply stopped. schema: type: string enum: - budget X-Hops: description: Every hop we took, in order, as a JSON array, measured up to the moment these headers left. Each hop is {"rung", "identity", "ms", "outcome"}, such as [{"rung":"h:dc","identity":"chrome","ms":412,"outcome":"blocked"},{"rung":"c:res","identity":"chrome","ms":2900,"outcome":"ok"}]. outcome is ok, blocked, needs_js or the error code of a failed try, such as timeout. The batch item field hops and the request inspector use the same shape. schema: type: string X-Cache: description: Present when the request could use the cache. hit, this answer is a recent copy of the page; miss, we fetched the page now. schema: type: string enum: - hit - miss X-Ladder-Truncated: description: Present with value true when your max_credits or your balance removed some routes we would have tried. schema: type: string enum: - "true" X-Truncated: description: Present when the page reached the most bytes this request may pull, and names that limit. max_credits means your max_credits set it, and the charge stays at or under max_credits. byte_cap means the route's or your plan's byte cap, or what your balance pays for, set it. The body is the part that fit. schema: type: string enum: - max_credits - byte_cap X-Network-Truncated: description: Present with value true when network=true and the list of captured responses was cut by a cap (200 responses, 8 MiB, a URL over 8 KiB or a body we could not keep). schema: type: string enum: - "true" X-Compact-Truncated: description: Present with value true when compact_html reached its cap of about 512 KiB (closing tags can run a little over). The part ends at an element boundary, with every open tag closed. schema: type: string enum: - "true" X-Document: description: Present when the url answered with a PDF, DOCX, PPTX or XLSX file. Every response_type is then made from that file, and html is the file turned into HTML. schema: type: string enum: - pdf - docx - pptx - xlsx X-Document-Units: description: With X-Document. The count the file was priced on, rounded up by its row in the price table. Pages (pdf), slides (pptx), characters of text (docx) or non-empty cells (xlsx). schema: type: integer X-Document-Truncated: description: Present with value true when the file was over 500 pages (or 500 slides, 1,500,000 characters, 500,000 cells). The first part was converted and priced. schema: type: string enum: - "true" X-Usage-Warning: description: On the first 200 after your grant passes 80% of its usage limit, once per grant. At 100% requests answer 429 until a new grant. schema: type: string content: text/html: schema: description: "html: the page as fetched. compact_html: the same page without scripts, styles, comments, svg, iframes, hidden elements and most attributes, whitespace collapsed, about 512 KiB at most. It keeps the structure, the text, the title and the attributes selectors need: id, class, href and src (absolute), alt, title, name, type, value, datetime, lang, role, aria-label, itemprop, itemtype, data-testid and data-test. With only_main_content=true (the default) it is the main content only." type: string text/markdown: schema: description: The page, or the PDF, DOCX, PPTX or XLSX file, converted to Markdown. type: string text/plain: schema: description: response_types=text. The same content as markdown, as plain text. type: string application/json: schema: description: With json, one object, one key per extract rule; a missing scalar is null and a missing list is empty. With links, every http(s) link on the page, absolute, once each, with its anchor text. With metadata, the requested URL, title, description, language, canonical, OpenGraph fields and every JSON-LD object on the page. With page_data, one object with json_ld, every JSON-LD object on the page, and state, every script of type application/json that has an id (such as __NEXT_DATA__), parsed and keyed by that id. With inspect, the page's structure as title, containers (blocks that repeat, each with its selector, count, a short sample, a ready extract rule and its field rules), headings, links, forms and data (the JSON scripts the page embeds, with their key paths and a @json rule). When the site answered 4xx, it adds note, which names the status. type: object additionalProperties: true multipart/mixed: schema: description: One part per requested response type, in request order, each with its own content type. type: string format: binary Refused: description: A refusal. Free, unless X-Billing is failure_cap. headers: X-Request-Id: $ref: "#/components/headers/X-Request-Id" Cache-Control: $ref: "#/components/headers/Cache-Control" X-Billing: description: blocked when we detected a block; it is free. failure_cap on an action_failed or blocked answer to a request with actions, above the failure cap; that answer is charged. schema: type: string enum: - blocked - failure_cap X-Credits-Charged: description: Sent only with X-Billing failure_cap. What this answer cost. schema: type: integer X-Credits-Route: description: The part of X-Credits-Charged that paid to reach the page, the engine, the proxy tier, a premium country and any data over the included amount. Sent with X-Credits-Charged; the three X-Credits parts always add up to it. schema: type: integer minimum: 0 X-Credits-Output: description: The part of X-Credits-Charged that paid for what we made of the page, the response types and a schema extraction. The screenshot is not in it, as it is not in X-Credits-Charged. schema: type: integer minimum: 0 X-Credits-Other: description: The part of X-Credits-Charged that paid for the rest, actions, and on a monitor's page the monitor check. schema: type: integer minimum: 0 content: application/json: schema: $ref: "#/components/schemas/Error" Error: description: Error. headers: X-Request-Id: $ref: "#/components/headers/X-Request-Id" Cache-Control: $ref: "#/components/headers/Cache-Control" content: application/json: schema: $ref: "#/components/schemas/Error" schemas: Error: type: object required: - error properties: error: type: object required: - code - message - report_url properties: code: type: string enum: - action_failed - already_member - already_on_plan - bad_extract - bad_origin - bad_parameter - bad_request - bad_session - bad_signature - bad_token - batch_running - blocked - body_too_large - busy - capacity_spent - checkout_closed - checkout_failed - checkout_open - convert_cap - daily_cost_cap - document_unreadable - fetch_error - forbidden_target - free_failures_spent - free_tier_cap - idempotency_mismatch - insufficient_credits - internal - last_owner - llm_global_stop - llm_token_cap - llm_unavailable - max_credits_month_too_low - max_credits_too_low - monitor_checks_exceeded - monitor_limit - monitor_paused - monitor_running - monitor_too_soon - no_subscription - not_eligible - not_enrolled - not_found - notice_changed - notice_required - not_subscribed - over_concurrency - over_plan - owner_only - past_due - payment_failed - period_cost_cap - rate_limited - recent_sign_in_required - replayed - spend_cap - target_error - target_rate_limited - timeout - unavailable - unsupported_content - unknown_host - unsupported_media_type - upstream_cap - wrong_scope message: type: string field: type: string description: The parameter to change. On 400, on 422 and on some 503s. report_url: type: string format: uri description: A dashboard page to tell us about this error. It holds only the request id, after a MapRequest: type: object required: - url additionalProperties: false properties: url: type: string description: The page to map from, http or https. Its host is the site. limit: type: integer minimum: 1 maximum: 5000 default: 500 description: The most URLs to return. search: type: string minLength: 1 maxLength: 256 description: Keep only URLs that contain this text. include_subdomains: type: boolean default: false description: Also keep URLs on subdomains of the site's host. MapResult: type: object required: - urls - truncated properties: urls: type: array items: type: object required: - url - source properties: url: type: string source: type: string enum: - sitemap - link truncated: type: boolean description: true when the limit, the byte cap or your balance stopped the map before every URL was read. stopped: type: object required: - code - message description: Present when a usage limit stopped the map before every URL was read, for example free_failures_spent. The code is the 429 a new request would get. properties: code: type: string message: type: string Geos: type: object required: - default_country - premium - routes properties: default_country: type: string description: The country a request without country_code exits from. premium: type: array description: Countries that add their price. items: type: string routes: type: array items: type: object required: - route - engine - proxy_tier - countries properties: route: type: string description: The route's name, like h:dc (HTTP, datacenter). engine: type: string enum: - http - browser proxy_tier: type: string enum: - datacenter - residential - mobile countries: type: array description: Uppercase ISO 3166-1 alpha-2 codes this route exits from. items: type: string Tier: type: object required: - host - tier - temporary - price - pending - quotes - capabilities properties: host: type: string tier: type: - integer - "null" minimum: 1 maximum: 5 description: The tier in effect today. temporary: type: - boolean - "null" description: true while the host has too few requests for a measured tier. A temporary tier can rise at once. price: type: - integer - "null" description: The quote for a typical page on this host, in credits, from its tier, before response types and premium geo. It is not the charge. Every request pays the price of the route that delivers the page, shown in quotes. pending: type: - object - "null" description: A scheduled change and the day it takes effect. required: - tier - effective_on - price properties: tier: type: integer effective_on: type: string format: date price: type: integer quotes: type: object description: What a request to this host with this proxy_tier reserves today, in credits, with the country_code and response_types you pass; with neither, before response types and premium geo. The same quote /v1 makes, so a max_credits of this value is never refused for price. Always present, also for a host with no tier. A proxy_tier we cannot serve right now is left out. browser is what an engine=browser page costs within its included bytes; the request reserves more, for overage up to the byte cap, and a smaller max_credits cuts the byte cap instead of refusing. It is left out while the browser cannot serve you. browser_residential is the same for engine=browser with proxy_tier=residential. With no country_code or response_types, each quote equals the base price of that route on the Plan page, on every host. The host's tier never changes what a route costs. properties: datacenter: type: integer residential: type: integer browser: type: integer browser_residential: type: integer mobile: type: integer additionalProperties: false capabilities: type: object description: The proxy tiers each engine can serve you on right now, from your key's rungs, our providers and running engines. An empty list means that engine cannot serve now. mobile appears only while a mobile provider serves. required: - http - browser properties: http: type: array items: type: string enum: - datacenter - residential - mobile browser: type: array items: type: string enum: - datacenter - residential - mobile additionalProperties: false DomainStats: type: object required: - host - days - requests - breakdown properties: host: type: string description: The host the answer is for, lowercase and without www. days: type: integer requests: type: integer breakdown: type: array items: type: object required: - engine - tier - requests - success_rate - mean_target_ms - block_share properties: engine: type: string tier: type: string requests: type: integer success_rate: type: number mean_target_ms: type: - number - "null" block_share: type: number ScrapeEnvelope: type: object required: - url description: The POST /v1 request body. Every value is a string, except headers, which is a list of strings, and timeout, max_credits and max_age, which take a whole JSON number or a string. An unknown key is a 400. properties: url: type: string format: uri description: Absolute http(s) URL to scrape, sent as it is. Nothing of ours is in the query string on this route, so the target's own query needs no encoding. method: type: string enum: - GET - POST default: GET description: The method sent to the target. POST is the one that may carry body and content_type. body: type: string contentEncoding: base64 description: The target request body, base64. Only with method POST. content_type: type: string maxLength: 128 description: Media type of body, for example application/json. Only with method POST. engine: type: string enum: - http - browser default: http description: Which engine fetches the page. Leave it out and we pick. http sends one plain request. browser renders the page in Chrome on a datacenter exit, GET only. country_code: type: string description: Exit country, ISO 3166-1 alpha-2. Default is the platform's configured country. enum: - AD - AE - AF - AG - AI - AL - AM - AO - AQ - AR - AS - AT - AU - AW - AX - AZ - BA - BB - BD - BE - BF - BG - BH - BI - BJ - BL - BM - BN - BO - BQ - BR - BS - BT - BV - BW - BY - BZ - CA - CC - CD - CF - CG - CH - CI - CK - CL - CM - CN - CO - CR - CU - CV - CW - CX - CY - CZ - DE - DJ - DK - DM - DO - DZ - EC - EE - EG - EH - ER - ES - ET - FI - FJ - FK - FM - FO - FR - GA - GB - GD - GE - GF - GG - GH - GI - GL - GM - GN - GP - GQ - GR - GS - GT - GU - GW - GY - HK - HM - HN - HR - HT - HU - ID - IE - IL - IM - IN - IO - IQ - IR - IS - IT - JE - JM - JO - JP - KE - KG - KH - KI - KM - KN - KP - KR - KW - KY - KZ - LA - LB - LC - LI - LK - LR - LS - LT - LU - LV - LY - MA - MC - MD - ME - MF - MG - MH - MK - ML - MM - MN - MO - MP - MQ - MR - MS - MT - MU - MV - MW - MX - MY - MZ - NA - NC - NE - NF - NG - NI - NL - NO - NP - NR - NU - NZ - OM - PA - PE - PF - PG - PH - PK - PL - PM - PN - PR - PS - PT - PW - PY - QA - RE - RO - RS - RU - RW - SA - SB - SC - SD - SE - SG - SH - SI - SJ - SK - SL - SM - SN - SO - SR - SS - ST - SV - SX - SY - SZ - TC - TD - TF - TG - TH - TJ - TK - TL - TM - TN - TO - TR - TT - TV - TW - TZ - UA - UG - UM - US - UY - UZ - VA - VC - VE - VG - VI - VN - VU - WF - WS - YE - YT - ZA - ZM - ZW proxy_city: type: string maxLength: 128 description: Exit city in the proxy provider's spelling, for example new_york. proxy_tier: type: string enum: - datacenter - residential - mobile default: residential description: Which proxy pool the request leaves from. Cheaper pools are blocked more often. identity: type: string enum: - firefox - chrome description: Which browser the fetch looks like on the wire. Leave it out and we pick. session_id: type: string pattern: ^[A-Za-z0-9_-]{1,64}$ description: Send the same value again to leave from the same IP. 1 to 64 letters, digits, - or _. Your sessions are yours alone. wait_until: type: string default: load description: Same rule as the query parameter. response_types: type: string default: html description: Comma-separated list of html, json, markdown, text, links, metadata, page_data, compact_html, inspect. Several means multipart/mixed, one part per type, in request order. actions: type: string description: Same rule as the query parameter. The recommended way to send type, so the text is not in a URL. screenshot: type: string enum: - viewport - full description: Same rule as the query parameter. A JPEG link in a part named screenshot. network: type: boolean description: Same rule as the query parameter. The page's XHR and fetch responses in a part named network. network_url_globs: type: string description: Same rule as the query parameter. only_main_content: type: string enum: - "true" - "false" default: "true" description: Same rule as the query parameter. false returns the whole page and needs markdown, text or compact_html in response_types. extract: type: string description: "Selector rules, required with response_types=json and only valid with it. : joined by |, at most 32 rules and 4096 bytes. Add @attr to read an attribute instead of the text, and a trailing * to get a list. The answer is bounded: 8 MB in total, 1000 matches for one list rule, 1 MB for one value. A rule that reaches a bound stops there and the object gains truncated: true. That is a delivered result and is charged, so truncated is the one name a rule cannot have. If the cut leaves every field empty and json is the only output, the answer is a free 422 convert_cap on field extract. Example: title:h1|links:a@href*" schema: type: string description: Same rule as the query parameter, as a JSON string. prompt: type: string maxLength: 2048 description: Same rule as the query parameter. extract_model: type: string enum: - fast - default - non_china description: Same rule as the query parameter. headers: type: array items: type: string pattern: ^[^:]+:.+$ description: Extra request headers, each as name:value. Host, Connection, Content-Length, Transfer-Encoding, TE, Trailer, Upgrade, Expect, Keep-Alive, Via, HTTP2-Settings and any Proxy- header are set for you; sending one is a 400. timeout: oneOf: - type: integer minimum: 1 maximum: 90 - type: string pattern: ^([1-9]|[1-8][0-9]|90)$ default: 60 description: Seconds to wait for the result, 1 to 90. max_credits: oneOf: - type: integer minimum: 1 maximum: 32767 - type: string pattern: ^([1-9][0-9]{0,3}|[12][0-9]{4}|3[01][0-9]{3}|32[0-6][0-9]{2}|327[0-5][0-9]|3276[0-7])$ description: The most credits one page may cost, 1 to 32767. Routes priced above it are left out; if none is left, a free 400 max_credits_too_low. A page whose bytes cost more is a 200 cut to fit, with X-Truncated max_credits. max_age: oneOf: - type: integer minimum: 0 maximum: 900 - type: string pattern: ^([0-9]|[1-9][0-9]|[1-8][0-9]{2}|900)$ description: Serve a copy of the page fetched at most this many seconds ago, 0 to 900, 300 when left out (a hit costs the cache price). 0 always fetches fresh.