# Map a site

> POST /v1/map lists a site's URLs from its sitemaps and its start page, without fetching their content.

**TL;DR**: send a start URL, get the site's URLs back in one call. Then pick
the pages you want and fetch them with `POST /v1` or a [batch](/guides/batches).
A map costs 4 credits on most sites. A map that finds no URL is free when the
start page was blocked, failed or an error page (a 4xx or 5xx, such as a 404).
A search that matches nothing on a page we read is still charged.

```sh
curl -s -X POST "https://get.megalania.io/v1/map" \
  -H "Authorization: Bearer $MEG_API_KEY" \
  -H 'Content-Type: application/json' \
  -d '{"url": "https://lab.megalania.io/", "limit": 500}'
```

```json
{
  "urls": [
    { "url": "https://lab.megalania.io/p/1", "source": "sitemap" },
    { "url": "https://lab.megalania.io/article", "source": "link" }
  ],
  "truncated": false
}
```

Try it on the Lab: a map of `https://lab.megalania.io/` lists every Lab page
from its sitemap, then the links on its home page.

## What it reads

1. The start page, on the route our router picks for the site. The links on
   it that stay on the site become `source: link`.
2. `/robots.txt`, for the sitemaps it names. With none named, `/sitemap.xml`.
3. Those sitemaps, and the sitemaps a sitemap index lists, gzipped ones
   included: at most 10 files. Their URLs are `source: sitemap`.

That is at most 12 fetches. `robots.txt` and the sitemaps are text, so they
use plain HTTP on the proxy tier the first fetch picked, never the browser.
The list
has sitemap URLs first, then start-page links, each URL once, without its
`#fragment`. The map reads sitemaps only on the site's own host (and its
subdomains when you ask for them), never anywhere else. `/robots.txt` and the
sitemap files are part of the map price and are never in the list.

## The body

| key | value | default |
| -- | -- | -- |
| `url` | the page to map from, `http` or `https`. Its host is the site. | required |
| `limit` | the most URLs to return, 1 to 5000 | 500 |
| `search` | keep only URLs that contain this text, 1 to 256 characters | none |
| `include_subdomains` | also keep URLs on subdomains of the site's host | `false` |

Any other key is a 400 `bad_parameter` naming it. Send an `Idempotency-Key`
header to make a retry safe, as on `/v1`.

## `truncated`

`true` means there may be more URLs than the list holds. Three things stop a
map early: `limit` URLs were found, the route's byte cap was reached, or your
balance could not pay for more. A map with `truncated: true` still answers
every URL it found. When a usage limit stopped it, such as today's free
browser and residential retries, the answer also carries
`"stopped": { "code": "free_failures_spent", "message": "..." }`: the code is
the 429 a new request would get.

## What it costs

| Map | Credits |
| -- | -- |
| HTTP datacenter, up to 384 KiB of wire | 4 |
| HTTP datacenter, above 384 KiB | 4, plus 1 for each started 128 KiB above |
| On a residential or browser route | 4, plus that route's normal page and byte prices |
| No URL found, and the start page was blocked, failed or an error page (4xx, 5xx) | 0 |

Every byte of every fetch counts, including fetches that failed and routes
the router tried first. `X-Credits-Charged` says what the map cost. Most maps
read well under 384 KiB.

## Errors

The same codes as `/v1`; [Error codes](/reference/errors) has them all. The ones you
may see here: `bad_parameter` (400), `bad_token` (401),
`insufficient_credits` (402: not enough for the cheapest route's map),
`forbidden_target` (403), `replayed` (409), `body_too_large` (413), a 429,
and `unavailable` (503). None of them is charged.

## Privacy

The URL list goes to you only. We do not store it and do not log it; the
audit row keeps the host, as for any request.