# LangChain

> Load pages and crawls as LangChain documents, and give a LangChain agent scrape, map and batch tools.

**TL;DR**: [install](#install) `langchain-megalania`, then
`MegalaniaLoader(urls).load()` for documents and `megalania_tools()` for an
agent. The key comes from `MEG_API_KEY`. Blocked pages and errors are free.

## Install

Python 3.10 or newer.

```sh
pip install langchain-megalania
```

## Documents

`MegalaniaLoader` reads each URL as Markdown. Each document's metadata has
`source`, `request_id`, `credits_charged` and `target_status`.

```python
from langchain_megalania import MegalaniaLoader

docs = MegalaniaLoader(["https://docs.megalania.io/"]).load()
print(docs[0].metadata["credits_charged"], docs[0].page_content[:200])
```

Pass any key of the [JSON envelope](/guides/fetch) as a keyword, for example
`country_code="DE"` or `response_types="text"`.

A blocked page raises `MegalaniaError` with `code` `blocked`. With
`continue_on_failure=True` the loader skips it, logs the code and the request
id, and goes on.

### A whole site

`crawl` runs a [batch](/guides/batches) from one start page and waits for it. Each
document is one page the crawl reached, with `source`, `depth` and
`target_status`.

```python
from langchain_megalania import MegalaniaLoader

docs = MegalaniaLoader("https://docs.megalania.io/", crawl={"max_pages": 20, "max_depth": 2}).load()
print(len(docs), [d.metadata["source"] for d in docs][:5])
```

## Tools for an agent

`megalania_tools()` returns five tools: `scrape`, `extract`, `map_site`,
`create_batch` and `get_batch`. Their inputs use the same names as the API.

```python
from langchain_megalania import megalania_tools

scrape = megalania_tools()[0]
print(scrape.invoke({"url": "https://docs.megalania.io/"})[:300])
```

Give them to any LangChain agent:

```python
from langchain.agents import create_agent
from langchain_megalania import megalania_tools

agent = create_agent("openai:gpt-5", tools=megalania_tools())
agent.invoke({"messages": [{"role": "user", "content": "What can the API at https://docs.megalania.io/ do?"}]})
```

**Each page costs at most 128 credits** by default. That covers every route, and
a residential Chrome page up to about 1.5 MiB. A bigger page comes back cut,
with a note that says so. Pass `megalania_tools(max_credits=...)` (up to
32767) to allow more. `max_credits` is per page, so a batch can cost up to
`max_credits` times its pages: **one `create_batch` call takes at most 100
pages** by default. More URLs are refused, and a crawl stops at 100 pages.
Pass `megalania_tools(max_pages=...)` to allow more. The agent can raise
neither. `max_chars` (default 100,000) cuts what a tool returns, so one page
cannot fill the context.

## LangChain.js

There is no separate JavaScript package. Connect our [MCP server](/guides/mcp) with
`@langchain/mcp-adapters`:

```js
import { MultiServerMCPClient } from "@langchain/mcp-adapters";

const client = new MultiServerMCPClient({
  megalania: { url: "https://mcp.megalania.io/", headers: { Authorization: `Bearer ${process.env.MEG_API_KEY}` } },
});
const tools = await client.getTools();
```

The code is MIT licensed at https://github.com/megalania/megalania-python.