> For clean Markdown of any page on these docs, append `.md` to the page URL > Full documentation index: https://you.com/docs/llms.txt > Search over these docs using a free, read-only MCP server (it is the best way to pick the right API(s) for the job and understand how to use them): https://you.com/docs/_mcp/server > Access web search, web fetch, and deep research tools using our product MCP server: https://api.you.com/mcp (use `/mcp/research` and `/mcp/finance` paths for Research and Finance Research respectively) > Call live Web Search for free: append `?profile=free` to the MCP server path > OpenAPI spec index: https://you.com/docs/openapi.json > How to authenticate: https://you.com/docs/using-the-api/authentication > Changelog: https://you.com/docs/changelog > Full onboarding instructions for agents: https://you.com/SKILL.md # Extract API > Web Search, Answer, Extract, Research, and Finance Research APIs—plus the Inference Router for open-weight models grounded in web search—that give your AI app real-time web intelligence. Start with free credits. ## Docs - [Extract API Quickstart](https://you.com/docs/guides/extract/quickstart.md): The Extract API fetches any webpage or PDF and parses it into clean Markdown or HTML, ready for an LLM. > **Note:** This page contains both a page directory (above) and the landing page content (below). The page directory is generated for agent use and does not appear on the landing page. > For clean Markdown of any page on these docs, append `.md` to the page URL > Full documentation index: https://you.com/docs/llms.txt > Search over these docs using a free, read-only MCP server (it is the best way to pick the right API(s) for the job and understand how to use them): https://you.com/docs/_mcp/server > Access web search, web fetch, and deep research tools using our product MCP server: https://api.you.com/mcp (use `/mcp/research` and `/mcp/finance` paths for Research and Finance Research respectively) > Call live Web Search for free: append `?profile=free` to the MCP server path > OpenAPI spec index: https://you.com/docs/openapi.json > How to authenticate: https://you.com/docs/using-the-api/authentication > Changelog: https://you.com/docs/changelog > Full onboarding instructions for agents: https://you.com/SKILL.md # Extract API Overview > **Tip** > > **Connect to You.com via MCP** `https://api.you.com/mcp`. > > Use Web Search for free `https://api.you.com/mcp?profile=free`. [MCP Server guide →](/docs/agents/mcp-server) > **Install the Docs MCP server** > > The Docs MCP server lets your agent search over the entirety of these docs. Free to use. > > Install `https://you.com/docs/_mcp/server`. See the [Docs MCP Server guide](/docs/agents/docs-mcp-server) to learn more. --- ## What Is the Extract API? The Extract API fetches any webpage or PDF and parses it into clean Markdown or HTML. Use it when you already have the URL and your agent needs the full document, such as a filing, a contract, or a research paper, instead of a search highlight. PDF tables come back as Markdown or HTML tables with their rows and columns intact. By default (`parse_effort: "standard"`), complex PDF pages, such as charts and images, also go through an advanced model. The Extract API replaces the [Contents API](/docs/api-reference/contents), which still works but will not receive new features. [Migrate From the Contents API](#migrate-from-the-contents-api) maps each Contents parameter to its Extract equivalent. --- ## Endpoints Each API has its own endpoint. Every request sends your API key in the `X-API-Key` header, except Inference Router requests, which send it as `Authorization: Bearer`. | API | Endpoint | Guide | API reference | | -------------------- | ------------------------------------------------------------------------------------------------------------------------------------ | ----------------------------------------- | ----------------------------------------------------------------------------------- | | Web Search API | `POST https://ydc-index.io/v1/search` (also `GET`) | [Overview](/docs/guides/search) | [/v1/search](/docs/api-reference/search/v1-search) | | Inference Router API | `POST https://api.you.com/v1/chat/completions` (OpenAI-compatible) and `POST https://api.you.com/v1/messages` (Anthropic-compatible) | [Overview](/docs/guides/inference-router) | [/v1/chat/completions](/docs/api-reference/inference-router/create-chat-completion) | | Extract API | `POST https://ydc-index.io/v1/extract` | [Overview](/docs/guides/extract) | [/v1/extract](/docs/api-reference/extract) | | Answer API | `POST https://api.you.com/v1/answer` | [Overview](/docs/guides/answer) | [/v1/answer](/docs/api-reference/answer/v1-answer) | | Research API | `POST https://api.you.com/v1/research` | [Overview](/docs/guides/research) | [/v1/research](/docs/api-reference/research/v1-research) | | Finance Research API | `POST https://api.you.com/v1/finance_research` | [Overview](/docs/guides/finance-research) | [/v1/finance_research](/docs/api-reference/finance-research/v1-finance_research) | The Inference Router, Answer, Research, and Finance Research APIs run only on `api.you.com`. Sending one of those paths to `ydc-index.io` returns `403` with `{"message": "Missing Authentication Token"}`, even when the key is valid. --- ## How It's Different From Full Page Extraction The Extract API and full page extraction in the Web Search API both return full page content, but they start from different inputs: | | Extract API | Web Search API + full page extraction | | -------------------- | ---------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------- | | **Starting point** | You already have the URL | You have a query, not URLs | | **Use case** | Read a known page or PDF on demand | Enrich search results with full content | | **URL source** | You provide it | You.com search discovers it | | **URLs per request** | One | Up to 100 results per search | | **Pricing** | \$0.90 per 1,000 URLs. For PDF URLs, `parse_effort: "standard"` (the default) adds an additional \$0.50 per 1,000 pages in the PDF | \$5.00 per 1,000 searches + \$1.00 per 1,000 pages fetched live | Use the Extract API when you know which URL to read. Use `extraction_mode: "full_page"` when you want full content returned alongside search results in one call. See [Page Content Extraction](/docs/guides/search/retrieve-page-content). --- ## What You Get Each request returns one JSON object. The example below answers a request with `full_page.extraction_formats: ["markdown", "html"]`: ```json { "url": "https://en.wikipedia.org/wiki/Retrieval-augmented_generation", "extraction_source": "blend", "title": "Retrieval-augmented generation - Wikipedia", "contents": { "full_page": { "markdown": "# Retrieval-augmented generation\n\n**Retrieval-augmented generation** (**RAG**) is a technique...", "html": "
Retrieval-augmented generation (RAG) is a technique...
" } }, "metadata": { "site_name": "Wikipedia", "favicon_url": "https://api.ydc-index.io/favicon?domain=en.wikipedia.org&size=128" } } ``` `contents.full_page` holds one field for each format you request in `full_page.extraction_formats`. When nothing could be extracted, the response is an empty object, so check for `contents` before you read it. --- ## Key Features ### Webpages and PDFs from one endpoint Send a webpage URL or a PDF URL to the same endpoint and read the result from the same `contents.full_page` fields. PDF tables keep their rows and columns in both Markdown and HTML output. ### PDF parse effort `parse_effort` sets how much work goes into each PDF page and has no effect on webpages: * `lite` extracts the content embedded in each page and adds no page charge, so a PDF costs the same \$0.90 per 1,000 URLs as a webpage. Use it for digitally created PDFs, such as a text-based contract. * `standard` (the default) also sends complex pages, such as charts and images, through an advanced model and adds an additional \$0.50 per 1,000 pages in the PDF. It returns the most complete output, so use it for PDFs in general and always when figures carry the information, such as an earnings report with revenue charts. ### Cache control `extraction_source` decides where the content comes from: | Value | Behavior | | ----------------- | ------------------------------------------------------------------------------------------------ | | `blend` (default) | Serves cached content when it exists and crawls the page live when it does not | | `cache` | Returns cached content only. Fastest, and `contents` is omitted when the page has no cached copy | | `fetch` | Always crawls the page live. Freshest content, at the cost of higher latency | ### End-to-end timeout `timeout` caps the whole request, including the cache lookup and the crawl, at 2–100 seconds (default 30). A request that runs past it returns `504` with `code` `timeout`. --- ## Parameters | Parameter | Type | Required | Description | | ------------------------------ | ---------------- | -------- | ------------------------------------------------------------------------------------------------ | | `url` | string | Yes | The webpage or PDF URL to extract | | `full_page.extraction_formats` | array of strings | No | `markdown`, `html`, or both. Default `["markdown"]`. An empty array is treated as `["markdown"]` | | `extraction_source` | string | No | `blend` (default), `cache`, or `fetch` | | `timeout` | integer | No | Seconds the request may take end to end, from 2 to 100. Default 30 | | `parse_effort` | string | No | PDF pages only: `lite` or `standard` (default) | [View full API reference](/docs/api-reference/extract) --- ## Errors Every `422`, `500`, `502`, `503`, and `504` response has the same body: an `error` object with a stable `code`, a human-readable `message`, a `retryable` flag, and a `request_id` to send to support. Branch on `code` and `retryable`, not on `message`. A `402` usually carries the same body, but the API gateway can also answer with its own, so branch on the `402` status itself. | Status | `code` | What to do | | ------ | --------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------- | | `422` | `invalid_request` | Fix the fields listed in `details.fields`. Unknown fields, including the Contents API's `crawl_timeout`, are rejected | | `422` | `invalid_url` | The URL can never be fetched. `details.reason` is `malformed`, `unsupported_scheme`, `private_address`, or `unresolvable` | | `422` | `pdf_too_large` | The PDF needs more time than `timeout` allows. Raise `timeout`, up to 100 | | `502` | `site_blocked`, `site_unreachable`, `unsupported_content` | The page itself can't be extracted, and retrying fails until it changes. `details.reason` has the cause | | `504` | `timeout` | Below the 100-second maximum, retry with a higher `timeout` | | `503` | `service_unavailable` | Retry after the `Retry-After` header | | `500` | `internal_error` | Retry only when `retryable` is `true` | | `402` | `payment_required` | Add credits at [you.com/platform](https://you.com/platform). `details.upgrade_url` carries the link when the body has it | `401` and `403` responses cover a missing, invalid, or unscoped API key. See [API Troubleshooting](/docs/support/troubleshooting). --- ## Common Use Cases ### Competitive intelligence Monitor competitor pricing, feature, or blog pages. Extract each page on a schedule, feed the Markdown to an LLM, and surface what changed. ```python from youdotcom import You competitor_pages = [ "https://competitor-a.com/pricing", "https://competitor-b.com/pricing", "https://competitor-c.com/features", ] with You() as you: for url in competitor_pages: page = you.extract(url=url, extraction_source="fetch") if page.contents: print(f"\n--- {page.title} ---") # Feed the Markdown into your LLM for summarization or diff print(page.contents.full_page.markdown[:500]) ``` `extraction_source: "fetch"` skips the cache so each run sees the live page. ### Document ingestion Turn filings, whitepapers, and documentation into Markdown for a vector store. PDFs and webpages come back in the same shape. ```python from youdotcom import You source_urls = [ "https://docs.example.com/api-reference", "https://example.com/reports/annual-report-2025.pdf", ] documents = [] with You() as you: for url in source_urls: page = you.extract(url=url, timeout=60) if page.contents: documents.append({ "source": page.url, "title": page.title, "content": page.contents.full_page.markdown, }) # Index the document into your vector store here ``` A longer `timeout` gives large PDFs room to finish. ### Research assistant Let users ask questions about a specific URL. Extract the page on the fly and pass it to your LLM as context. ```python from youdotcom import You def fetch_url_context(you: You, url: str) -> str: page = you.extract(url=url) if not page.contents: return "" return page.contents.full_page.markdown or "" # User asks: "Summarize this page for me" with You() as you: context = fetch_url_context(you, "https://example.com/long-report") prompt = f"Summarize the following page content:\n\n{context}" # Pass prompt to your LLM ``` --- ## Best Practices ### Request only the formats you need Markdown is the default and the right choice for LLM prompts. Add `html` only when your code needs the markup. ### Pick the cache behavior per job Use the default `blend` for most reads, `cache` when latency matters more than freshness, and `fetch` when you need the page as it is right now. ### Match `parse_effort` to the PDF Keep the default `standard` for the most complete extraction from any PDF. Switch to `lite` for text-based PDFs when skipping the per-page charge matters more than the extra coverage. ### Read many URLs with concurrent requests The Extract API takes one URL per request. To read a list of URLs, send the requests concurrently and stay within your [rate limits](/docs/rate-limits). ### Retry only what can succeed Check `retryable` before you retry. A `502` for a blocked site or an `invalid_url` fails again on every attempt, while `503`, `504`, and retryable `500` responses can succeed on a later try. --- ## Migrate From the Contents API | Contents API | Extract API | | -------------------------------- | ------------------------------------------------------------------------------------------- | | `POST /v1/contents` | `POST /v1/extract` | | `urls`, up to 10 per request | `url`, one per request. Send several requests concurrently for a list | | `formats: ["markdown", "html"]` | `full_page.extraction_formats: ["markdown", "html"]` | | `formats: ["metadata"]` | Not needed. `metadata` is part of the response | | `crawl_timeout` (per URL, 1–60s) | `timeout` (whole request, 2–100s). `crawl_timeout` is rejected with `422` | | `max_age` | `extraction_source`. Use `fetch` to bypass the cache or `cache` to read only cached content | | Response is an array of pages | Response is one object | | `page.markdown`, `page.html` | `contents.full_page.markdown`, `contents.full_page.html` | --- ## Pricing The Extract API costs \$0.90 per 1,000 URLs. For PDF URLs, `parse_effort: "standard"` (the default) adds an additional charge for each page in the PDF: \$0.50 per 1,000 pages. `lite` adds no page charge, and `parse_effort` has no effect on webpages. A 350-page PDF at `standard` adds \$0.175. See the [Billing page](/docs/administration/billing#extract-api) for complete pricing details. For volume discounts, annual pricing, or enterprise features, visit [you.com/pricing](https://you.com/pricing) or contact [api@you.com](mailto:api@you.com). --- ## Next Steps #### [Quickstart](/docs/guides/extract/quickstart) Make your first Extract API request #### [API Reference](/docs/api-reference/extract) Full parameter reference, request and response schemas, and error codes #### [Web Search API Overview](/docs/guides/search) Find URLs with search, then extract the pages you need #### [Page Content Extraction](/docs/guides/search/retrieve-page-content) Get full page content alongside search results in one call