Skip to navigation

Extract API Overview

Fetch any webpage or PDF and parse it into clean Markdown or HTML with one request.
View as MarkdownOpen in Claude

Add You.com to your agent or IDE via MCP—connect at https://api.you.com/mcp. Try search with no signup at https://api.you.com/mcp?profile=free (you-search and you-discover). Research and finance use /mcp/research and /mcp/finance, or ?tools= on /mcp. MCP Server guide →

Install the Docs MCP server

The Docs MCP server lets your agent search over the entirety of these docs. Free to use.

Install https://you.com/docs/_mcp/server. See the Docs MCP Server guide to learn more.


What Is the Extract API?

The Extract API fetches any webpage or PDF and parses it into clean Markdown or HTML. Use it when you already have the URL and your agent needs the full document, such as a filing, a contract, or a research paper, instead of a search highlight.

PDF tables come back as Markdown or HTML tables with their rows and columns intact. By default (parse_effort: "standard"), complex PDF pages, such as charts and images, also go through an advanced model.

The Extract API replaces the Contents API, which still works but will not receive new features. Migrate From the Contents API maps each Contents parameter to its Extract equivalent.


Endpoints

Each API has its own endpoint. Every request sends your API key in the X-API-Key header, except Inference Router requests, which send it as Authorization: Bearer.

APIEndpointGuideAPI reference
Web Search APIPOST https://ydc-index.io/v1/search (also GET)Overview/v1/search
Inference Router APIPOST https://api.you.com/v1/chat/completions (OpenAI-compatible) and POST https://api.you.com/v1/messages (Anthropic-compatible)Overview/v1/chat/completions
Extract APIPOST https://ydc-index.io/v1/extractOverview/v1/extract
Answer APIPOST https://api.you.com/v1/answerOverview/v1/answer
Research APIPOST https://api.you.com/v1/researchOverview/v1/research
Finance Research APIPOST https://api.you.com/v1/finance_researchOverview/v1/finance_research

The Inference Router, Answer, Research, and Finance Research APIs run only on api.you.com. Sending one of those paths to ydc-index.io returns 403 with {"message": "Missing Authentication Token"}, even when the key is valid.


How It’s Different From Full Page Extraction

The Extract API and full page extraction in the Web Search API both return full page content, but they start from different inputs:

Extract APIWeb Search API + full page extraction
Starting pointYou already have the URLYou have a query, not URLs
Use caseRead a known page or PDF on demandEnrich search results with full content
URL sourceYou provide itYou.com search discovers it
URLs per requestOneUp to 100 results per search
Pricing$0.90 per 1,000 URLs. For PDF URLs, parse_effort: "standard" (the default) adds an additional $0.50 per 1,000 pages in the PDF$5.00 per 1,000 searches + $1.00 per 1,000 pages fetched live

Use the Extract API when you know which URL to read. Use extraction_mode: "full_page" when you want full content returned alongside search results in one call. See Page Content Extraction.


What You Get

Each request returns one JSON object. The example below answers a request with full_page.extraction_formats: ["markdown", "html"]:

{
"url": "https://en.wikipedia.org/wiki/Retrieval-augmented_generation",
"extraction_source": "blend",
"title": "Retrieval-augmented generation - Wikipedia",
"contents": {
"full_page": {
"markdown": "# Retrieval-augmented generation\n\n**Retrieval-augmented generation** (**RAG**) is a technique...",
"html": "<h1>Retrieval-augmented generation</h1>\n<p><b>Retrieval-augmented generation</b> (<b>RAG</b>) is a technique...</p>"
}
},
"metadata": {
"site_name": "Wikipedia",
"favicon_url": "https://api.ydc-index.io/favicon?domain=en.wikipedia.org&size=128"
}
}

contents.full_page holds one field for each format you request in full_page.extraction_formats. When nothing could be extracted, the response is an empty object, so check for contents before you read it.


Key Features

Webpages and PDFs from one endpoint

Send a webpage URL or a PDF URL to the same endpoint and read the result from the same contents.full_page fields. PDF tables keep their rows and columns in both Markdown and HTML output.

PDF parse effort

parse_effort sets how much work goes into each PDF page and has no effect on webpages:

  • lite extracts the content embedded in each page and adds no page charge, so a PDF costs the same $0.90 per 1,000 URLs as a webpage. Use it for digitally created PDFs, such as a text-based contract.
  • standard (the default) also sends complex pages, such as charts and images, through an advanced model and adds an additional $0.50 per 1,000 pages in the PDF. It returns the most complete output, so use it for PDFs in general and always when figures carry the information, such as an earnings report with revenue charts.

Cache control

extraction_source decides where the content comes from:

ValueBehavior
blend (default)Serves cached content when it exists and crawls the page live when it does not
cacheReturns cached content only. Fastest, and contents is omitted when the page has no cached copy
fetchAlways crawls the page live. Freshest content, at the cost of higher latency

End-to-end timeout

timeout caps the whole request, including the cache lookup and the crawl, at 2–100 seconds (default 30). A request that runs past it returns 504 with code timeout.


Parameters

ParameterTypeRequiredDescription
urlstringYesThe webpage or PDF URL to extract
full_page.extraction_formatsarray of stringsNomarkdown, html, or both. Default ["markdown"]. An empty array is treated as ["markdown"]
extraction_sourcestringNoblend (default), cache, or fetch
timeoutintegerNoSeconds the request may take end to end, from 2 to 100. Default 30
parse_effortstringNoPDF pages only: lite or standard (default)

View full API reference


Errors

Every 422, 500, 502, 503, and 504 response has the same body: an error object with a stable code, a human-readable message, a retryable flag, and a request_id to send to support. Branch on code and retryable, not on message. A 402 usually carries the same body, but the API gateway can also answer with its own, so branch on the 402 status itself.

StatuscodeWhat to do
422invalid_requestFix the fields listed in details.fields. Unknown fields, including the Contents API’s crawl_timeout, are rejected
422invalid_urlThe URL can never be fetched. details.reason is malformed, unsupported_scheme, private_address, or unresolvable
422pdf_too_largeThe PDF needs more time than timeout allows. Raise timeout, up to 100
502site_blocked, site_unreachable, unsupported_contentThe page itself can’t be extracted, and retrying fails until it changes. details.reason has the cause
504timeoutBelow the 100-second maximum, retry with a higher timeout
503service_unavailableRetry after the Retry-After header
500internal_errorRetry only when retryable is true
402payment_requiredAdd credits at you.com/platform. details.upgrade_url carries the link when the body has it

401 and 403 responses cover a missing, invalid, or unscoped API key. See API Troubleshooting.


Common Use Cases

Competitive intelligence

Monitor competitor pricing, feature, or blog pages. Extract each page on a schedule, feed the Markdown to an LLM, and surface what changed.

from youdotcom import You
competitor_pages = [
"https://competitor-a.com/pricing",
"https://competitor-b.com/pricing",
"https://competitor-c.com/features",
]
with You() as you:
for url in competitor_pages:
page = you.extract(url=url, extraction_source="fetch")
if page.contents:
print(f"\n--- {page.title} ---")
# Feed the Markdown into your LLM for summarization or diff
print(page.contents.full_page.markdown[:500])

extraction_source: "fetch" skips the cache so each run sees the live page.

Document ingestion

Turn filings, whitepapers, and documentation into Markdown for a vector store. PDFs and webpages come back in the same shape.

from youdotcom import You
source_urls = [
"https://docs.example.com/api-reference",
"https://example.com/reports/annual-report-2025.pdf",
]
documents = []
with You() as you:
for url in source_urls:
page = you.extract(url=url, timeout=60)
if page.contents:
documents.append({
"source": page.url,
"title": page.title,
"content": page.contents.full_page.markdown,
})
# Index the document into your vector store here

A longer timeout gives large PDFs room to finish.

Research assistant

Let users ask questions about a specific URL. Extract the page on the fly and pass it to your LLM as context.

from youdotcom import You
def fetch_url_context(you: You, url: str) -> str:
page = you.extract(url=url)
if not page.contents:
return ""
return page.contents.full_page.markdown or ""
# User asks: "Summarize this page for me"
with You() as you:
context = fetch_url_context(you, "https://example.com/long-report")
prompt = f"Summarize the following page content:\n\n{context}"
# Pass prompt to your LLM

Best Practices

Request only the formats you need

Markdown is the default and the right choice for LLM prompts. Add html only when your code needs the markup.

Pick the cache behavior per job

Use the default blend for most reads, cache when latency matters more than freshness, and fetch when you need the page as it is right now.

Match parse_effort to the PDF

Keep the default standard for the most complete extraction from any PDF. Switch to lite for text-based PDFs when skipping the per-page charge matters more than the extra coverage.

Read many URLs with concurrent requests

The Extract API takes one URL per request. To read a list of URLs, send the requests concurrently and stay within your rate limits.

Retry only what can succeed

Check retryable before you retry. A 502 for a blocked site or an invalid_url fails again on every attempt, while 503, 504, and retryable 500 responses can succeed on a later try.


Migrate From the Contents API

Contents APIExtract API
POST /v1/contentsPOST /v1/extract
urls, up to 10 per requesturl, one per request. Send several requests concurrently for a list
formats: ["markdown", "html"]full_page.extraction_formats: ["markdown", "html"]
formats: ["metadata"]Not needed. metadata is part of the response
crawl_timeout (per URL, 1–60s)timeout (whole request, 2–100s). crawl_timeout is rejected with 422
max_ageextraction_source. Use fetch to bypass the cache or cache to read only cached content
Response is an array of pagesResponse is one object
page.markdown, page.htmlcontents.full_page.markdown, contents.full_page.html

Pricing

The Extract API costs $0.90 per 1,000 URLs. For PDF URLs, parse_effort: "standard" (the default) adds an additional charge for each page in the PDF: $0.50 per 1,000 pages. lite adds no page charge, and parse_effort has no effect on webpages.

A 350-page PDF at standard adds $0.175.

See the Billing page for complete pricing details.

For volume discounts, annual pricing, or enterprise features, visit you.com/pricing or contact api@you.com.


Next Steps