> For clean Markdown of any page on these docs, append `.md` to the page URL
> Full documentation index: https://you.com/docs/llms.txt
> Search over these docs using a free, read-only MCP server (it is the best way to pick the right API(s) for the job and understand how to use them): https://you.com/docs/_mcp/server
> Access web search, web fetch, and deep research tools using our product MCP server: https://api.you.com/mcp (use `/mcp/research` and `/mcp/finance` paths for Research and Finance Research respectively)
> Call live Web Search for free: append `?profile=free` to the MCP server path
> OpenAPI spec index: https://you.com/docs/openapi.json
> How to authenticate: https://you.com/docs/using-the-api/authentication
> Changelog: https://you.com/docs/changelog
> Full onboarding instructions for agents: https://you.com/SKILL.md

# Extract

POST https://ydc-index.io/v1/extract
Content-Type: application/json

The Extract API fetches any webpage or PDF and parses it into clean Markdown or HTML. Use it after a Web Search API call, when your agent needs the full document, such as a filing, a contract, or a research paper, instead of a highlight.

PDF tables come back as Markdown or HTML tables with rows and columns intact. Set `parse_effort=standard` to also send complex PDF pages, such as charts and images, through an advanced model for the most complete output.

Reference: https://you.com/docs/api-reference/extract

## Authentication

- `X-API-Key` header (required) — A unique API Key is required to authorize API access. [Get your API Key with free credits](https://you.com/platform).

## Request

### Body (application/json)

This endpoint expects an ExtractRequest.

- `url` (string, required) — The URL to extract content from.
- `full_page` (FullPageConfig, optional) — Options for the returned page content. When omitted, the page is returned as Markdown.
- `extraction_source` (enum, optional, default: blend) — Indicates where `full_page` content comes from. `blend` serves cached content when it is available and crawls the page live when it is not. This is the default. `cache` returns cached content only. It is the fastest option, and `contents` is omitted for results that have no cached content. `fetch` always crawls the page live, which returns the freshest content at the cost of higher latency.
  - Allowed values: `blend`, `cache`, `fetch`
- `timeout` (integer, optional, default: 30) — Seconds the request may take end to end, including the cache lookup and the crawl. Must be between 2 and 100. Default is 30. Past it the request returns a 504.
- `parse_effort` (enum, optional, default: standard) — How much effort to spend parsing PDF pages. Has no effect on web pages. `lite` extracts the content embedded in each page. Use it for digitally created PDFs when you want the lowest cost. `standard` also sends complex pages, such as charts and images, through an advanced model, and is billed per page. Use it when you want the most complete output. For example, `lite` works well on a text-based contract, while an earnings report with revenue charts returns more complete content under `standard`.
  - Allowed values: `lite`, `standard`

## Response

### 200

A JSON object containing the extracted content from the URL

- `url` (string, optional) — The URL whose content was extracted.
- `extraction_source` (string, optional) — Where the request allowed content to come from. `blend` serves cached content when it is available and crawls the page live when it is not.
- `title` (string, optional) — The title of the webpage.
- `contents` (ExtractContents, optional) — The extracted content from the URL.
- `metadata` (ExtractMetadata, optional) — Metadata about the webpage.

## Errors

### 401 Unauthorized Error

Unauthorized. Problems with API key.

- `detail` (string, optional) — Error detail message.

### 402 Payment Required Error

Payment Required. The prepaid credit balance is depleted. Retrying fails until credits are added at `details.upgrade_url`. The API gateway can also answer with its own body, so branch on the 402 status rather than the body.

- `error` (ExtractErrorDetail, required)

### 403 Forbidden Error

Forbidden. API key lacks scope for this path.

- `detail` (string, optional)

### 422 Unprocessable Entity Error

Unprocessable Entity. The request can't succeed as sent. `code` is `invalid_request` when the body failed validation, with one entry per problem in `details.fields`. Unknown fields, including the removed `crawl_timeout`, are rejected. `invalid_url` means the URL can never be fetched (malformed, not http or https, a private address, or a hostname that doesn't exist), with the cause in `details.reason`. `pdf_too_large` means the PDF needs more time than `timeout` allows, so a higher `timeout` can let it through.

- `error` (ExtractErrorDetail, required)

### 500 Internal Server Error

Internal Server Error. `code` is `internal_error`. Retry only when `retryable` is `true`, and honor the `Retry-After` header when it is present. If the error persists, contact support with the `request_id`.

- `error` (ExtractErrorDetail, required)

### 502 Bad Gateway Error

Bad Gateway. The page itself can't be extracted, and retrying fails until the page changes. `code` is `site_blocked`, `site_unreachable`, or `unsupported_content`, with the cause in `details.reason`. `unsupported_content` can also include `details.content_type`.

- `error` (ExtractErrorDetail, required)

### 503 Service Unavailable Error

Service Unavailable. A dependency, including DNS, was down or rate limited. Retry after the number of seconds in the `Retry-After` header.

- `error` (ExtractErrorDetail, required)

### 504 Gateway Timeout Error

Gateway Timeout. The request didn't finish within `timeout`. `details.timeout_s` is the timeout that applied. Below the 100-second maximum, retry with a higher `timeout`.

- `error` (ExtractErrorDetail, required)

## Types

### FullPageConfig

Options for the returned page content. When omitted, the page is returned as Markdown.

- `extraction_formats` (list of enum, optional, default: ["markdown"]) — Content formats to return, one or both of `markdown` and `html`. An empty array is treated as `["markdown"]`.
  - Allowed values: `markdown`, `html`

### ExtractContents

The extracted content from the URL.

- `full_page` (ExtractContentsFullPage, optional) — The page content in each format requested in `full_page.extraction_formats`.

### ExtractMetadata

Metadata about the webpage.

- `site_name` (string, optional, nullable) — The OpenGraph site name of the webpage.
- `favicon_url` (string, optional) — The URL of the favicon of the webpage's domain.

### ExtractErrorDetail

- `code` (enum, required) — Stable error code to branch on. New codes may be added, but existing codes are never renamed or removed.
  - Allowed values: `invalid_request`, `invalid_url`, `payment_required`, `pdf_too_large`, `site_blocked`, `site_unreachable`, `unsupported_content`, `timeout`, `service_unavailable`, `internal_error`
- `message` (string, required) — Human-readable explanation. The wording can change, so don't parse it.
- `retryable` (boolean, required) — Whether sending the same request again can succeed.
- `request_id` (string, required, nullable) — The You.com ID for this request. Include it when you contact support.
- `details` (ExtractErrorDetails, optional) — Extra context for some codes. Omitted when there is none.

### ExtractContentsFullPage

The page content in each format requested in `full_page.extraction_formats`.

- `markdown` (string, optional) — The Markdown content of the page.
- `html` (string, optional) — The HTML content of the page.

### ExtractErrorDetails

Extra context for some codes. Omitted when there is none.

- `fields` (list of ExtractErrorDetailsFieldsItems, optional) — `invalid_request` only. One entry per validation problem.
- `reason` (string, optional) — Cause of the error. For `invalid_url`: `malformed`, `unsupported_scheme`, `private_address`, or `unresolvable`. For `site_blocked`: `unsupported_site`, `anti_bot`, or `login_required`. For `site_unreachable`: `tls_error`, `connection_failed`, `load_failed`, or `not_found`. For `unsupported_content`: `unsupported_type`, `too_large`, `not_a_pdf`, `unparseable`, `no_text`, or `unreadable_text`.
- `content_type` (string, optional) — `unsupported_content` only, when known. The content type of the page.
- `timeout_s` (integer, optional) — `timeout` and `pdf_too_large` only. The timeout, in seconds, that applied to the request.
- `page_count` (integer, optional) — `pdf_too_large` only, when known. The number of pages in the PDF.
- `upgrade_url` (string, optional) — `payment_required` only. Where to add credits.

### ExtractErrorDetailsFieldsItems

- `field` (string, optional) — Dotted path to the invalid field, such as `full_page.extraction_formats.0`, or `body` when the body itself can't be parsed.
- `message` (string, optional) — What is wrong with the field.

## Examples

**Request**

```json
{
  "url": "https://en.wikipedia.org/wiki/Retrieval-augmented_generation",
  "full_page": {
    "extraction_formats": [
      "markdown",
      "html"
    ]
  },
  "extraction_source": "blend"
}
```

**Response**

```json
{
  "url": "https://en.wikipedia.org/wiki/Retrieval-augmented_generation",
  "extraction_source": "blend",
  "title": "Retrieval-augmented generation - Wikipedia",
  "contents": {
    "full_page": {
      "markdown": "# Retrieval-augmented generation\n\n**Retrieval-augmented generation** (**RAG**) is a technique that enables large language models (LLMs) to retrieve and incorporate new information from external data sources...",
      "html": "<h1>Retrieval-augmented generation</h1>\n<p><b>Retrieval-augmented generation</b> (<b>RAG</b>) is a technique that enables large language models (LLMs) to retrieve and incorporate new information from external data sources...</p>"
    }
  },
  "metadata": {
    "site_name": "Wikipedia",
    "favicon_url": "https://api.ydc-index.io/favicon?domain=en.wikipedia.org&size=128"
  }
}
```

**SDK Code**

```python
import os

import requests

response = requests.post(
    "https://ydc-index.io/v1/extract",
    headers={"X-API-Key": os.environ["YDC_API_KEY"]},
    json={
        "url": "https://en.wikipedia.org/wiki/Retrieval-augmented_generation",
        "full_page": {"extraction_formats": ["markdown", "html"]},
    },
)
response.raise_for_status()
page = response.json()

# An empty object means nothing was extracted
if page.get("contents"):
    print(f"Title: {page.get('title')}")
    print(page["contents"]["full_page"]["markdown"][:500])

```

```typescript
const apiKey = process.env.YDC_API_KEY;
if (!apiKey) {
  throw new Error("Set the YDC_API_KEY environment variable");
}

const response = await fetch("https://ydc-index.io/v1/extract", {
  method: "POST",
  headers: {
    "X-API-Key": apiKey,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    url: "https://en.wikipedia.org/wiki/Retrieval-augmented_generation",
    full_page: { extraction_formats: ["markdown", "html"] },
  }),
});

if (!response.ok) {
  throw new Error(`Extract API error ${response.status}: ${await response.text()}`);
}

const result = await response.json();
console.log(result);

```

```javascript
const response = await fetch("https://ydc-index.io/v1/extract", {
  method: "POST",
  headers: {
    "X-API-Key": process.env.YDC_API_KEY,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    url: "https://en.wikipedia.org/wiki/Retrieval-augmented_generation",
    full_page: { extraction_formats: ["markdown", "html"] },
  }),
});

if (!response.ok) {
  throw new Error(`Extract API error ${response.status}: ${await response.text()}`);
}

const result = await response.json();
console.log(result);

```

```go
package main

import (
	"fmt"
	"strings"
	"net/http"
	"io"
)

func main() {

	url := "https://ydc-index.io/v1/extract"

	payload := strings.NewReader("{\n  \"url\": \"https://en.wikipedia.org/wiki/Retrieval-augmented_generation\",\n  \"full_page\": {\n    \"extraction_formats\": [\n      \"markdown\",\n      \"html\"\n    ]\n  },\n  \"extraction_source\": \"blend\"\n}")

	req, _ := http.NewRequest("POST", url, payload)

	req.Header.Add("X-API-Key", "<apiKey>")
	req.Header.Add("Content-Type", "application/json")

	res, _ := http.DefaultClient.Do(req)

	defer res.Body.Close()
	body, _ := io.ReadAll(res.Body)

	fmt.Println(res)
	fmt.Println(string(body))

}
```

```java
import com.mashape.unirest.http.HttpResponse;
import com.mashape.unirest.http.Unirest;

HttpResponse<String> response = Unirest.post("https://ydc-index.io/v1/extract")
  .header("X-API-Key", "<apiKey>")
  .header("Content-Type", "application/json")
  .body("{\n  \"url\": \"https://en.wikipedia.org/wiki/Retrieval-augmented_generation\",\n  \"full_page\": {\n    \"extraction_formats\": [\n      \"markdown\",\n      \"html\"\n    ]\n  },\n  \"extraction_source\": \"blend\"\n}")
  .asString();
```

```csharp
using RestSharp;

var client = new RestClient("https://ydc-index.io/v1/extract");
var request = new RestRequest(Method.POST);
request.AddHeader("X-API-Key", "<apiKey>");
request.AddHeader("Content-Type", "application/json");
request.AddParameter("application/json", "{\n  \"url\": \"https://en.wikipedia.org/wiki/Retrieval-augmented_generation\",\n  \"full_page\": {\n    \"extraction_formats\": [\n      \"markdown\",\n      \"html\"\n    ]\n  },\n  \"extraction_source\": \"blend\"\n}", ParameterType.RequestBody);
IRestResponse response = client.Execute(request);
```

```swift
import Foundation

let headers = [
  "X-API-Key": "<apiKey>",
  "Content-Type": "application/json"
]
let parameters = [
  "url": "https://en.wikipedia.org/wiki/Retrieval-augmented_generation",
  "full_page": ["extraction_formats": ["markdown", "html"]],
  "extraction_source": "blend"
] as [String : Any]

let postData = JSONSerialization.data(withJSONObject: parameters, options: [])

let request = NSMutableURLRequest(url: NSURL(string: "https://ydc-index.io/v1/extract")! as URL,
                                        cachePolicy: .useProtocolCachePolicy,
                                    timeoutInterval: 10.0)
request.httpMethod = "POST"
request.allHTTPHeaderFields = headers
request.httpBody = postData as Data

let session = URLSession.shared
let dataTask = session.dataTask(with: request as URLRequest, completionHandler: { (data, response, error) -> Void in
  if (error != nil) {
    print(error as Any)
  } else {
    let httpResponse = response as? HTTPURLResponse
    print(httpResponse)
  }
})

dataTask.resume()
```