How to Build a News Search Pipeline With the You.com Web Search API

TLDR: You do not need a separate news API to get news results programmatically. The You.com Web Search API returns a results.news array alongside web results whenever your query has news intent, with publication timestamps, recency filters, and domain controls. This guide shows a working request, the news-specific fields, how to filter by time window and outlet, and the failure modes that quietly corrupt a news pipeline. Every parameter name here comes from the official API reference.

How do you get news results from an API? Send a query with news intent to https://ydc-index.io/v1/search with your X-API-Key header, and the response includes a results.news array of article objects. Filter that array with freshness, country, and include_domains instead of building a second pipeline.

Why Does One Endpoint Return Both Web and News Results?

The Web Search API classifies queries automatically. A query about a breaking event or a recent announcement returns news articles in the results.news array, next to the general web results in results.web. A query about a timeless topic does not trigger the news section at all. There is no separate endpoint, no news mode flag, and no second API key. You write one client, and the classification decides what comes back.

That design matters for pipelines. If you scrape or poll multiple outlet feeds, every source brings its own markup, rate limits, and publication-date quirks. A single structured response with normalized fields removes that entire layer.

What Fields Do News Results Carry?

Each news result includes title for the headline, description for the summary, url for the article link, and page_age for the publication timestamp in ISO 8601. When an image is associated, thumbnail_url carries it. When the request uses extraction_mode: "full_page", the result also gains a contents object with the full article text.

The page_age field is the one to design around. It is what lets you sort, deduplicate, and window your ingestion honestly, rather than guessing recency from the snippet.

What Does a Working News Query Look Like?

Here is a request that pulls recent coverage on a topic, limited to the last day, from outlets you specify.

curl -s -X POST https://ydc-index.io/v1/search \
  -H "X-API-Key: $YDC_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "query": "interest rate decision",
    "count": 20,
    "freshness": "day",
    "country": "US",
    "include_domains": ["reuters.com", "apnews.com", "bloomberg.com"]
  }' | jq '.results.news[] | {title, url, page_age}'

The count parameter caps results per section, up to 100. The freshness parameter accepts day, week, month, year, or a custom range like 2026-09-15to2026-09-22, inclusive of both dates. Domain lists support up to 500 entries.

How Do You Filter News by Country and Language?

country takes an ISO 3166-1 alpha-2 code such as US, GB, or DE, and focuses results geographically. language takes a BCP 47 code such as EN or DE. Combine them when your application serves a region: a German financial tracker sets country: "DE" and language: "DE" once, and every query inherits the focus.

These two filters answer different questions, so set them independently. Country focuses where results are relevant, which changes the outlet mix. Language constrains what language the results are written in, which changes what your downstream processing can parse. A tracker serving Canadian users in both English and French runs two configurations with the same country code and different language codes, rather than one compromise setting.

One caution on scope: these filters narrow the candidate set, and a narrower set can mean fewer news results for a niche topic. When a query that should have coverage comes back empty, relax the language filter first, since language mismatches exclude more real coverage than geography does.

How Do You Get Full Article Text Instead of Snippets?

Snippets rarely carry enough for analysis. Add an extraction object with extraction_mode: "full_page" and every news result gains a contents object with the full article, as markdown, HTML, or both via extraction.full_page.extraction_formats. This runs on the same request as the search, so there is no second fetch loop for results you were going to read anyway. Extraction may add cost depending on configuration, which the billing page details.

For pages you already have URLs for, the Contents API takes a list of URLs directly, without a search query, and is the right tool for re-polling known sources.

How Do You Turn Results Into a Monitoring Loop?

A one-shot query is a snapshot. A monitor is a loop with state. The shape that works: run the query on a schedule, window each run with a rolling freshness value, deduplicate new results against what you have already stored, and store the page_age alongside the URL so your timeline survives the source drifting away.

import os, json, urllib.request

def fetch_news(query, seen):
    req = urllib.request.Request(
        "https://ydc-index.io/v1/search",
        method="POST",
        headers={
            "X-API-Key": os.environ["YDC_API_KEY"],
            "Content-Type": "application/json",
        },
        data=json.dumps({
            "query": query,
            "count": 20,
            "freshness": "day",
        }).encode(),
    )
    with urllib.request.urlopen(req, timeout=30) as resp:
        body = json.load(resp)
    items = body.get("results", {}).get("news") or []
    fresh = [i for i in items if i["url"] not in seen]
    seen.update(i["url"] for i in items)
    return sorted(fresh, key=lambda i: i.get("page_age") or "", reverse=True)

seen = set()
for article in fetch_news("interest rate decision", seen):
    print(article.get("page_age"), article["title"], article["url"])

The deduplication set is the state that makes the loop safe to re-run. Without it, every poll re-reports the same coverage and your alert volume teaches people to ignore it.

What Silently Corrupts a News Pipeline?

Three failure modes cause most damage. First, temporal keywords in the query fight the freshness filter. A query containing phrases like "this week" widens the window to the broader of the two timeframes, so keep time words out of the query text and let freshness do the constraining. Second, deduplication. Two outlets syndicating the same wire story produce near-identical articles with different URLs. Deduplicate on normalized title or URL rather than raw URL alone. Third, relying on result order as a proxy for recency. Sort by page_age yourself, since relevance order is not chronological order.

When Should You Use the Research API Instead?

Search returns sources. The Research API reads them and returns a cited synthesis, which is the right tool when your application needs an answer assembled across coverage rather than the raw article list. A monitoring dashboard wants the Web Search API. A briefing generator wants the Research API.

How Do You Verify the Pipeline Is Working?

Two checks catch problems early. Sort the page_age values and confirm the oldest falls inside your freshness window, which catches the temporal-keyword widening described above. Then confirm the response actually contains a news section for a query that should have news intent, since a classification miss means your pipeline saw zero articles and reported all-clear. Log both checks, and alert on absence of news results rather than treating it as quiet.

Where Do You Go Next?

Start with the cURL example above against a live topic, and inspect the raw JSON before writing ingestion code. The live news guide documents every news-specific field and parameter, and the Web Search API hub page covers the full parameter surface. The date filter guide goes deeper on freshness windows and custom ranges.