Introducing web search monitors

How to Build a News Search Feed for AI Apps with the Context.dev News API

Context.dev News lets an AI app search current and historical news in plain language or by company (name, domain, ticker or ISIN), and returns articles with source, date, type and a story_id for collapsing duplicate coverage. These structured fields and matching passages let an LLM answer questions with verifiable source citations, while you can optionally pull full-article passages using Context.dev Highlights.

Tools used in this guide

  • Context.dev News (natural-language or company news search)
  • Context.dev Highlights (optional, for pulling the relevant passages from a full article)
  • The LLM or agent framework you already use

What does an AI app need from a news search API?

Large language models frequently offer confident answers about companies or market events that were accurate six months ago, entirely missing recent product launches, funding rounds, regulatory actions, or acquisitions. Teams introducing live news search across millions of articles into their agentic workflows face a unique set of engineering challenges that go beyond traditional keyword search.

To power an effective AI agent context window, a news search integration must solve several data structured problems simultaneously. The table below maps the core requirements for an AI news feed directly to the capabilities provided by the Context.dev News API.

AI Engineering RequirementContext.dev News API Field
Plain-language topic discoverysearchBy: { type: "natural_language", query }
Exact entity resolutionsearchBy: { type: "entity", entity } (domain, ticker with optional exchange, isin, name)
Document classificationfilterBy.articleType (editorial, press_release, regulatory_filing, advisory)
Time-bounded retrievalfilterBy.date (from and to in inclusive epoch milliseconds)
Syndication deduplicationstory_id per article plus match.level and match.confidence
Context-window efficiencydescription (matching passages up to 2,000 chars for natural-language search) + optional Context.dev Highlights (POST /web/scrape with formats.highlights: true)
Predictable usage pricing1 credit per 10 returned results, rounded up (X-Credits-Used and key_metadata.credits_consumed)

Step 1: Search news in plain language

Context.dev News makes it easy to search open-ended topics in plain language by setting a single parameter type. When an agent needs to discover general trends, such as "companies deploying AI agents for enterprise customer support" or "chip export restrictions to China", you can pass that exact phrase directly to the API.

To execute a plain-language search, set searchBy.type to "natural_language" and pass your plain-language string into the query field. You can also pass an optional limit parameter, which accepts an integer from 1 to 100 (and defaults to 10).

Here is how you call the endpoint using the official TypeScript SDK:

TypeScript
import ContextDev from "context.dev";
 
const client = new ContextDev({ apiKey: process.env.CONTEXT_DEV_API_KEY });
const news = await client.news.search({
  searchBy: { type: "natural_language", query: "chip export restrictions to China" },
  limit: 10,
});
 
console.log(news.data);

You can achieve the exact same result by hitting the REST endpoint (POST https://api.context.dev/v1/news/search) via cURL. Your application code simply needs to pass a Bearer token in the Authorization header:

Shell
curl https://api.context.dev/v1/news/search \
  -H "Authorization: Bearer $CONTEXT_DEV_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "searchBy": {
      "type": "natural_language",
      "query": "chip export restrictions to China"
    },
    "limit": 10
  }'

Step 2: Get news about one company

Context.dev News resolves specific companies deterministically so your AI agent does not confuse similarly named entities. When a user asks about Apple, an LLM keyword search might bring back articles about fruit farming, but a structured platform that looks up specific entities deterministically retrieves the exact corporate entity.

To search for a specific organization, set searchBy.type to "entity". You then specify searchBy.entity using one of four lookup identifiers:

  1. By Domain: Provide the canonical web address.
JSON
{
  "searchBy": {
    "type": "entity",
    "entity": {
      "type": "domain",
      "domain": "stripe.com"
    }
  }
}
  1. By Ticker: Look up public companies. You can pass the ticker alone, or provide an optional exchange identifier: entity: { type: "ticker", ticker: "AAPL", exchange: "NASDAQ" }.
  2. By ISIN: Use the International Securities Identification Number: entity: { type: "isin", isin: "US0378331005" }.
  3. By Company Name: Use the plain text name of the business: entity: { type: "name", name: "Stripe" }.

If the requested company cannot be resolved, this is considered an unknown company. The API returns a 404 status code, which consumes 1 credit.

Step 3: Filter and sort

Context.dev News ensures your application only consumes relevant tokens by filtering out irrelevant coverage before it ever reaches your agent. The filterBy object allows you to restrict results based on publisher metadata, article language, article type, and publication date.

Importantly, filterBy accepts exactly one category filter at a time, alongside an optional date range. Combining multiple category filters in the same request returns a validation error.

The available filter categories are:

FieldValuesMax items
sourceDomainPublisher domains, each 1 to 253 characters.3
sourceCountryLowercase ISO 3166-1 alpha-2 country codes.3
articleLanguageSupported language codes, such as ["en", "de"].3
articleTypeeditorial, press_release, regulatory_filing, advisory.3

Note that language filters will keep articles whose language cannot be successfully detected.

To restrict results to a rolling window (such as the last 24 hours or 30 days) and verify article freshness, use date.from and date.to. These are inclusive epoch milliseconds, and from must be at most to. Either bound can be omitted.

JSON
{
  "searchBy": {
    "type": "entity",
    "entity": {
      "type": "domain",
      "domain": "stripe.com"
    }
  },
  "filterBy": {
    "articleType": [
      "press_release"
    ],
    "date": {
      "from": 1785542400000,
      "to": 1788220799999
    }
  },
  "sortBy": {
    "type": "newest"
  }
}

Filters operate exactly the same across natural-language and company searches, but sorting rules differ depending on your search type. For company searches (searchBy.type: "entity"), sortBy.type accepts "newest" (which is the default) or "relevance". For natural-language searches (searchBy.type: "natural_language"), results are always ranked by relevance with a boost for recent articles. Passing sortBy: { type: "newest" } in a natural-language search returns a validation error.

Step 4: Match exact phrases

Context.dev News supports powerful query operators inside natural-language searches to give you granular control over keyword extraction. You can group terminology and explicitly remove irrelevant subjects to maintain a high signal-to-noise ratio.

When formatting your searchBy.query string, you can use:

  • Double quotes ("..."): Match an exact phrase. For example, "interest rates" ensures the words appear sequentially.
  • OR operator: Place an uppercase OR between quoted phrases or terms to match either phrase. For example: "electric vehicles" OR "hybrid engines".
  • Minus (-) exclusions: Prefix a term or quoted phrase with a minus sign to exclude articles containing it. For example, "apple" -"iphone" -"macbook" helps isolate agricultural news from technology coverage.

Step 5: Page through results

Context.dev News relies on deterministic cursor pagination to help developers process paginating through large historical datasets without skipping articles or entering endless loops. The API uses an opaque cursor system based on has_more and next_cursor.

You control page size with the limit parameter, accepting an integer from 1 to 100 (defaulting to 10). When a response indicates has_more is true, you send the returned next_cursor as the cursor property on your subsequent request. You must keep your searchBy, filterBy, and sortBy parameters completely unchanged across pages, only modifying the cursor or limit.

A cursor is entirely opaque. You should never construct or edit one manually. For natural-language queries, there is a fixed limit: the API returns up to 100 matches across all pages combined.

When writing a backfill loop, you must save your data and deduplicate by article id. You should safely stop execution when has_more is false, next_cursor is absent, or you encounter a repeated cursor. For long backfills, set a fixed date window (date.from and date.to), save your cursor after each processed page, and manage your rate limits before requesting the next batch.

The logic flows like this pseudocode:

cursor = absent
repeat:
  request the same search with cursor
  save data, deduplicating by article id
  stop when has_more is false or next_cursor is absent
  cursor = next_cursor

Step 6: Collapse syndicated copies with story_id

Context.dev News provides specialized clustering fields to prevent duplicate syndication from overwhelming an LLM context window. News wires and syndication networks routinely republish the exact same story across dozens of publisher sites. If you feed all of these into an AI agent, the duplicated tokens drown out unique information.

Every article returned in the data array includes an id (the unique article identifier) and a story_id (the story cluster identifier). When a press release is syndicated across multiple outlets, each copy features a distinct id and url but shares the exact same story_id.

To clean up your AI feed, group or deduplicate your results by story_id, falling back to id if story_id is null or absent. This collapses duplicate coverage, allowing your application to pass a single representative article per news event to the model.

For company searches (searchBy.type: "entity"), you also receive a match object containing match.level and match.confidence. Filtering your grouped results by match.level ensures you keep primary coverage where the target company is the main subject of the article, rather than just a passing mention in a broader industry roundup.

Step 7: Pass articles to an LLM with citations

Context.dev News supplies precisely the right payload fields to ground an LLM generation with accurate, verifiable citations. When parsing your API response, you should extract specific metadata to format into your system prompt.

Pass the following fields to your model:

  • title: The article headline.
  • description: The article summary or snippet. For natural-language searches, this description field actually includes matching passages up to 2,000 characters, saving you a secondary scraping step.
  • published_at: The publication timestamp, crucial for helping the LLM reason about sequence and recency.
  • source: Publisher metadata (like domain and name) for direct attribution.
  • url: The canonical article link. Always keep the url and published_at timestamp attached to every passage block so the agent can cite sources cleanly in its final answer.

Sometimes, the 2,000 character matching passages in the description are not enough, and your agent needs deeper detail from a specific article URL. In this case, you can use Context.dev Highlights. You call POST /web/scrape with formats: { highlights: true } and provide highlightsParams: { query, maxCharacters }.

The highlightsParams.query is required and can be between 1 and 2,000 characters. The highlightsParams.maxCharacters limit defaults to 4,000 but accepts a range from 100 to 20,000 characters across the returned passages. Highlights read the first 400,000 characters of the page Markdown after content filters, returning relevant passages in chronological page order within highlights.data. You will see bracketed headings for context and ellipses marking omitted text. Always check highlights.success before attempting to read highlights.data.

Here is an example Highlights request payload:

JSON
{
  "url": "https://example.com/article",
  "formats": {
    "highlights": true
  },
  "highlightsParams": {
    "query": "What are the key terms and timeline announced?",
    "maxCharacters": 1500
  }
}

Complete TypeScript Pipeline Example

Below is a complete, production-ready function that brings these concepts together. It searches by domain for press releases over the last 30 days, pages with a cursor safely, deduplicates by id, collapses syndicated copies using story_id, and maps the final output into a clean format ready for an LLM.

TypeScript
import ContextDev from "context.dev";
 
export async function getCompanyNews(domain: string) {
  const client = new ContextDev({ apiKey: process.env.CONTEXT_DEV_API_KEY });
  
  const to = Date.now();
  const from = to - 30 * 24 * 60 * 60 * 1000;
 
  let hasMore = true;
  let currentCursor: string | undefined = undefined;
  
  const seenIds = new Set<string>();
  const seenStoryIds = new Set<string>();
  const processedArticles = [];
 
  while (hasMore) {
    const response = await client.news.search({
      searchBy: {
        type: "entity",
        entity: { type: "domain", domain }
      },
      filterBy: {
        articleType: ["press_release"],
        date: { from, to }
      },
      sortBy: { type: "newest" },
      limit: 100,
      ...(currentCursor && { cursor: currentCursor })
    });
 
    for (const article of response.data) {
      if (seenIds.has(article.id)) continue;
      seenIds.add(article.id);
 
      const clusterId = article.story_id || article.id;
      if (seenStoryIds.has(clusterId)) continue;
      seenStoryIds.add(clusterId);
 
      processedArticles.push({
        title: article.title,
        url: article.url,
        published_at: article.published_at,
        source: article.source
      });
    }
 
    if (!response.has_more || !response.next_cursor || response.next_cursor === currentCursor) {
      hasMore = false;
    } else {
      currentCursor = response.next_cursor;
    }
  }
 
  return processedArticles;
}

What it costs

Before launching your agent to production, you should read the precise credit billing rules to understand exactly how your application consumes credits.

Context.dev News costs 1 credit per 10 returned results, rounded up. If you query an unknown company and receive a 404 response, that lookup costs 1 credit. If your agent proceeds to a second step using the optional Scrape API, a base Scrape costs 1 credit (or 2 if browser actions are enabled), and requesting Highlights with at least one successfully extracted passage adds +1 credit to that request.

Your organization shares a single credit balance across all its API keys and managed workloads. You can programmatically track your usage by reading key_metadata.credits_consumed and key_metadata.credits_remaining inside JSON responses, or by inspecting the X-Credits-Used response header. Every new account receives 1,000 free API credits every month with no credit card required.

Frequently Asked Questions

How do I get only press releases or regulatory filings for a company?

Context.dev News allows you to isolate official announcements by passing filterBy: { articleType: ["press_release", "regulatory_filing"] } in your request. You can pass up to 3 values at a time from the allowed list of editorial, press_release, regulatory_filing, and advisory.

How do I avoid the same story appearing many times?

Context.dev News provides the fields necessary to filter out syndication spam by deduplicating by id across all your returned pages, and then grouping or collapsing syndicated copies by story_id in your application code before passing the text to your LLM.

Can I search news by stock ticker?

Context.dev News makes ticker lookups simple by setting searchBy to { type: "entity", entity: { type: "ticker", ticker: "AAPL", exchange: "NASDAQ" } }. The exchange field is optional, and this method works alongside native support for looking up entities by domain, isin, and name.

How far back can I page through results?

Context.dev News lets you define exact historical windows by setting date.from and date.to using inclusive epoch milliseconds. You then page sequentially using next_cursor while has_more is true. Note that natural-language searches return up to 100 matches maximum across all pages, while company searches can page indefinitely through historical windows using a fixed date range.

How much does a Context.dev News search cost?

Context.dev charges exactly 1 credit per 10 returned results, rounded up. Additionally, it costs 1 credit when a company lookup returns a 404 for an unknown company.


Ready to build a reliable, deduplicated news feed for your AI agents? Get your API key at https://www.context.dev and read the full integration details in the News Overview documentation.

Start building with live web context.

Scrape, search, and research the web with one API. Bring the results straight into your product.

1,000free credits

Every month. No credit card required.

Start for freeRead the docs