Openbenchmarks runs an independent web search benchmark for AI agents. We did not run it, and its harness, code and dataset are public. Last measured 21 October 2026, it puts Context.dev second of the 13 vendors on its factual lookup task, at 98.0% extracted-answer accuracy, behind Exa's type=fast endpoint at 99.3%.
Second is a good place to be on a board we had no hand in. It is also a narrow second, on a task that strips away everything except the search API itself, so here is what the board measures, where we landed, and what the numbers cost.
What the board measures
The web search benchmark has three tasks: factual lookup, hard retrieval, and multi-hop search. Context.dev appears only on factual lookup, so nothing in this post speaks to hard retrieval or multi-hop, and the 98.0% figure should not be stretched to cover them. The board lists 13 vendors across the three tasks. Eleven of them ran factual lookup, through 17 endpoint configurations, since several vendors entered more than one mode.
Factual lookup is a single question with one verifiable answer: a recent company event, dated after the extraction model's knowledge cutoff. The 300 questions are the same set behind the site's company news search benchmark; the factual lookup section of the web search board is that benchmark, reproduced. Every endpoint receives the user question verbatim, one request, at most 10 results. Nothing is rewritten per vendor.
Everything else is held constant. An extraction model, gpt-5.6-terra at medium reasoning, reads only the returned titles and snippets and writes an answer. A separate judge from a different model family, claude-opus-5 via Amazon Bedrock, scores that answer against human-labelled ground truth taken from official wires and newsrooms. The judge never fetches the live page. The only thing that changes between rows is which search API was called.
With no search tool at all, gpt-5.6-terra answers 10.9% of these questions correctly, so a vendor at 92% is not 92 points better than nothing. The harness, scoring code and vendor runners are public in the factual-lookup-company-news-search repository, and a 100-question public slice of the dataset, with gold answers, is on Hugging Face as openbenchmarks/OB-News-Websearch. Board scores use the locked 300-question set, so the public slice will not reproduce the board, but the scoring is the same. The site also notes that it is re-running every vendor in the same window, because scores depend on when each vendor was benchmarked (see the notice on the board).
Where Context.dev landed
The run called POST api.context.dev/v1/web/search with numResults=10 and highlights capped at 1,200 characters per result. That is our Search API with per-result highlights on, as described in the search docs.
On accuracy, the column the site leads with, the top of the web search band reads: Exa fast 99.3%, Context.dev 98.0%, Exa instant 97.7%, Perplexity 97.3%, Linkup fast 96.7%, SERP 96.0%, Firecrawl 95.3%, Nimble lite 94.3%, Brave LLM context 94.0%. Eight more endpoints sit between 87.7% and 93.3%. The gap from us to first is 1.3 points, which on 300 questions is about four answers. The gap from us down to the tenth endpoint is 4.7 points.
Answer recall at five, AR@5, asks a stricter question: did any of the top five snippets already contain the full ground-truth answer, before any model touched them. Context.dev scored 98.3%, second to Exa fast at 99.3% and level with Perplexity. That is the number we care about most, because it describes what an agent gets back from one call before it does any work of its own. Almost every time, the answer is already sitting in the first five results.
| Endpoint | Accuracy | AR@5 | List price per 1,000 queries | $ per 1,000 correct |
|---|---|---|---|---|
| Exa (type=fast) | 99.3% | 99.3% | $7.00 | $7.05 |
| Context.dev | 98.0% | 98.3% | $4.40 | $4.49 |
| Exa (type=instant) | 97.7% | 97.3% | $7.00 | $7.17 |
| Perplexity | 97.3% | 98.3% | $5.00 | $5.14 |
| Linkup (fast) | 96.7% | 94.7% | $5.00 | $5.17 |
| SERP (RapidAPI) | 96.0% | 95.0% | $3.00 | $3.13 |
| Firecrawl | 95.3% | 96.7% | $5.00 | $5.24 |
| Nimble (lite) | 94.3% | 93.7% | $1.10 | $1.17 |
Numbers as published on openbenchmarks.com on 21 October 2026. List prices are the published pay-as-you-go rates the site links to on each vendor's pricing page.
Cost per correct answer
The cost column is the one we would point a procurement team at. Openbenchmarks divides each endpoint's published pay-as-you-go price per 1,000 queries by its accuracy, which gives dollars per 1,000 correct answers. A cheap endpoint that is wrong a lot gets more expensive under that arithmetic.
Context.dev comes out at $4.49 per 1,000 correct answers, from a list price of $4.40 per 1,000 queries (two credits per ten results with highlights, at $2.20 per 1,000 credits). That is the lowest figure among the five most accurate endpoints: Exa fast is $7.05, Exa instant $7.17, Perplexity $5.14, Linkup fast $5.17. The whole 300-question run cost $1.32 in search fees on our side.
Cheaper endpoints exist lower on the board, at lower accuracy. Among the five that answered more than 96% of the questions, Context.dev is the cheapest seat, and the difference compounds: at a million correct answers, the gap to the endpoint in first place is about $2,560.
The company news board: web search against dedicated news APIs
The company news search benchmark asks the same 300 questions of 21 endpoints, with a twist. It splits them into two bands: general web search APIs that search the open web, and dedicated news indexes built specifically for press releases, wires and company newsrooms. The two bands are ranked inside themselves rather than against each other, because, as the site puts it, a single ordering would assert that the two are drop-in substitutes, which is a procurement claim the benchmark does not test. The code for this board lives in its own repository, openbenchmarks-labs/company-news.
The comparison across the bands is still sound, and the site says so. Both bands answered the identical 300 questions under the identical protocol, so the gap between the best web search API and the best news index is a measured difference in result quality, not an artifact of the two being asked different things.
The gap is wide. The best dedicated news index, Seltz with scope=news, scored 58.3%. Autobound's company news events endpoint scored 30.0%, PredictLeads 20.7%, and Datahyena 5.3%, which is below the 10.9% a model manages with no search at all. Context.dev, a general web search API, scored 98.0% on the same questions. Three of the four news indexes have no free-text query; they were sent a company domain plus an event category instead, which the site documents. That is a fair reading of what those products offer, and the question is still the one an agent has.
We have an opinion here, and it is the obvious one. If the job is "what did this company announce," a feed built only from wires is not automatically the better tool. In this test, a search API that reads the open web beat the best of them by nearly forty points. Our own news search is for teams that want news-shaped results, judged by the same standard: did the answer come back.
The site is clear about what the board does not measure. It does not test query rewriting or multi-hop searches; the question goes in as-is. It does not vary search depth; every endpoint returns ten results. And it does not measure streaming, webhooks or time-to-index, so a news index that pushes an item minutes after publication gets no credit here. A team buying a news feed for freshness should test freshness.
How we read this
On an independent board we did not touch, Context.dev is the second most accurate web search API for recent company facts, within 1.3 points of first, with the lowest cost per correct answer of anyone near the top. On the same questions it answered nearly forty points more of them than the best product built specifically for company news.
Read the methodology rather than our summary of it; the harness is on GitHub, the dataset is on Hugging Face, and the board updates. We also publish a benchmark of our own, TRACE, which draws a fresh sample of pages every day for scraping rather than search, under the same rule: open code, open data, and page-level results anyone can download. Every third-party board we appear on is collected, with links to the source, on our benchmarks page.
FAQs
How did Context.dev score on the Openbenchmarks web search benchmark?
On the factual lookup task, last measured 21 October 2026, Context.dev scored 98.0% extracted-answer accuracy on 300 company-news questions, second of 13 vendors behind Exa type=fast at 99.3%, and 98.3% answer recall at five.
How much does Context.dev cost per correct answer on this board?
$4.49 per 1,000 correct answers, from a list price of $4.40 per 1,000 queries. That is the lowest figure among the five most accurate endpoints; Exa fast is $7.05 and Perplexity $5.14.
How does a web search API compare with dedicated company news APIs?
On the identical 300 questions, Context.dev scored 98.0% while the best dedicated news index, Seltz, scored 58.3%, followed by Autobound at 30.0%, PredictLeads at 20.7% and Datahyena at 5.3%.