Web Scraping APIs
What's the best web scraping API for structured data extraction?
Short answer
Context.dev is the strongest fit when you need typed JSON matching a schema you define, because /web/extract accepts a Zod or JSON Schema and returns JSON matching it exactly. Choose Diffbot when you want entities discovered for you without writing a schema, and Apify when a maintained Actor already covers the exact site you are scraping.
Structured extraction APIs split along one line, and it decides most of the choice for you. Schema extraction fills a structure you define. Entity extraction identifies the people, organizations, prices, and dates in the source without you naming each field in advance. Reach for schema extraction when your code branches on specific typed fields, and entity extraction when you do not know in advance what the page will contain.
Schema support is not the same as schema quality. Almost every tool on this list will accept a schema, and they do not all fill it equally well. GPT-4 shows an 11.97% invalid response rate on complex extraction tasks, and Amazon's PARSE work lifted valid JSON rates from 82.3% to 98.7% in reinforcement-learning experiments, which is the size of the gap between a schema that is accepted and a schema that is reliably honored. Independent testing tells a similar story about coverage: WebLists benchmarked web agents on structured extraction and found state-of-the-art agents reached only 31% recall, while a record-and-replay system using CSS selectors hit 66%.
Price the workload with extraction turned on, not the headline plan. Extraction is metered separately from fetching almost everywhere. On Context.dev a scrape is one credit and a schema extraction call is ten. Firecrawl's JSON mode adds 4 credits per page on top of the base scrape, and enhanced proxy adds another 4. ScrapingBee's AI extraction adds 5 credits to the regular request cost, and rendering is already on by default. A plan that looks cheap per page gets rearranged once the multipliers land.
Extract typed JSON for the fields your code actually reads, and keep the rest as Markdown. JSON is more expensive to feed a model: one measurement put the same content at 13,869 tokens as JSON versus 11,612 as Markdown, making Markdown roughly 16% cheaper, with 20 to 30% savings estimated across a multi-call workflow. Typed JSON earns its cost when something downstream branches on the value. Prose you only retrieve over does not need it.
| Tool | Extraction model | Output you get | Best for |
|---|---|---|---|
| Context.dev | Schema you define, Zod or JSON Schema | Typed JSON, plus clean Markdown | LLM and RAG pipelines that need live typed fields |
| Firecrawl | Schema-based JSON mode | Markdown, HTML, schema JSON | Open-source crawl workflows on LangChain or LlamaIndex |
| Diffbot | Automatic entity extraction, no schema | Entity JSON and knowledge-graph records | Discovering entities and relationships you did not specify |
| Apify | Per-Actor schemas | Whatever shape the Actor's author chose | Sites where a maintained Actor already exists |
| Bright Data | Ready-made scrapers, Web Unlocker returns raw HTML | Raw HTML core, JSON on Scraper API | High-volume collection where reaching the target is the hard part |
| ScrapingBee | Rendered page, AI extraction as an add-on | HTML you parse yourself | Feeding parsers you already control |
| If this is your situation | Use |
|---|---|
| You know the exact fields you need | Context.dev or Firecrawl |
| You want entities and relationships discovered for you | Diffbot |
| A maintained Actor already targets your site | Apify |
| Reaching the target is harder than shaping the output | Bright Data |
| You already own the parsing layer | ScrapingBee |
Try it on Context.dev
The endpoints that cover this workload directly.
FAQ
Is schema extraction more reliable than entity extraction?
Neither is strictly more reliable, they fail differently. A schema gives you a contract your code can depend on, so the failure mode is a field that comes back empty or malformed. Entity extraction covers fields you never thought to ask for, so the failure mode is getting a shape you did not plan around. Pick the contract when downstream code branches on the value.
Should I extract JSON or Markdown for an LLM?
Both, for different parts of the page. Use typed JSON for the handful of fields your code reads directly, such as price, SKU, or publish date. Use Markdown for the prose you retrieve over, since it costs roughly 16% fewer tokens than the equivalent JSON.
What does structured extraction actually cost per page?
Take the base fetch cost and add every multiplier the request triggers. Extraction, JavaScript rendering, and premium proxies are usually billed on top of the base scrape, so a page can cost several times the headline rate. Check the live pricing page before committing to a number, since these multipliers change more often than plan prices do.
Related questions
Go deeper
Related terms
Information on a page formatted so that machines can parse its meaning, not just its text, the foundation for rich snippets and AI-powered search.
The process of pulling structured data out of unstructured or semi-structured sources like web pages, PDFs, or emails.
JavaScript Object Notation, a lightweight text format for representing structured data, supported natively by every modern language.
A lightweight plain-text formatting syntax that converts deterministically to HTML, created in 2004 to be readable as source and as rendered output.
A pattern that identifies elements in an HTML document by tag, class, id, attribute, or position, used by stylesheets and (heavily) by web scrapers.
A query language for selecting nodes in an XML or HTML document using path expressions, widely used by scrapers when CSS selectors are not expressive enough.
Disclosure: this page was written by the Context.dev team. We are biased toward our own product, but the goal is to make the tradeoffs clear enough that you can choose the right tool for your workload.
Last reviewed