TL;DR
- WebCrawlerAPI's published test reports usable Markdown on 92.1% of 806 scored URLs for Context.dev versus 89.7% for Firecrawl. Scrapingbench's agent-task test reports 95% versus 80% on 40 scrape tasks. These are independent reports with different methods, not results from our proposed five-vendor benchmark.
- We will test Context.dev, Firecrawl, Apify, Zyte, and ScrapingBee on static, JavaScript-rendered, and blocked public pages.
- For each workload, we will ask which API returns usable JSON or Markdown for AI agents. We will measure usable-output rate, latency, output quality, and cost per usable page.
- Our five-vendor benchmark remains unrun. We will publish its fixtures, runnable code, configurations, sample sizes, failures, and raw results before making any winner, reliability, or cost-per-usable-page claims about it.
Why most scraping API comparisons can't be trusted
A web scraping API comparison cannot support a winner if readers cannot inspect or rerun the test. Vendor-published benchmarks and marketing roundups often leave out the tested URLs, request settings, or failed attempts. For an AI agent, a successful request also says little about whether the returned Markdown or JSON contains the fields the agent needs.
Our earlier API roundup introduces available products, while our Playwright TCO article examines the infrastructure and cost of running browsers yourself. This benchmark asks a different question. How do Context.dev, Firecrawl, Apify, Zyte, and ScrapingBee perform on the same static, JavaScript-rendered, and blocked public pages?
Two external reports already provide measured evidence, but neither answers this article's five-vendor question. WebCrawlerAPI measured page retrieval and Markdown cleanliness, while Scrapingbench measured whether an agent completed specific tasks. We will publish our own fixture set, runnable harness, configurations, scoring rubric, run dates, pricing assumptions, and raw results. Our five-vendor results remain pending until the tests execute.
Evaluation table: what will be measured and how
We will test each API against the same static, JavaScript-rendered, and blocked public-page workloads. The table defines the measures, not results. We will mark workload support only after testing each vendor's documented configuration.
| Vendor | Workload types supported | Usable-output rate | Latency (p50/p95) | JSON/Markdown quality score | Cost per usable page | Status |
|---|---|---|---|---|---|---|
| Context.dev | To be measured | Pending | Pending | Pending | Pending | Pending |
| Firecrawl | To be measured | Pending | Pending | Pending | Pending | Pending |
| Apify | To be measured | Pending | Pending | Pending | Pending | Pending |
| Zyte | To be measured | Pending | Pending | Pending | Pending | Pending |
| ScrapingBee | To be measured | Pending | Pending | Pending | Pending | Pending |
Usable-output rate will count pages that return the required content in a form an AI agent can use. We will report median and 95th-percentile latency across requests, score JSON and Markdown against a published rubric, and divide total test charges by usable pages to calculate cost. The final results will include sample sizes, failure counts, and raw request data alongside the scores.
Published third-party results, not our five-vendor benchmark
| Source and measure | Context.dev | Firecrawl | Other measured providers |
|---|---|---|---|
| WebCrawlerAPI: usable Markdown on 806 URLs | 92.1% | 89.7% | WebCrawlerAPI 88.0%; Crawl4AI Cloud 88.0%; Jina Reader 86.8%; Spider 84.0%; Crawlbase 70.0% |
| WebCrawlerAPI: extraction F1, default settings | 0.176 | 0.539 | WebCrawlerAPI 0.623; Crawl4AI Cloud 0.562 |
| Scrapingbench: agent scrape tasks completed, 40 tasks | 38/40 (95%) | 32/40 (80%) | Spider 37/40 (93%); Scrapfly 36/40 (90%); String 35/40 (88%) |
WebCrawlerAPI defines usable Markdown as containing at least 10% of expected main-content words. Its extraction F1 separately penalizes sampled page boilerplate. Context.dev's default output scored lower on that cleanliness proxy than Firecrawl's, despite its higher usable-output rate. With Context.dev's mainContentOnly option enabled, its F1 rose to 0.567, while usable-output success fell to 89.7%. The report used different dates, timeout budgets, and retry settings across tools. WebCrawlerAPI built one of the tested products, and the URL dataset comes from Firecrawl. Read its methodology and settings.
Scrapingbench tests an agent's correct answers, not a raw page success rate. Its scoring method fixes the agent's prompt and task limits and checks answers by script. The scrape track mostly tests documentation headings. Context.dev's 15-point lead over Firecrawl on the 40 scrape tasks has a reported 95% site-clustered interval of 4.9 to 27.0 points. Neither report tested Apify, Zyte, or ScrapingBee, or establishes the five-vendor blocked-page, JSON-quality, and cost-per-usable-page results still pending here.
Benchmark design: fixtures, harness, and scoring rubric
The benchmark will test three kinds of public pages because each requires different retrieval work. Static HTML pages expose content in the initial response. JavaScript-rendered pages require a browser or equivalent rendering step to reveal content. Bot-protected pages test how each API handles access restrictions where testing is permitted. We will publish a stable public fixture set or the exact URL list, along with expected fields and page snapshots, so you can inspect what each request was meant to retrieve.
A runnable harness will send the same eligible URLs to Context.dev, Firecrawl, Apify, Zyte, and ScrapingBee. Each vendor will receive the same timeout and retry budget. We will record the endpoint or Actor used, its version where applicable, rendering settings, and any other vendor-specific configuration. Default and premium features will run as separate configurations rather than mixing their results. The harness, configuration, run dates, and pricing assumptions will accompany the raw responses.
The scoring rubric will count a page as usable only when its output contains the required content and passes a published quality threshold. For JSON, the rubric will check required fields and their values against the fixture. For Markdown, it will check content coverage, reading order, and whether navigation or other page chrome obscures the requested material. Usable-output rate will divide passing pages by all attempted pages, including timeouts and errors. We will report median and 95th-percentile latency by workload, plus cost per usable page calculated from billable requests, including retries. Sample sizes and failure counts will remain visible beside each score.
Live sites can change between requests, and bot protections can vary by time or requester. We will retain snapshots and repeat runs so readers can distinguish page changes and access variance from API behavior. Before testing a URL, we will check robots.txt and applicable access terms. Pages that do not permit the planned requests will not enter the test set. No scores or vendor verdicts will appear until the harness has run.
The five APIs under test
Context.dev
Best for Context.dev is a strong candidate if you want to replace an internal crawler with one managed API for scraping, crawling, and structured data delivery. Its fit for static, JavaScript-rendered, and blocked pages remains unverified until the benchmark runs.
What it is Context.dev provides Scrape, Crawl, and Answers endpoints for retrieving web pages and extracting data. Scrape supports JavaScript-rendered pages, while Markdown and JSON output can feed LLM pipelines. MCP offers another way to connect agents to live web data.
Pros The single-API approach can reduce the engineering needed to connect retrieval, rendering, and extraction. Context.dev also pitches consolidation of separate proxy-rotation, CAPTCHA-solving, and scraping vendor contracts. That pitch matters if you would otherwise maintain those components yourself, but the benchmark must test the quality of the output they produce.
Cons Your application still owns business-specific validation of extracted data. WebCrawlerAPI's report found that Context.dev's default Markdown retained more boilerplate than Firecrawl's, with an extraction F1 of 0.176 versus 0.539. Context.dev's main-content-only setting raised F1 to 0.567 but reduced its usable-output rate from 92.1% to 89.7%. Those measurements do not establish a Cloudflare or Turnstile success rate.
Pricing Paid plans start at $20 per month for 10,000 credits. A basic scrape costs 1 credit, and successful JSON extraction adds 4. The benchmark will record the plan used for each run, then calculate cost per usable page. No comparative cost claim is available yet.
Firecrawl
Best for Firecrawl is worth evaluating if you need a managed crawl that turns discovered pages into content an AI agent can read. Its fit for any tested workload remains unproven until the benchmark runs.
What it is Firecrawl provides scraping and crawling APIs with LLM-ready output, including Markdown. Its crawl workflow gathers pages for conversion, while Context.dev positions its single API around scraping, crawling, and structured data delivery. Both offer managed approaches, so the benchmark will compare their outputs rather than treat the products as different infrastructure categories.
Pros Firecrawl's crawl-focused approach gives developers a direct way to collect page content for LLM pipelines. Its own enterprise comparison also shows why it belongs in an evaluation of scalable crawling, though vendor positioning does not establish benchmark performance.
Cons In WebCrawlerAPI's default-Markdown test, Firecrawl returned usable content for 89.7% of 806 scored URLs versus Context.dev's 92.1%, but Firecrawl's extraction F1 was 0.539 versus Context.dev's default 0.176. In Scrapingbench's scrape track, the agent finished 32 of 40 tasks with Firecrawl versus 38 with Context.dev. None of these results establishes which vendor will deliver better JSON for this benchmark.
Pricing The benchmark will record Firecrawl's plan, billing assumptions, and charges for the tested requests. Cost per usable page remains pending.
Apify
Best for Developers who need a site-specific workflow or want to adapt an existing scraper rather than use the same extraction endpoint for every site.
What it is Apify runs Actors, which are hosted programs for tasks such as crawling and extraction. You can build an Actor or choose one from its marketplace. Context.dev and Firecrawl offer managed extraction endpoints instead, so their standard requests require less workflow selection and configuration.
Pros An Actor can encode site-specific navigation and extraction rules. That flexibility helps when a standard endpoint cannot express the steps a task needs.
Cons Actor inputs, output formats, and maintenance vary by implementation. For the benchmark, we will name the Actor and configuration used for each workload, then score its output against the same rubric as the managed APIs. A result for one Actor will not represent every Apify Actor.
Pricing Compare the full cost of each configured Actor run, including any applicable marketplace charges. We will publish those assumptions with the results rather than quote an unverified price here.
Zyte
Best for Zyte is worth evaluating if you need managed extraction at scale and want the provider to handle proxy and access infrastructure. Its fit for the three benchmark workloads remains untested.
What it is Zyte API provides web retrieval and extraction through a managed service. Zyte's Smart Proxy Manager heritage helps explain its focus on access management, but that history does not establish how Zyte API will perform on this test set.
Pros Zyte gives you an alternative to maintaining your own proxy infrastructure. The benchmark will test whether its output works for AI agents across static, JavaScript-rendered, and blocked public pages.
Cons A managed API offers less control over retrieval than a browser workflow you operate yourself. We cannot assign Zyte a reliability or scale advantage without results.
Pricing We will record the applicable Zyte configuration and pricing assumptions when the tests run, then calculate cost per usable page. No cost comparison is available yet.
ScrapingBee
Best for Developers who want to fetch individual pages through an API without maintaining browsers or proxies.
What it is ScrapingBee accepts a URL and returns page content. You can request JavaScript rendering for pages that need a browser, while ScrapingBee handles proxy routing.
Pros The per-request model fits an agent that needs to retrieve a known URL on demand. Optional rendering lets you use the same API for simpler pages and pages whose content loads through JavaScript.
Cons Page retrieval alone does not guarantee that an agent receives fields in its required JSON schema. If your pipeline needs structured records, test ScrapingBee's extraction options against that schema and validate the returned fields. A URL-by-URL API call also leaves crawl discovery and orchestration to your application.
Pricing Record the applicable credit rate and credits consumed for each test configuration before calculating cost per usable page. No cost comparison is available until the benchmark runs.
Managed extraction versus Actor-based workflows
Managed APIs are the simpler fit when you want to send a URL and receive data for an AI pipeline without maintaining a scraper. With Context.dev, we provide scraping, crawling, and structured output through one API. You still decide which pages to collect and validate the data your application uses, but you do not maintain the retrieval and rendering infrastructure.
Apify gives you a different kind of control. You can run an existing Actor or write one to handle a particular site and extraction task. That flexibility helps when you need custom navigation or site-specific logic. It also means you may need to configure, update, and monitor the Actor as the site changes. An existing Actor can reduce that work, but its behavior and maintenance depend on its creator.
The billing models need separate scrutiny. A managed API may charge for requests, pages, or features, while an Actor workflow may incur platform usage charges and Actor-specific fees. Neither model has a cost advantage without a defined workload. The benchmark will compare billed cost per usable page and document the configuration used for each vendor. It will not treat engineering maintenance as a measured API charge.
Reproducing this benchmark yourself
We will publish a replication package with the fixture URLs or stable public test set, runnable harness code, pinned dependencies, and commands for running each workload. The package will include per-vendor configuration for Context.dev, Firecrawl, Apify, Zyte, and ScrapingBee. Configuration files will record rendering options, output formats, timeout and retry budgets, and any paid features enabled. Credentials will remain private, but the settings needed to reproduce each request will be public.
We will publish the scoring rubric and the evidence behind each result. Dated run logs will show sample sizes, every request's outcome, failure categories, and raw outputs where redistribution is permitted. The rubric will define what counts as usable output and how JSON and Markdown quality are scored. Pricing snapshots from the test dates will identify tiers, credit multipliers, and billable retries used to calculate cost per usable page. Results will remain pending until we run the harness and publish those records.
We will use public fixtures rather than arbitrary live targets because a changing page or access rule can make another developer's run incomparable or unauthorized. Before testing each URL, we will check its robots.txt rules and applicable terms of service. Blocked-page tests will use fixtures we are permitted to access, and we will document what protection each fixture presents. The published results will separate fixture changes and anti-bot variation across runs from API failures. A result on an authorized fixture will not establish how any vendor performs against every protected site.
Workload-specific verdicts
We have not run the benchmark, so we cannot name a winner for any workload yet. Once tests finish, we will give separate verdicts for static pages, JavaScript-rendered pages, and permitted bot-protected pages. Each verdict will use dated raw results and disclose sample sizes, failures, usable-output rates, latency, output quality, and cost per usable page.
For static pages, we will compare whether each API returns the required content without unnecessary rendering costs. For JavaScript-rendered pages, we will check whether the output includes content added after scripts run and preserves the fields an AI agent needs. For blocked pages, we will report the access conditions and variation across runs rather than treating a single successful request as proof of reliability.
Context.dev offers a managed API for scraping, crawling, and structured output without internal crawler infrastructure to maintain. Apify's Actor-based model offers more control over custom extraction logic. Neither distinction establishes a benchmark winner. We will disclose where Context.dev wins and where it trails Firecrawl, Apify, Zyte, or ScrapingBee, based on the published results rather than revising the verdict to fit a marketing claim.
FAQ
When will results be published?
We will publish results after running the harness and checking the raw records. We have not set a publication date, and we will not name winners before the tests run.
How will you disclose sample sizes and failures?
We will report attempted pages and usable outputs for each vendor and workload. The raw results will include failed attempts, such as timeouts and unusable responses, rather than counting only successful pages.
How does this differ from your earlier articles?
Our API roundup surveys products, while our Playwright TCO article examines infrastructure costs. This benchmark will publish a runnable harness and score API outputs against the same disclosed fixtures and rubric.
How will you calculate cost per usable page?
We will divide the total charges for a workload, including failed requests and retries, by its number of usable outputs. We will publish the pricing assumptions and vendor configurations used in that calculation.
How should I choose before results are available?
Consider Context.dev when you want scraping, crawling, and structured output through one managed API. Consider Apify when you need to select or customize Actors, and use DIY Playwright when you need direct browser control and can maintain the infrastructure. These are fit considerations, not benchmark verdicts.
What this benchmark is built to answer
Choose scraping infrastructure for AI agents using evidence you can reproduce on the pages your agents need. A static page, a JavaScript-rendered page, and a blocked page can place different demands on retrieval and extraction. Until the tests run, we cannot say which API performs best on any of them.
We will publish dated results and raw data in this article, alongside the fixtures, harness configuration, and scoring rubric needed to check our findings. Run the harness against your own permitted fixtures, or try Context.dev's API on the pages your agent needs to process.