Supermemory builds the memory engine for AI agents. Developers connect it through an API, plugins, or MCP, and it extracts, stores, and recalls context so agents keep learning across sessions instead of starting from zero.
A lot of that context comes from the web. Supermemory's connectors crawl websites and index documents on behalf of its users, which made the crawler one of the first pieces of infrastructure the team wanted to consolidate. Context.dev now runs it.
One API instead of several
Mahesh Sanikommu, founding engineer and product lead at Supermemory, found Context.dev through Yahia's posts on X. The problem he wanted to solve was sprawl: website extraction, PDF extraction, and crawling each ran through a different tool.
"The main thing is we want to have one stop solution to solve all our problems coming from website extractions, PDF extractions, also our web crawler."
Context.dev covers all three from one API and one key, so the team could replace the stack piece by piece rather than add yet another vendor to it.
Benchmarked by an agent, not by hand
Supermemory did not integrate Context.dev manually. Mahesh pointed an agent at the onboarding flow and let it run the evaluation:
"Honestly, agent onboarding is pretty good. Our agent was able to test and benchmark Context.dev with our existing solutions we use. Results are really promising, especially how the API is exactly what we would expect a service like Context should be."
The benchmark ran against the crawling and extraction services Supermemory was already using, on the team's own workloads. The results were strong enough that the team moved straight to production:
"We went from trying to prod live in a few hours and never looked back."
The Web Crawler connector
The clearest place Context.dev shows up in Supermemory is the Web Crawler connector. A user hands it a start URL and a crawl depth, and Supermemory crawls the pages, turns them into documents, and keeps them synced on a schedule.
Each sync in the history above is a crawl Context.dev performed: pages fetched, rendered, and returned as clean content ready to index. Supermemory keeps the connector UI and the memory layer, and Context.dev handles the part that used to be a crawler the team maintained themselves.
The outcome
Supermemory did not ship a new feature with Context.dev. It replaced the infrastructure behind existing ones, mainly its web crawler, and simplified the stack in the process:
- Trial to production in hours: an agent benchmarked Context.dev against the existing services and the team went live the same day.
- One provider for web data: website extraction, PDF extraction, and crawling now run through one API.
- A crawler the team no longer maintains: the Web Crawler connector's syncs run on Context.dev.
Asked for feedback after going live, Mahesh kept it short: "Everything is perfect. Nothing to complain right now."
Crawling websites into memory or a knowledge base for agents? Explore web crawling with Context.dev, the Web Scraping API, and RAG pipelines.