What is Selenium?
An open-source framework that drives real web browsers programmatically, originally built for end-to-end testing and now widely used for scraping JavaScript-heavy sites.
Also known as: Selenium WebDriver
Selenium exposes a single API (the WebDriver protocol) that controls Chrome, Firefox, Edge, and Safari. Your script tells the browser to open a URL, click an element, fill a form, or read the rendered DOM, and Selenium translates each call into the browser's native automation channel. Because it drives an actual browser, every page behaves the same way it would for a human visitor: JavaScript executes, fetch calls fire, cookies persist.
For scraping, that fidelity is the main draw. A static HTTP fetch of a React or Angular page returns a near-empty shell, the data only arrives after the bundle runs. Selenium waits for the script to settle and then hands you the fully hydrated DOM. The cost is speed: a Selenium session is one to two orders of magnitude slower than a raw HTTP request, and you pay for the memory of a full browser per worker.
Selenium is older than Puppeteer or Playwright and supports more languages (Python, Java, Ruby, C#, JavaScript) which is why it remains popular in QA shops and any team with a non-Node stack. For greenfield scraping work most engineers now reach for Playwright first, but Selenium's ecosystem of bindings and plugins keeps it relevant.
In the wild
- Driving Chrome to log in to a partner portal and download a daily report
- Running an end-to-end test suite in CI against a staging environment
- Scraping a single-page app whose data only appears after a few hundred milliseconds of XHR