Call me Ahab: Webscraping tooling, approaches and dataset obsession

About this event

Presentation - Call me Ahab: Webscraping tooling, approaches and obsession with white whale datasets Webscraping isn't one thing, or done for one reason. Drawing on scrapers built in very different forms, from once-a-minute API pollers to fleets that run for years to keep full mirrors, this talk breaks webscraping down across spectra and considerations. Its opportunity is as wide as the internet, and it gets harder to reach every year. Unbounded wants come with unbounded costs: time, effort, money, and risk. We'll cover libraries and architectural choices, where agentic and ML tools can fit, and scaling traps. You might already have a white whale, too.

Attending