Self-Healing Scrapers: Let One Agent Write Another Agent's Instructions
A scraper usually breaks the moment a website redesigns its layout. This one doesn't — it researches the page itself, and re-researches it when something changes This article walks through building exactly that, with Claude Code. The business need Let's imagine a situation where parents have three kids in three different schools. None of the schools agree on how to publish a timetable. The formats are different: one puts up clean, per-class HTML pages. Another exports its entire grade level as a single PDF. What the parents want is one combined, printable schedule covering all three kids. Writing an HTML scraper for the first school and a PDF scraper for the second is an afternoon's work. What is more, the timetable structures may change in time. The process The solution I landed on with Claude Code was a self-correcting process based on LLM agents. There are two agents involved. The first, a research agent , studies a school's site and writes plain-English ins...