Self-Healing Scrapers: Let One Agent Write Another Agent's Instructions
Self-correcting AI agents are a practical way of fetching data from various webpages. Instead of a scraper that breaks the moment a website changes, one AI agent researches the site, gets the data and corrects itself when the page changes. This article walks through building exactly that, end to end, with Claude Code. The business need Let's imagine a situation where parents have three kids in three different schools. None of the schools agree on how to publish a timetable. The formats are different: one puts up clean, per-class HTML pages. Another exports its entire grade level as a single PDF. What the parents want is one combined, printable schedule covering all three kids. Writing an HTML scraper for the first school and a PDF scraper for the second is an afternoon's work. What is more, the timetable structures may change in time. The process The solution I landed on with Claude Code was a self-correcting process based on LLM agents. There are two agents involved. T...