Sources & methodology

How we collect & process data

• Collection. A scheduled scraper reads public city and school-district websites, document APIs, business-directory records, news feeds, and state/federal data services. It identifies itself with a contact address, obeys each site's robots.txt, and waits at least 3 seconds between requests to the same host.
• Cleaning. Text is normalized — HTML entities decoded, Unicode standardized, invisible characters removed, whitespace collapsed — so summaries read cleanly.
• Deduplication. Repeated items are merged: documents by web address, news and agenda items by title and date, school records by school (combining the Future Ready PA profile with federal enrollment figures).
• Summaries. News snippets are tidied (mastheads and ALL-CAPS letterheads removed, trimmed to a sentence or two). Budget briefs use extracted public facts. If OpenAI summarization is enabled on the server, it can only summarize those verified facts.
• Verification. Records are schema-validated, budget figures are sanity-checked (implausible values are dropped rather than shown), and a weekly link checker flags documents whose links go dead.

Known limitations

• Budget totals. CASD budget totals are extracted from the official final budget forms when the tables can be verified. The city page uses the latest detailed official city budget PDF found, plus newer official minutes for the 2026 adopted-budget status.
• Meetings & votes. We don't yet parse a meeting schedule or vote records; we link to the official agendas, minutes, and BoardDocs instead.
• CASD document host & robots.txt. The CASD file host returns an error for its robots.txt rather than rules. Under RFC 9309 §2.3.1.4, an unavailable robots.txt means no crawl restrictions are published, so we treat those documents as link-only — we show the official link for you to click through, and do not download or re-host the files.
• BoardDocs. The CASD BoardDocs site disallows automated access in its robots.txt, so we honor that and link to it directly rather than scraping it.
• Accuracy. Data is collected automatically and may contain errors, delays, or omissions. Always confirm anything important with the linked official source.

Source status

Curated official links

Financial documents

Budget, audit, and annual-report documents we've indexed. The full set of 743+ documents (agendas, minutes, packets, policies) lives in the official libraries: City of Coatesville and CASD document library.