Building a Tech Zeitgeist Machine: News Harvesting, Dirty Data, and the Audacity of Mr. Spruce

NOTE: Architecture is in active evolution: events through Kafka, bytes via a Media Gateway into MinIO, analytics in ClickHouse, and a thin Read API for the GUI. Ingest writes WARCs; summaries are sidecar objects keyed by the exact text hash. This post tracks the big ideas; fine‑grained topics (topics/