Language
English (en)
Date Uploaded
Summer 8-31-2026
Activity source
Original (with AI assistance). Anthropic’s Claude Code was used as a development tool for drafting materials, starter code, and tests, under the author’s direction and review; Google’s Gemini API is the LLM students integrate as a component of the system they build.
Summary
Students build a streaming vandalism detector for live Wikipedia edits in which a fast rule-based tier decides which few of ~1,500 edits per minute are worth escalating to a slow, rate-limited real LLM (Gemini) — confronting the cost, latency, and failure modes of putting AI inside a production data pipeline.
Extended Summary
Modern data systems increasingly place an LLM in the loop, but an LLM is the slowest, most expensive, and least reliable component such a system will ever call. This assignment makes that tension concrete. Students implement a two-tier cascade over Wikipedia’s live edit stream: a Kafka + Flink triage layer scores every edit with cheap heuristics and escalates only ambiguous cases to Google’s Gemini API on its free tier — where they encounter genuine rate limits (HTTP 429s) and must implement exponential backoff, budget-constrained prioritization (which ambiguous edits most deserve one of ~15 LLM calls per minute?), and an audit log. A batch auditor then checks the LLM’s verdicts against Wikipedia’s own ground truth (community-reverted edits). Students submit a design brief before coding, checkpoint reflections during, and a scaling-up design memo after. The assignment assumes no prior Kafka or Flink experience (a Dockerized environment and warm-up are provided) and takes roughly 9–12 hours over two weeks. Under the course’s AI-collaborative policy, students are expected to use AI coding assistants and to document how they used them — so the assignment exercises both building with AI and building systems around AI. A companion CS2-level version (data structures only, no infrastructure) is submitted separately.
Student Learning Objectives
- Design and implement a streaming pipeline with Kafka and Flink (keyed state, timers, side outputs).
- Integrate a rate-limited external LLM API into a real-time system, handling live 429/5xx errors with exponential backoff and jitter.
- Reason quantitatively about throughput, latency, backlog, and cost in a tiered architecture, and defend design decisions under an explicit call budget.
- Audit an AI component’s decisions against ground truth with a batch job, and characterize where the cascade succeeds and fails.
- Collaborate effectively and accountably with AI coding assistants on a nontrivial systems task.
Assignment Type
Out-of-Class
Course level
500 or above
Used in course?
no
Type of Student-AI Collaboration Required
Collaboratively generated by human and AI
Type of AI Task(s)
Problem Solving
Second Type of AI Task
Data Analysis
Third Type of AI Task
Other (enter in Notes to Administrator field)
Uploader/Author Affiliation
Faculty
Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.
Document Type
Teaching Material
Recommended Citation
Goodman, Ilan, "Guardians of the Record: A Two-Tiered Streaming Cascade with Kafka, Flink, and a Real LLM" (2026). Generative AI Teaching Activities. 19.
https://openscholarship.wustl.edu/ai_teaching/19