Ship a Production Retrieval System

• ✓Build a five-service retrieval system — candidate generation, feature store, eligibility rules, ranking and an orchestrator with a latency budget
• ✓Measure the gap between an offline metric and the same metric served through a running system
• ✓Reproduce training/serving skew deliberately, and measure what a drifted id mapping costs in recall
• ✓Detect a stale index that degrades recommendations silently, with no error and no alert
• ✓Measure how often hand-written eligibility rules overrule a learned ranker, and price the rule
• ✓Choose and defend a failure policy for a ranker that exceeds its latency budget
• ✓Attribute every lost point of recall to a named cause, then fix the two that are cheapest

Build the system that surrounds a recommendation model in production — the services, the batch job, and the latency budget — and learn to find the quality loss that lives in the plumbing rather than the model. Assumes you have already built a retrieval engine; do Build a Two-Tower Retrieval Engine first if two-tower models and ANN indexes are new to you.