Monitoring and continual learning

How this question breaks down

1. 1DriftCore20 minA model is trained on a snapshot of the world and served in the world as it is now. Drift is the gap between the two. The inputs can change, the base rate can change, or the link between inputs and outcomes can change. Only the last one always hurts, and it is the hardest to see without labels.
2. 2RetrainingCore17 minEvery model gets older from the moment its training data ends. How often to retrain is a measured trade between the quality lost per day of age and the cost and risk of each run. A retrain is also a deploy, so it needs the same gates, and a trigger only helps if the pipeline behind it is fast.
3. 3Online learningCore21 minOnline learning updates the model from each labelled event within minutes instead of retraining from a snapshot. It pays when items and tastes live for hours, and it costs a streaming label join, per-feature learning rates, careful parameter sync and guards against learning from broken data.

Work through the parts in order and you’ve covered the full design. Each part is one topic.