Exploration vs exploitation

How this question breaks down

1. 1BanditsCore20 minPick among a few options while still learning which is best. Regret is the price of learning, and ε-greedy, UCB and Thompson sampling are three ways to pay less of it. Contextual bandits make the choice per reader.
2. 2SlotsCore24 minReserve a few positions in a ranked feed for items the model is unsure about, fill them with a logged random policy, and graduate or retire each item once it has enough evidence. The questions are how many slots, where, for what, and how to use the data.
3. 3LoopsCore18 minA deployed model chooses the data it will be retrained on, so it can make itself, its users and its supply narrower each cycle. Four kinds of loop, how to measure each, and the few mechanisms that break them.

Work through the parts in order and you’ve covered the full design. Each part is one topic.