Error, slice and fairness analysis

How this question breaks down

1. 1ErrorsCore18 minRead a sample of the model's mistakes, sort them into causes, count the causes, and fix the biggest one you can fix. It is the cheapest way to find what a new metric, feature or model would actually buy.
2. 2SlicesCore21 minMeasure the model on named groups (new customers, thin credit files, each market) as well as overall, because an aggregate gain can hide a regression in a group that matters. Then make those per-slice numbers a release gate.
3. 3FairnessCore19 minFairness metrics are slice metrics on protected groups with one extra difficulty. You have to choose which rate to equalise (approvals, error rates, or what a score means), and when the groups' base rates differ you cannot equalise all of them.

Work through the parts in order and you’ve covered the full design. Each part is one topic.