Resampling and weighting.

Hatchway is a fictional marketplace for secondhand furniture. Its trust team wants a model that scores each new listing for the chance it is a scam: a fake sofa, a deposit request for a table that does not exist. A 12-week training window holds 24,000,000 labelled listings, of which 48,000 were confirmed scams. That is 2 in every 1,000. The two weeks after the window are held back for validation and test.

Training on all of it as logged works, but it is wasteful. Almost every row is a legitimate listing that the model classifies correctly after the first few passes, so most of the compute goes into confirming what the model already knows. A weak learner can also settle on predicting 'not a scam' for everything, a failure accuracy hides, as why imbalance hurts shows.

There are four ways to push back. Use fewer negatives (undersampling). Use more positives, either as exact copies or as synthetic points built between real ones (oversampling, SMOTE). Leave the rows alone and make each positive count for more in the loss (class weights). Or change the loss so that examples the model already gets right count for less, whichever class they are in (focal loss).

Each one has the same side effect. The model learns the class mix it was shown, not the one it will face. A model trained with one scam in 14 rows will tell you a listing is 30% likely to be a scam when the real chance is about 1%. Moderators sorting a queue by score will not notice, but anything that reads the number as a probability will: a threshold, an expected-loss calculation, a dashboard of predicted scam volume.

Every scam survives the sample and each legitimate listing survives with probability w. So among listings that look alike, the sample holds the same scams but only a fraction w of the honest ones, and the odds of a scam in the sample are the true odds divided by w: odds′ = odds ÷ w. A model fitted to the sample learns odds′. To get back, multiply by w: odds(q) = w × p′ ÷ (1 − p′). Turning odds back into a probability gives q = p′ ÷ (p′ + (1 − p′) ÷ w), and taking logs gives the same thing as a constant shift, logit(q) = logit(p′) + ln w.

The one assumption is that the sample keeps negatives at random with respect to the features. If the keep rate depended on, say, the listing's category, the shift would differ by category and a single ln w would be wrong.

Random oversampling repeats positive rows until the mix looks the way you want. It adds no information, and a deep tree or a big network can simply memorise the repeated rows. SMOTE (Chawla et al., 2002) tries to do better by building new positives. For a scam x_i it picks one of its k = 5 nearest scam neighbours, x_nn, draws a gap uniformly from [0, 1], and emits x_new = x_i + gap × (x_nn − x_i): a point somewhere on the segment between the two.

A worked case with two numeric features. Scam A asks ₹4,000 and comes from a seller account 2 days old. Its neighbour, scam B, asks ₹6,000 from an account 6 days old. With gap 0.3 the synthetic scam asks 4,000 + 0.3 × 2,000 = ₹4,600 from an account 2 + 0.3 × 4 = 3.2 days old. That is a believable listing.

Now the cases where it breaks. The seller's city is Pune for A and Delhi for B; there is no city 30% of the way between them. The listing's text embedding can be interpolated arithmetically, but the midpoint of two scam descriptions is not a description of anything. Two scams whose neighbourhood is full of honest listings produce a synthetic scam in the middle of them, teaching the model that honest listings look suspicious. And SMOTE run before the train and test split finds neighbours in the test set, so test information leaks into training and the offline numbers look better than production will.

Focal loss multiplies cross-entropy by (1 − p_t)^γ, so the factor is the model's own error raised to a power: FL = −α(1 − p_t)^γ log p_t. The table leaves α out; with γ = 2, an example the model already gets right at p_t = 0.9 counts 100 times less than it would under plain cross-entropy. Lin et al. (2017) built it for one-stage object detectors, which score on the order of 100,000 candidate boxes per image, nearly all of them obvious background. There γ = 2 with α = 0.25 worked best. It targets easy examples of either class, not the minority class as such. Its outputs lose cross-entropy's guarantee of calibration and there is no closed-form prior correction, so recalibrate on natural-rate held-out data (Platt or isotonic). The distortion is not always harmful: Mukhoti et al. (2020) found focal loss often left deep networks better calibrated than cross-entropy did.

For Hatchway's GBDT it is rarely worth the trouble. It matters where the bulk of the examples are trivially easy and swamp the gradient: dense detection, or scoring long candidate lists.

If Hatchway split scams into 30 subtypes, a few common and most rare, inverse-frequency weights would go to extremes: the rarest subtype would get a huge weight from a handful of rows. Cui et al. (2019) weight each class by the inverse of its effective number of samples, (1 − β^n)/(1 − β) for n examples and β just below 1. The effective number grows almost linearly for small n and levels off for large n, because the 10,000th example of a common class mostly repeats what the model has seen, while the 10th example of a rare class still adds something new.