PR-AUC is the area under the precision-recall curve.
It focuses entirely on how well the model handles the positive class.
It is the better choice when positives are rare.
Unlike ROC-AUC it is not fooled by a huge pile of easy negatives.
Fraud and disease detection are typical cases where you should prefer it.
PR-AUC plots precision against recall and measures the area underneath.
Both of those numbers look only at the positive class.
That is exactly why it survives heavy imbalance.
Adding a million more easy negatives barely changes the curve.
The same change would flatter ROC-AUC considerably.
So for fraud, disease and defect detection this is the more honest number.
Its baseline is not 0.5, which surprises people.
A random model scores roughly the fraction of rows that are positive.
With one percent positives, random guessing scores about 0.01.
So a PR-AUC of 0.4 can be excellent, depending on the base rate.
Always report the positive rate beside it, or the score cannot be read.
It cannot be compared across datasets with different balance.
Average precision is the usual way to compute it and is close enough to the same thing.
Like ROC-AUC it is threshold-free and judges the quality of the ranking.
Use it to choose a model, then choose the threshold as a separate step.
Precision plotted against recall, as the threshold sweeps from high to low.
Neither has anywhere in it, and that single fact is why this curve behaves under imbalance where ROC does not.
The area beneath is usually computed as average precision, which is a sum rather than an interpolation.
The sum form is used because linearly interpolating a PR curve is optimistic: the curve between two points is not a straight line.
This is the part that is most often got wrong.
A random classifier scores the class prevalence, not one half.
With 1% positives, a random model scores 0.01, so a PR-AUC of 0.4 is a strong result, and it would look like failure to anyone reading it as an ROC-AUC.
Always quote the prevalence next to the score. Without it the number cannot be interpreted at all.
ROC divides false positives by the number of negatives, which is huge, so a flood of false alarms barely moves the curve.
PR divides false positives by the number of flags raised, which is small, so the same flood halves the precision.
Use ROC-AUC when the classes are near balanced, or when you genuinely care about performance on the negative class.
Use PR-AUC when positives are rare and the cost is in the false alarms you make chasing them, which is most of fraud, retrieval, moderation and screening.
One question on this card. Every option explains itself once you pick, right or wrong.
55 concepts in this module, then a graded test with every answer explained.