Beyond AveragesScience in practice

Credit scoring: fit a model, read the ROC curve, set the cutoff

Interactive credit scoring demo: a logistic regression fitted live on 2,000 synthetic loan applicants, with ROC curve, AUC, Gini and KS, a confusion matrix at any threshold, adverse-action reasons for a declined applicant, a profit-maximising approval cutoff, and a calibration plot.

A credit scorecard answers one question for every applicant: how likely is this loan to default? Logistic regression remains the industry workhorse for that question because its coefficients read as odds ratios and its output is a probability that can feed pricing, provisioning and capital models. This module fits one live, in the browser, on 2,000 synthetic personal-loan applicants described by debt-to-income, credit utilisation, length of credit history and recent delinquencies, and shows the fitted coefficients next to the true ones that generated the data.

Validation is where most of the practical judgement lives. The model is trained on 70% of applicants and measured on the 600 it has not seen. The ROC curve and its summary, AUC (or Gini = 2·AUC − 1 in credit-industry convention), measure how well it ranks bad borrowers above good ones; at the defaults AUC is about 0.83. The KS statistic marks the score where the two groups are furthest apart. A threshold slider turns scores into approve/decline decisions and fills in the confusion matrix, approval rate and bad rate among approved loans, and a "why was this applicant declined?" box lists the principal reasons for the declined applicant closest to the cutoff: the kind of specific reasons US lenders must give under ECOA and Regulation B.

The cutoff itself is an economic decision, not a statistical one. With a profit per good loan and a loss per default, a loan is worth booking only if its default probability is below gain / (gain + loss): 0.17 at the defaults ($1,200 profit, $6,000 loss), far from the naive 0.5, which would approve almost everyone. A reliability plot then checks calibration: a model can rank well and still mis-state default rates, which matters for IFRS 9 and CECL provisioning and for risk-based pricing. The data are synthetic and the model is illustrative, not a validated scorecard.

Profit-optimal approval cutoff

decline if p̂ > t*, t* = G / (G + L)

p̂ is the predicted probability of default, G the profit on a good loan and L the loss on a default. For a calibrated model, approving is worth it whenever (1 − p)·G > p·L, which rearranges to p < t*.

Frequently asked questions

What is a good AUC or Gini for a credit scorecard?

Retail application scorecards typically land around AUC 0.70–0.80 (Gini 0.40–0.60); behavioural scorecards that use account history often score higher. AUC only measures ranking, so it should always be read alongside calibration and the business outcome at the chosen cutoff.

Why not approve everyone with a predicted default probability below 50%?

Because the costs are asymmetric. If a default loses five times what a good loan earns, approving an applicant only pays when their default probability is below 1/6, about 17%. The right cutoff comes from the profit and loss per loan, not from the symmetry of the probability scale.

What is the difference between discrimination and calibration?

Discrimination (AUC, Gini, KS) is whether the model ranks risky applicants above safe ones. Calibration is whether a predicted 8% default rate actually produces about 8% defaults. Approve/decline decisions need good ranking; pricing, provisioning and capital need good calibration too.

Interactive explorer

The live model with sliders, charts and real-time statistics loads on this page. If it does not appear, enable JavaScript or open https://www.beyondaverages.org/credit-scoring in a modern browser.

Related modules

  • Dice & convolution: Interactive 3D dice simulator. Roll up to five dice and watch the empirical sum histogram converge to the exact convolution probabilities: uniform to triangular to bell-shaped, the central limit theorem made visible.
  • Cards & hypergeometric: Interactive card-dealing simulator. Deal hands from a 3D deck and compare empirical frequencies of aces, pairs and colors against exact hypergeometric probabilities, and see why a deck's memory matters.
  • Frequency: Interactive Poisson distribution explorer for insurance claim counts and hospital admissions. Move the rate λ and watch P(X=k), the mode, and the probability of zero or extreme counts update live.
  • Severity: Interactive loss severity explorer comparing Log-Normal, Pareto (heavy-tail) and Gamma distributions. Adjust parameters and see the mean, median, 95th and 99th percentile claim sizes, and the share of total cost from the largest 5% of claims, update in real time.