loan-approval-s1

A small, fast, calibrated "System 1" model that predicts whether a home-loan application in the Finance Loan Approval Prediction dataset would have been labelled approved. It returns a single-pass typed answer (approve / reject), a calibrated probability, a confidence score and a needs_review flag.

⚠️ This is a predictive estimate, not a credit decision. It reproduces patterns in historical labels from a 614-row public dataset of unknown provenance. It must not be used as the sole basis for approving or denying credit, and it has not been validated for any real lending population. See Limitations.

Architecture	extra trees (scikit-learn pipeline) + Platt (sigmoid) calibration
Hyperparameters	{"clf__max_depth": 3, "clf__min_samples_leaf": 3}
Artifact	model.skops (596 KB, safe-to-load skops format)
Latency	~5.8 ms per single applicant on a laptop CPU; no GPU
Decision threshold	P(approve) ≥ 0.39 (tuned for balanced accuracy on out-of-fold dev predictions)
Review flag	needs_review = true when confidence < 0.7
Excluded inputs	Gender, Married (protected attributes), Loan_ID
License	Apache-2.0 (code + weights); training data CC0

Why this design (and how it relates to Laya)

Laya is an open (Apache-2.0), non-autoregressive "System 1" decision model. It takes a text/JSON state plus typed questions and returns typed answers with calibrated probabilities in one forward pass (~33 ms), trained against strictly proper scoring rules. This model keeps the same contract: a typed answer, one fast pass, a calibrated probability, and selection by a proper scoring rule. It uses an architecture matched to the data, which is 11 structured columns and 614 labelled rows.

Fine-tuning Laya itself was considered and rejected. It is a transformer text encoder with hundreds of millions of parameters; this model has a few thousand tree nodes. Laya's fine-tuning recipe expects GPUs, every row would have to be serialised as text, and 491 training rows are far too few to adapt an encoder reliably. A compact tabular model is faster, runs on any CPU, is the standard strong baseline for small tabular data, and re-trains in minutes (scripts/train.py).

Usage

pip install "git+https://huggingface.co/om24485/loan-approval-s1"

from loan_s1 import LoanApprovalModel

model = LoanApprovalModel.from_pretrained("om24485/loan-approval-s1")
model.predict({
    "Dependents": "0", "Education": "Graduate", "Self_Employed": "No",
    "ApplicantIncome": 5000, "CoapplicantIncome": 1500,      # monthly
    "LoanAmount": 120, "Loan_Amount_Term": 360,               # thousands, months
    "Credit_History": 1, "Property_Area": "Semiurban",
})
# [{'prediction': 'approve', 'p_approve': 0.83, 'confidence': 0.83, 'needs_review': False,
#    'threshold': 0.39, 'model_version': '1.0.0', 'disclaimer': '...'}]

Inputs use the Kaggle schema. Any field except ApplicantIncome / CoapplicantIncome may be missing (None); invalid categories, negative amounts or a Credit_History other than 0/1 raise InputError. Gender and Married are accepted but ignored.

REST API

git clone https://huggingface.co/om24485/loan-approval-s1 && cd loan-approval-s1
pip install -r requirements.txt fastapi uvicorn
MODEL_PATH=. uvicorn api.app:app --port 8000
curl -X POST localhost:8000/predict -H 'content-type: application/json' \
  -d '{"applicants":[{"ApplicantIncome":5000,"CoapplicantIncome":0,"LoanAmount":120,"Loan_Amount_Term":360,"Credit_History":1,"Property_Area":"Urban"}]}'

Or docker build -t loan-s1 . && docker run -p 8000:8000 loan-s1. The repo also ships handler.py, so it can be deployed as a Hugging Face Inference Endpoint (custom handler, CPU instance) and called with {"inputs": {...applicant...}}.

Training data

• Source: Kaggle krishnaraj30/finance-loan-approval-prediction-data (CC0), a re-upload of the widely circulated "Dream Housing Finance" loan-prediction practice dataset. Original collection process, time period, geography and labelling policy are undocumented.
• Rows: 614 labelled (train.csv; 422 approved / 192 rejected, 68.7 % / 31.3 %) and 367 unlabelled (test.csv).
• Split: stratified 80/20 with seed 42: 491 dev rows for training, tuning and calibration; 123 held-out rows evaluated once. Exact IDs are in splits.json; the training file's SHA-256 is in config.json.
• Missing values: Credit_History 8 %, Self_Employed 5 %, LoanAmount 4 %, Dependents/Loan_Amount_Term/Gender 2 %. Missing credit history is kept as its own category ("unknown") because it is informative.
• Engineered features: log incomes, total income, EMI (LoanAmount×1000 / term), EMI-to-income, loan-to-annual-income, post-EMI balance, has-coapplicant flag, plus Education, Self_Employed, Property_Area, Dependents.

Method

1. Per-architecture hyperparameter search (5-fold CV, ROC-AUC) on dev.
2. Comparison with 5×5 repeated stratified CV on dev. Every fold fits a sigmoid-calibrated model and scores its calibrated probabilities.
3. Selection by log-loss (a strictly proper scoring rule). Among models within one standard error of the best, the smallest one wins (random_forest was best; within 1 SE: random_forest, extra_trees; selected: extra_trees).
4. Class imbalance: cost-sensitive training (class_weight="balanced"), then calibration to restore true-prior probabilities, then a decision threshold chosen on out-of-fold predictions to maximise balanced accuracy. Balanced accuracy, per-class F1 and PR-AUC for the minority (reject) class are reported.
5. Final calibrated fit on all dev rows; one evaluation on the held-out test.

model	CV log-loss ↓	CV ROC-AUC ↑	Brier ↓	ECE ↓	size KB	ms/row
random_forest	0.498 ± 0.011	0.757 ± 0.012	0.161	0.086	1779	5.9
credit_history_rule	0.500 ± 0.011	0.697 ± 0.012	0.160	0.041	3	0.8
extra_trees (selected)	0.503 ± 0.010	0.739 ± 0.011	0.162	0.077	436	5.9
hist_gradient_boosting	0.510 ± 0.010	0.735 ± 0.013	0.165	0.092	137	5.3
mlp	0.530 ± 0.007	0.718 ± 0.010	0.173	0.093	32	1.5
logistic_regression	0.535 ± 0.008	0.718 ± 0.011	0.173	0.098	5	1.4
decision_tree	0.546 ± 0.008	0.710 ± 0.014	0.180	0.101	7	1.4
prior_baseline	0.622 ± 0.001	0.500 ± 0.000	0.215	0.004	4	1.4

prior_baseline always predicts the base rate. credit_history_rule is a logistic regression on Credit_History alone.

Evaluation on held-out test (n = 123, never used for training, tuning, calibration or threshold choice)

metric	model	95 % bootstrap CI	credit-history-only baseline
ROC-AUC	0.842	[0.755, 0.912]	0.752
Log-loss	0.412		0.420
Brier score	0.125	[0.091, 0.161]	0.127
ECE (10 bins)	0.085		0.050
Accuracy @ 0.39	0.854	[0.789, 0.911]	0.854
Balanced accuracy	0.770	[0.687, 0.851]	0.770
F1 approve / reject	0.903 / 0.700		0.903 / 0.700
Recall reject	0.553		0.553
PR-AUC (reject class)	0.766		0.666

Confusion matrix at the threshold (rows = truth): rejected → 21 predicted reject, 17 predicted approve; approved → 1 predicted reject, 84 predicted approve.

Read this honestly. On this dataset Credit_History carries almost all of the signal. The model ranks applicants better than the one-feature rule (higher ROC-AUC, lower log-loss), which makes its probabilities and the review flag more useful. Its hard approve/reject decisions at the chosen threshold are identical to the rule's on this test split. The test set has 123 rows, so confidence intervals are wide.

Calibration. Cost-sensitive training distorts probabilities: the uncalibrated model's ECE is 0.231 (log-loss 0.532). Platt scaling brings it to 0.085.

Selective prediction. With review confidence 0.7, 88 % of test applicants are auto-labelled (accuracy 0.861); the rest are flagged needs_review (accuracy 0.800).

Feature importance (permutation, ΔROC-AUC on dev)

feature	importance
Credit_History_cat	0.2080 ± 0.0142
Property_Area	0.0491 ± 0.0097
Has_Coapplicant	0.0183 ± 0.0073
Education	0.0078 ± 0.0033
Loan_Amount_Term	0.0052 ± 0.0015
LoanAmount_log	0.0049 ± 0.0009
CoapplicantIncome_log	0.0046 ± 0.0024
Loan_to_Income	0.0041 ± 0.0005

Fairness checks

Gender and Married are not used. Adding them back does not help (dev CV log-loss 0.506 with vs 0.503 without; ROC-AUC 0.735 vs 0.739). Proxies can still exist: CoapplicantIncome and Has_Coapplicant correlate with marital status. Subgroup results on the held-out test follow. The groups are small, so treat them as indicative only.

group	n	actual approve rate	predicted approve rate	accuracy	false-approve rate
Gender=Female	25	0.56	0.72	0.84	0.36
Gender=Male	96	0.73	0.84	0.86	0.46
Married=No	44	0.57	0.73	0.80	0.42
Married=Yes	79	0.76	0.87	0.89	0.47
Property_Area=Rural	34	0.65	0.91	0.68	0.83
Property_Area=Semiurban	49	0.73	0.78	0.96	0.15
Property_Area=Urban	40	0.68	0.80	0.88	0.38
Education=Graduate	100	0.71	0.84	0.85	0.48
Education=Not Graduate	23	0.61	0.74	0.87	0.33

The model approves more often than the historical labels in every group, because the threshold favours recall on approvals. Rural applicants have the highest false-approve rate on this split.

Limitations

• Not a credit decision system. Outputs are statistical estimates of a historical label, not an assessment of creditworthiness, affordability or regulatory eligibility. Any real use needs human review, adverse-action reasons, local-law compliance (e.g. ECOA/Reg B, FCRA, EU AI Act high-risk obligations, RBI guidelines) and validation on the target population.
• Tiny, undocumented dataset. 614 rows of unknown origin, time period and labelling policy. The labels may encode past lender bias. Results do not transfer to other lenders, products, currencies or eras.
• Single dominant feature. Performance depends heavily on Credit_History. When it is missing the model is much less certain.
• Units. Incomes are monthly, LoanAmount is in thousands and the term is in months, as in the source data. Out-of-range inputs (e.g. annual income) extrapolate silently, beyond the basic validation.
• Small test set. 123 held-out rows; see the bootstrap intervals above.
• No drift monitoring is included. Re-validate before and during any deployment.

Reproducibility

python -m venv .venv && . .venv/bin/activate
pip install -r requirements.txt
python scripts/train.py --data path/to/train.csv --seed 42
python -m pytest tests

Trained with Python 3.11.15, scikit-learn 1.9.1, numpy 2.4.6, pandas 3.0.6. Load the artifact with the same scikit-learn minor version. LoanApprovalModel.from_pretrained refuses artifacts that contain non-scikit-learn/numpy types.

Files

model.skops (weights), config.json (threshold, features, versions, data hash), metrics.json (all metrics), model_comparison.csv, splits.json, test_predictions.csv, src/loan_s1/ (preprocessing + inference), scripts/train.py, api/app.py, handler.py, Dockerfile, tests/.

License

Code and weights: Apache-2.0. Training data: CC0 1.0 (as declared on Kaggle).