← All drillsAudit a Loan Model
What you'll be able to do- ✓Explain a single approval decision with a Shapley estimator you build, and compare it with permutation importance
- ✓Find the features that act as proxies for a sensitive attribute
- ✓Measure approval and error rates by group, with a bootstrap interval on every gap
- ✓Show that removing the sensitive column does not remove the disparity
- ✓Trade accuracy against parity with per-group thresholds, and say why not every fairness measure can be equalised
- ✓Write a model card that states the model's limits and who must review it
Audit a model the way a risk team does — explain one decision, measure how it treats each group with honest uncertainty, test the obvious fix, and document what cannot be fixed.
⌁ The model-risk and responsible-AI review that credit, insurance and hiring models go through before and after launch — FICO, IBM's AI Fairness 360, and the regulators who ask for explanations.
Start this internshipCreate an account to unlock the 4 sections, the workbench, and askThili.
BeginSections
1. Audit a Loan Model
🔒 locked2. Lesson 1 - Why did it say no?
🔒 locked3. Lesson 2 - Does it treat groups alike?
🔒 locked4. Lesson 3 - Fix what you can, write down what you can't
🔒 lockedDig deeper
📄A Unified Approach to Interpreting Model Predictions (Lundberg & Lee, 2017, arXiv:1705.07874)
paper📄"Why Should I Trust You?" Explaining the Predictions of Any Classifier (Ribeiro et al., 2016, arXiv:1602.04938)
paper📄Equality of Opportunity in Supervised Learning (Hardt, Price & Srebro, 2016, arXiv:1610.02413)
paper📄Model Cards for Model Reporting (Mitchell et al., 2019, arXiv:1810.03993)
paper🔗OpenML Adult dataset (id 1590, version 2) — origin Becker & Kohavi, UCI Machine Learning Repository, 1996
article