Subgroup-stratified, calibration-aware fairness auditing as a pip-installable library — the uncertainty-aware statistical machinery that mainstream toolkits do not expose as first-class functions.
| Capability | AIF360 | Fairlearn | fairscope |
|---|---|---|---|
| Per-subgroup AUC confidence interval (DeLong) | — | — | ✓ |
| Per-subgroup Expected Calibration Error | — | — | ✓ |
| Subgroup significance test + correction | — | — | ✓ |
| Subgroup-stratified recalibration | partial | — | ✓ |
| Five-axis cross-platform protocol (CPFE) | — | — | ✓ novel |
| Per-node / federated audit | — | — | ✓ |
Verified against AIF360 0.6.1 and Fairlearn 0.14.0 (checked 2026-06). Those toolkits do bias mitigation; fairscope does uncertainty-aware measurement — they are complementary.
A fairness number without an uncertainty band is a coin flip you can't see. fairscope treats fairness auditing as a statistical measurement problem: for every protected subgroup it reports not just an accuracy or AUC, but a DeLong confidence interval, a calibration error, and whether the gap between subgroups is statistically significant after multiple-comparison correction. It then hands you a subgroup-stratified recalibration interface to actually fix the calibration it just measured — with the error reported before and after. One from_scores(...) call produces a report with tables, forest plots, and reliability diagrams.
AIF360 and Fairlearn are excellent at bias mitigation — they change the model. Neither exposes the uncertainty-aware measurement a reviewer actually asks for: a per-subgroup AUC with an analytic confidence interval, per-subgroup calibration error, or a corrected significance test on the gap between groups. fairscope fills exactly that gap, and adds one genuinely new protocol — the five-axis Cross-Platform Fairness Evaluation — on top. Everything except CPFE ports a documented method and cites its source; it invents no new mathematics.
Point it at scores and a protected attribute; it returns an audit. Under the hood, each domain dispatches the same statistical core, then layers on domain-specific reporting.
It turns the research program's methods into something anyone can pip install: the exact per-subgroup, uncertainty-aware statistics used across the diabetes, mortgage, and mental-health-NLP papers, packaged once, tested to a reference value, and cited to source. Fairness measurement stops being bespoke notebook code and becomes a reusable, reviewable library.
Complementary to AIF360 / Fairlearn, not a replacement — measurement, not mitigation. The federated module provides no privacy guarantee; committed fixtures are small, synthetic, and labelled as such.
fairscope ports methods from the author's own papers — it invents no new mathematics, and each function cites its source: the diabetes external-validation study (IEEE CIPHER-2026), the five-axis CPFE mental-health NLP audit, the federated-learning diabetes work, and both mortgage-lending papers.