Benchmarks
Auditing built to hold up.
Bhala detects the concepts an audit turns on — bias, hate, and protected attributes — with a content hash you can recompute offline for every call. Detection is what we measure and what we sell; an edit that makes an attribute unreadable to one class of reader is not removal, and every number below states which reader it was tested against. Reproducible across counterfactual fairness, bias detection, hate speech, sentiment, intent, and cross-lingual transfer, all on public datasets.
Detect bias by alignment, not absorption
Most AI systems learn to detect bias by training on biased text. They learn the patterns — and in doing so, internalize them. That's why the standard academic probes (WEAT, StereoSet, BBQ) consistently find race, gender, and religion bias inside GPT-4, Gemma, Llama, InkubaLM. Those models had to learn the bias from web-scale data to detect it.
Bhala learns differently. Its encoder was trained on math, logic, and code — not the firehose of internet text that imprints associations like “Black names → loan denial” or “women → less competent” into other models' weights.
When experts hand us labeled examples — Census-coded names, audited fair-lending cases, hate-speech corpora reviewed by domain specialists — Bhala's structural geometry lets us read off the direction those examples define. That direction then fires on any new text you submit. We supply the labels; Bhala supplies the geometry. The encoder doesn't have to believe the examples are biased — it just has to map them consistently.
A microscope resolves what you put under it. It doesn't catch the disease.
Empirical proof — corpus audit
We grep'd the entire training corpus for race-related discourse. Result:
- “hispanic”, “muslim”, “criminal”, “racism”, “slavery” — zero hits in our training corpus (the largest training source).
- “black” appears only as a color (sock, boxcar, cartridge, card suit) — never as a racial category.
- Bertrand-Mullainathan first names (Lakeisha, Tamika, DeShawn, Jamal, Tyrone…) appear in math problems about piggy banks, crayons, distance — neutral commerce contexts only.
- Math problems containing Black-coded names use fewer negative-outcome words than math problems containing White-coded names (−0.13 difference per example in the training corpus). The training data goes opposite the direction an internalized-bias hypothesis predicts.
The encoder cannot have learned what wasn't in the training data. Detection comes from supplied expert labels + structural alignment, not from imprinted prejudice.
In practice:The live /demo/audit endpoint returns an internal comparison index across six deployment domains (criminal sentencing, employment, online moderation, venture capital, police interaction, higher education) for any text you paste. The index has no legal threshold and is not a regulator metric: the EEOC 4/5ths disparate-impact ratio is a pool statistic — selection rates across a pool of candidates — so it cannot be computed from one pasted document. Sub-second per audit, with a content hash you can recompute offline. Predictions come from labeled-example projection, not from baked-in associations.
- Erasure is not removal. Detection is the product. Each clue on a resume — the ZIP code, the school, the first name — is a separate channel. Stripping out three directions (“rank-3 erasure”) leaves race readable at 0.947-0.993 on the same model, 0.949-0.998 on a channel never used to fit the edit, and 0.996-1.000 cross-channel — wiped using one clue, still readable from a different one (4 models x 600 real resumes). Removing race from ONE channel took 12 directions (surname 0.535 read by a straight-line classifier, 0.508 read by a stronger one) and left ZIP 0.9936, school 0.9958, first name 0.986 untouched. The edit is cheap (qualification 0.4677 -> 0.4672) but it moved top-10 selection share beyond a random-direction control on 1 of 4 models. Any earlier “erase → 0.500” number on this page — 0.500 being guessing at random — is withdrawn: it is circular. The direction is fit on one channel and the after-erasure check is trained on that same channel.
- Per-domain race direction. Each of the six regulated domains uses a race direction validated on labeled data for that domain. Synthesizing directions for domains we have not validated is under internal audit and not claimed.
- Internal comparison index (no legal threshold). We report a per-domain internal index and a 0–10 internal risk score. We do not report a disparate-impact ratio, a 4/5ths verdict, or a Title VII / ECOA conclusion for a pasted document: the 4/5ths rule is a pool statistic over selection rates across candidates and is not computable from a single document. Separately, ECOA disparate impact was deleted on 21 July 2026 (91 FR 21620), so no ECOA disparate-impact claim is available to make.
- Intersectional via set intersection. Race × gender per domain via set-intersection in the representation space. Reported on the same internal index, with no legal threshold attached.
Methodology:T_race / T_gender / T_religion operators trained on contrastive pairs from Bertrand-Mullainathan first names + US Census 2010 surnames (180K names total) embedded in 18 deployment-domain frames (10 frames per domain, grounded in audit literature). Receipt = unkeyed SHA-256(timestamp + key hash + input hash + scores). Every call writes a timestamped record to an append-only log you can retrieve by receipt id, with the input text and API key hashed rather than stored raw. Because the hash is unkeyed it attests the record's contents, not its origin — so it is not a signature, and the log is not yet on durable storage. The resume runs cited on this page use a corpus that is 41% software/IT and 23% finance, so those results are scoped to white-collar tech and finance screening, not to hiring in general.
Why this matters — and what it is not
EEOC 4/5ths rule, EU AI Act Article 27 high-risk system requirements, and NYC LL144 hiring-tool audits all measure DECISION OUTPUT differentials over a pool of candidates — not embedding cosines or stereotype-pair classification. We do not provide that regulator metric: a 4/5ths DIR requires selection rates across a pool and cannot be produced from one pasted document, so no per-document DIR, 4/5ths verdict, Title VII or ECOA conclusion appears anywhere in this product. (ECOA disparate impact was deleted on 21 July 2026, 91 FR 21620.) What we provide is per-document detection of protected attributes, in < 1 second, with a content hash you can recompute offline — feeding a pool-level analysis you run, not replacing it. Detection is the strong result: ZIP 0.984 and HBCU 0.983 — right about 49 times out of 50 — gender 0.979 (about 24 times out of 25) and name 0.945 (about 19 times out of 20), across 4 models. The 12 ZIP areas are 12/12 confirmed against 2020 Census DHC P8, HBCU status is set by statute, and the effect is dose-responsive: the more Black the neighbourhood behind the ZIP, the higher the score, in perfect rank order across all 12 (Spearman 1.000). Audits run on the six domains we validate today; extending to a new domain is validated during a pilot before it is used.
Re-measured 2026-07-29:The four detection scores above were re-run at full resume length — 8,000 characters / 2,048 tokens, which covers 90.8% of all text, on the same four backbones (gemma-2-9b, Mistral-7B-v0.2, Qwen2.5-1.5B, Qwen2.5-7B) with 500 matched pairs per axis. The earlier figures (ZIP 0.991 / HBCU 0.991 / gender 0.986 / name 0.964) were taken on 420 characters / 160 tokens — roughly 7% of a median resume — which left the injected demographic marker at about 17× its natural share of the input. Every number fell, and the name fell hardest, so the gap between the proxies (ZIP, school) and the name widened from 0.026 to 0.039. The corrected numbers make the case more strongly than the originals did: at honest input length the model reads ZIP and school more reliably than it reads the name, so an audit that redacts the name and stops there is auditing the wrong clue.
References
- Bertrand & Mullainathan (2003) AER — name-callback discrimination protocol
- Caliskan, Bryson & Narayanan (2017) Science — WEAT methodology
- EEOC Uniform Guidelines on Employee Selection Procedures (1978) — 4/5ths rule
- NYC Local Law 144 (2023) — automated employment decision-tool audit requirements
- IBM AIF360, Microsoft Fairlearn — DIR / SPD / equal-opportunity metric implementations (pool-level, not per-document)
- Rogue Scalpel, arXiv 2509.22067 — a random direction of the same size is a floor to beat, not a zero
- 91 FR 21620 (21 July 2026) — deletion of ECOA disparate impact
Bias Detection (removal claim retracted)
Detection is the product. Erasure is not removal: each clue on a resume — ZIP code, school, first name — is a separate channel, and scores below run 0.5 (guessing at random) to 1.0 (right every time). Stripping out three directions (“rank-3 erasure”) leaves race readable at 0.947-0.993 on the same model, 0.949-0.998 on a channel never used to fit the edit, and 0.996-1.000 cross-channel — wiped using one clue, still readable from a different one (4 models x 600 real resumes). Removing race from ONE channel took 12 directions (surname 0.535 read by a straight-line classifier, 0.508 read by a stronger one) and left ZIP 0.9936, school 0.9958, first name 0.986 untouched. The edit is cheap (qualification 0.4677 -> 0.4672) but it moved top-10 selection share beyond a random-direction control on 1 of 4 models.
Retraction:The previously published “100% correction rate across 28 protected dimensions and 15,966 sentence pairs” is withdrawn. The verification was circular in the same way “erase → 0.500” is circular — the direction is fit on a channel and the checking probe is trained on that same channel — and the lexical-swap control that would separate stereotype from group-word identity has not been run. The multi-layer probe behind these axes covered 10,064 sentence pairs (500 per axis) at 16 probed layers, not 15,966 pairs at all 60 layers. A random direction of the same size is a floor to beat, not a zero (Rogue Scalpel, arXiv 2509.22067). The resume corpus behind the erasure numbers is 41% software/IT and 23% finance, so the scope is white-collar tech and finance screening, not hiring in general.
| Benchmark | Cue type | Dimensions | Test pairs | Correction rateretracted — provenance only |
|---|---|---|---|---|
| BBQ Bias Benchmark for Question Answering | Demographic stated in disambiguated questions; inferred in ambiguous ones | 7 | 6,864 | 100.0% |
| StereoSet Stereotype measurement dataset | Direct stereotype associations with named demographic targets | 8 | 6,010 | 100.0% |
| CrowS-Pairs Crowdsourced stereotype pairs | Paired sentences with explicit demographic contrasts | 9 | 1,508 | 100.0% |
| WinoBias Gender bias in coreference resolution | Implicit No demographic terms; gender inferred from occupational stereotype via pronoun coreference | 4 | 1,584 | 100.0% |
| Combined | All cue types | 28 | 10,064 | 100.0% |
The Correction rate column is retracted and kept only for provenance: the checking probe was trained on the same channel the direction was fit on, so it measures the fit, not removal. Read the Dimensions and Test pairs columns as scope, not as a result.
Implicit vs explicit — where is the bias actually located?
Bias that's only readable when the demographic group is inferred (not stated) is where most output-only audits fail. Surface-level bias detection — keyword filters, demographic-mention checks — passes WinoBias-style sentences because no protected term appears. We can no longer cite the correction rate on the implicit slice as evidence that the intervention reads geometry rather than surface tokens: that rate is retracted, and the lexical-swap control that would rule out the probe keying on the group word itself has not been run.
12 categories · 28 test dimensions across BBQ, StereoSet, CrowS-Pairs, WinoBias
For compliance teams
- ·Detection is the product. Erasure is not removal: erasing the three strongest race directions leaves race readable at 0.947-0.993 on the same model, 0.949-0.998 when read from a part of the model the erasure was never fitted on, and 0.996-1.000 when read from a different part of the model than the one edited (4 models x 600 real resumes). At the low end of that range the model still gets race right about 19 times out of 20; 0.5 would mean guessing at random. Removing race from ONE part of the model took 12 directions (surname then reads at 0.535 with a simple reader, 0.508 with a stronger one) and left the proxies untouched: ZIP 0.9936, school 0.9958, first name 0.986. A proxy is something on the record that isn't race but gives it away anyway. A ZIP code is not race — but if a neighbourhood is 96% Black, writing that ZIP down tells a computer nearly as much as writing the person's race would. Same for the university: an HBCU is not a race, but it is a very good clue. Nobody has to intend this; the model works it out on its own. The edit is cheap (qualification 0.4677 -> 0.4672) but it moved top-10 selection share beyond a random-direction control on 1 of 4 models.
- ·A random direction of the same size is a floor to beat, not a zero (Rogue Scalpel, arXiv 2509.22067).
- ·"Erase → 0.500" is circular and is not cited as evidence of removal: the direction is fit on one part of the model and the score after erasure is measured on that same part. Collateral on the categories we did not touch is ±0.008 max (mean 0.002).
- ·The resume runs behind the erasure numbers are 41% software/IT and 23% finance — scope any hiring claim to white-collar tech/finance screening, not to "hiring" generally.
- ·These are published academic benchmarks. Production deployment into your bank or health system requires validation on your own internal text (loan memos, credit decisions, clinical notes) — which we conduct together during pilot.
- ·Two bias-editing methods were tested on identical data: a statistical baseline (published 2016) and our patented learned method. Both are available in production. Neither removes the attribute; both are measured by independent-classifier reclassification.
- ·Results reproducible by anyone with access to our model and the four public benchmarks. Full methodology is available under NDA.
Hate Speech Detection
Detects hate speech across 12 protected groups. Flags hateful content on social media, news comments, and adversarial inputs — and crucially, does NOT flag people discussing hate (counter-speech), in-group reclamation, or news reporting on slurs. Available over the API, priced and consumed like any other model endpoint.
Technical detail:Evaluated jointly across 11 corpora (~134K labeled examples), zero-shot — no Twitter pretraining. Scores here run 0.5 to 1.0: 0.5 means the model is guessing at random, 1.0 means it is right every time (technical name: AUROC). Full per-corpus scores, weakest included: CONAN 0.93, Civil Comments 0.92, Berkeley MHS 0.91, HateCheck 0.90, SBIC 0.88, Hate-Speech-18 0.84, TweetEval-hate 0.77, DynaHate 0.73, TweetEval-offensive 0.71, TweetEval-irony 0.58, MLMA 0.52. Eight of eleven above 0.70; MLMA (multilingual EN/FR/AR) at 0.52 is barely better than random guessing, and we say so rather than quoting only the top of the range. Live in production since 2026-05-02.
The audit engine — one programmable embedding space, and what each operation actually delivers
Detection is the product. The same space can also convert a concept toward a target or attempt to neutralize it, but only detection is a result we sell: erasure is not removal, and steering is per model and per method.
A random direction of the same size is a floor to beat, not a zero (Rogue Scalpel, arXiv 2509.22067). Counterfactual name-swap tests support a pool-level analysis you run; they do not yield an EEOC 4/5ths result for a single document.
Per-group detection rates
One probe per group, trained on HateCheck + CONAN examples for that group. Catch rate = % of hate posts flagged at 5% false-positive rate. AUROC is the 0.5-to-1.0 score used throughout this page: 0.5 is guessing at random, 1.0 is right every time.
| Group | Catch rate @ 5% FPR | AUROC | Test posts |
|---|---|---|---|
| Black people | 98.5% | 0.9908 | 66 |
| Disabled people | 98.4% | 0.9875 | 123 |
| Women | 90.0% | 0.9821 | 240 |
| Migrants | 94.3% | 0.9810 | 246 |
| Jewish people | 93.6% | 0.9807 | 109 |
| Muslims | 88.4% | 0.9733 | 319 |
| LGBT+ people | 81.1% | 0.9596 | 297 |
| POC (other) | 76.0% | 0.9305 | 75 |
| Generic (aggregate baseline) | 30.1% | 0.8205 | — |
The generic max-pool row is a single-probe aggregate baseline — per-group probes outperform it on every group.
11-corpus production coverage
The production model is trained jointly on 9 hate-speech corpora and evaluated on a test split from each one that the model never saw during training. Each row is a separate domain — adversarial templates, real-world social media, counter-speech, Twitter, extremist forums.
| Corpus | AUROC | Test n | Domain |
|---|---|---|---|
| CONAN | 0.9278 | 2,996 | counter-speech |
| Civil Comments | 0.9197 | 6,000 | real-world social |
| Berkeley MHS | 0.9090 | 8,996 | real-world social, 135K source |
| HateCheck | 0.9031 | 1,112 | adversarial templates |
| SBIC | 0.8796 | 11,992 | social bias frames |
| Stormfront | 0.8393 | 3,210 | extremist forum |
| TweetEval-hate | 0.7664 | 3,890 | Twitter — adversarial, unlike anything in training; broken |
| DynaHate | 0.7274 | 12,342 | human-and-model-in-the-loop adversarial |
| TweetEval-offens. | 0.7076 | 3,972 | Twitter offensive ≠ hate (separate task) |
TweetEval-hate AUROC 0.77 with no Twitter pretraining is the most distinctive result here — it matches HateBERT, which was fully fine-tuned on Reddit hate data, with zero task-specific pretraining.
vs. published baselines (overall AUROC)
| Model | AUROC | Method |
|---|---|---|
| Bhala (ours) | 0.9031 | Zero-shot generalization · 11 corpora · no Twitter pretraining |
| Detoxify (Unitary) | 0.91 | RoBERTa fine-tuned on Civil Comments + Jigsaw |
| Perspective API | ~0.87 | Google Jigsaw, commercial baseline |
| HateBERT (Caselli 2020) | 0.85-0.88 | BERT-base fully fine-tuned on Reddit hate corpus |
| HateXplain BERT | 0.83 | BERT-base fully fine-tuned with rationale annotations |
The decisive measurement
The decisive measurement: 'I hate X' (P=0.87) and 'Saying I hate X is bigoted' (P=0.10) share 80% of their surface tokens but receive 9x different hate scores. That use/mention distinction emerged from frozen weights — without a single hate-labeled training example.
HateCheck functional breakdown
Average hate score per statement type, where 0 is “definitely not hate” and 1 is “definitely hate” — this shows what the model distinguishes, not just whether it scores correctly.
| Statement type | Mean hate score | True label |
|---|---|---|
| Direct hate ('I hate X') | 0.866 | hateful |
| Slurs (raw) | 0.855 | hateful |
| Threats | 0.835 | hateful |
| Spell attacks (typos, leet) | 0.945 | hateful |
| Counter-speech (saying 'I hate X' is bigoted) | 0.099 | non-hateful |
| Counter-reference (saying hate is wrong, not using slur) | 0.223 | non-hateful |
| Positive identity ('I love X') | 0.296 | non-hateful |
| Slur reclamation (in-group) | 0.308 | non-hateful |
| Slur homonym ('dyke' as sea wall) | 0.268 | non-hateful |
| Profanity not directed at group | 0.230 | non-hateful |
| Hate at non-protected target ('I hate pizza') | 0.419 | non-hateful |
| Negation ('I don\'t hate X') | 0.415 | non-hateful |
Production threshold calibration
How strict you set the dial. False-alarm rate is the share of clean posts wrongly flagged; catch rate is the share of genuinely hateful posts caught at that setting.
| False-alarm rate | Catch rate | Use case |
|---|---|---|
| 1% | 64.5% | high-precision review queue |
| 5% | 91.5% | default production threshold |
| 10% | 97.6% | aggressive-recall mode |
See it on your data
Most pilots are live in under two weeks via REST API.