FairStress is a benchmark designed to measure demographic fairness in large language models (LLMs) beyond whether models reach the same final choice in equivalent scenarios. The benchmark focuses on whether models apply the same criteria and resist changing well-supported decisions regardless of which demographic group benefits.
What FairStress tests
The dataset contains 300 realistic two-candidate decision scenarios spanning 29 domains, including hiring, healthcare, lending, education, housing, and criminal justice. Scenarios include both clear-cut cases, where one candidate is better supported by the evidence, and ambiguous cases with no objectively superior choice. Demographic identity is varied from absent to implied to explicitly labeled.
FairStress also measures decision stability under adversarial prefixes—matched persuasive statements that argue for one candidate using authority, historical disadvantage, legal concerns, or social consensus. Combining identity conditions and prefixes expands the set to more than 13 million test cases. A 50K-item FairStress-Core subset is provided to reduce evaluation cost while retaining diagnostic power.
Key findings from model evaluations
The report evaluates open-weight and frontier models including Olmo, Llama, Qwen, Gemma, Claude, and models in the GPT family. When prompts contained no demographic information, models largely chose the stronger candidate at similar rates. As identity became more explicit, behavior grew less symmetric. The report describes an “anti-stereotypical over-correction” pattern: models were more likely to abandon a correct decision when doing so favored a minority-coded candidate.
Measured effects include average gaps of 6.9 percentage points with implied identity and 9.3 points with explicit labels in abandoning correct answers. On FairStress-Core, frontier models such as Claude Opus 5 and GPT-5.5 selected minority-coded candidates about 60–65% of the time when neither candidate was objectively stronger. Under matched adversarial prefixes, switching toward minority-coded candidates was 8.8–42.4 percentage points more common across tested runs.
Additional analyses found differences in explanations: minority-coded choices used more terms like “resilience,” “diversity,” and “barriers,” while majority-coded explanations used more procedural language. Limited activation analyses on Gemma-3-12B-it and Qwen2.5-7B-Instruct showed internal activations predicted demographic side with ~73% accuracy; removing names reduced that accuracy by ~18 points.
The report also compares FairStress with BBQ and attributes divergent bias directions partly to evaluation format: generated answers and chat templates produced different results than fixed-option probability scoring.
Most debiasing interventions tested did not reliably eliminate asymmetry; one activation-steering method reduced Qwen2.5-7B-Instruct’s accuracy on clear decisions from 98% to 47%. Counterfactual chain-of-thought reduced some asymmetry while largely preserving accuracy but did not remove all differences on ambiguous cases.
FairStress is available for download and is being integrated into Olmo-eval for broader model comparisons.
Original source: Hugging Face Blog