OpenAI introduced BioBasicsBench, a benchmark designed to measure how AI agents interpret raw experimental evidence from routine wet‑lab work. Developed with Ligo, the benchmark spans 100 tasks across five domains to test whether agents can read data, identify errors, and recognize when evidence is insufficient to support a conclusion.
What BioBasicsBench tests
The benchmark covers five areas: Protein Methods, Imaging & Cytometry, Quantification & Assays, Nucleic Acid Methods and Molecular Cloning. Inputs preserve the forms scientists encounter in the lab, including gel and blot images, plate photographs, microscopy stacks, sequencing files, CSV/Excel instrument exports, protocol notes, videos and reference sheets.
Tasks were written from scientists’ experimental data and paired with expert grading rubrics. Each task underwent an internal review by a Ligo scientist who did not author it, an external review by a PhD‑level wet‑lab scientist, and a final internal check. Agents return structured answers scored against rubric criteria that separate correct conclusions from partially correct analyses.
Evaluation results and observed failure modes
OpenAI evaluated seven models (GPT‑5.2, GPT‑5.4, GPT‑5.5, GPT‑5.6 Sol, GPT‑6.1 Sol, GPT‑6 Luna and GPT‑6 Astra) across 37 model and reasoning‑effort settings, with five attempts per task, producing 18,500 results. OpenAI reported mean reward (average task score as a percent of available credit) and 95% confidence intervals.
At the ‘xhigh’ reasoning setting, OpenAI reported GPT‑6 Astra achieved a mean reward of 66.7% (95% CI 60.3–73.1%) while GPT‑5.6 Sol scored 43.8% (95% CI 37.6–50.0%). OpenAI noted GPT‑6.1 Sol scored 62.0% and listed per‑model output tokens per task alongside scores.
OpenAI highlighted improvements in Astra’s multimodal handling and contextual reasoning: examples include recognizing overlapping dye signals that invalidate simple brightness‑based dead‑cell calls, rejecting a mislabeled antibiotic on a control plate, and identifying a z‑stack in microscope frames. However, OpenAI also reported persistent errors: assuming a preferred interpretation over control evidence, discounting observations that conflict with a favored conclusion, and miscounting colonies (an overcount of about 12% in one example) that would inflate downstream efficiency estimates.
OpenAI published BioBasicsBench on November 6, 2026, and reported the benchmark’s methodology, the seven tested models, 37 settings and a dataset of 18,500 model attempts.
Original source: OpenAI News