Open data
Atlas of AI-written science failures
Errata and audit status for QNFO papers, regenerated daily. Free to read, no token. Licence: QNFO-ULA 2.1.
Data card
# QNFO record of how AI-written papers fail (2026-11-09)
This is an open record of the errors that the QNFO research fleet's own checks found in the fleet's own published papers. The papers are written by AI language models in an automated pipeline; so are the checks. The record is published because the failures are the useful part: it is a dataset of how AI-written science goes wrong, kept with the status of every finding, including the findings the fleet later rejected.
## Files (all under https://pub-426be63219f54be3932467bcbaf805f5.r2.dev/atlas/ai-science-failures/)
| file | records | bytes | sha256 |
|---|---|---|---|
| errata.jsonl | 1508 | 2930827 | 4be4d55a29baed68954fbc14b96b73dba294865a62721443aa66b877e5dce85f |
| papers-status.jsonl | 731 | 366158 | 41882288b1cc332733815a5b798362f34b0d188f692ab8855790952a5ce55977 |
| manifest.json | 1 | see manifest | - |
## Counts at generation (2026-11-09T14:31:34.110Z)
- Findings: 1508 (by status {"closed":1059,"resolved":208,"rejected":235,"open":6}; by severity {"high":1504,"medium":3,"low":1}).
- Published papers covered by papers-status.jsonl: 731; with at least one finding: 579; with an open finding: 6; with a human review recorded: 0.
- Left out: 6 reader-submitted reports (they come from outside the fleet and may identify a person); 0 text fields withheld because they looked like a credential.
## Fields of errata.jsonl
`id`, `paper_slug` (the paper at https://papers.qnfo.org/papers/<slug>), `paper_version` (the version the finding was made against, when the evidence says so), `detected_at`, `detector` (the check that produced the finding), `finder_model` and `confirmer_model` (set only when the evidence line names them: a first model flagged a passage and a second model confirmed it against the paper text), `severity`, `status`, `claim` (the passage or claim at issue), `falsification` (why it is wrong), `evidence`, `remediation`, `updated_at`, `ai_generated` (always true).
`status` is the fleet's own workflow state, not an external editor's verdict: `open` has not been answered; `resolved` and `closed` were answered (usually by a paper revision); `rejected` means the fleet's confirmation step did not uphold the finding, which is not proof that the claim in the paper is right.
## Fields of papers-status.jsonl
One line per published paper: `slug`, `title`, `version`, `url`, `last_revised_at`, the errata counts (`errata_total`, `errata_open`, `errata_closed`, `errata_resolved`, `errata_rejected`), `last_errata_detected_at`, `provenance` (`fleet-generated` means no person reviewed the paper; `ai-assisted-human-reviewed` is set only with a named reviewer) and `human_review_recorded`. A paper with no findings may simply have been checked less thoroughly; zero findings is not evidence that a paper is right (see the limits).
## Limits you should know before using it
- The findings are model judgements. The finder and the confirmer are normally different models, but language models share errors, so a flaw that every model misses is not in this record, and a finding both models share may be a false positive. No person is recorded as having reviewed an individual finding.
- Only QNFO's own papers are covered, and they are one author's research programme in a narrow field (p-adic and ultrametric mathematics, topological quantum computation and neighbours). Rates here say nothing about AI-written science in general.
- Detection effort differs by period and by check, so counts over time and across papers are not rates of error per paper.
- The records quote passages of the papers so a reader can judge each finding. The papers are under the same licence as this record.
## Contamination
Do not train or tune a paper-checking model on this record and then evaluate it on the same records or the same papers: split by `paper_slug`. The record is public and will be in future training data.
## Licence
LicenseRef-QNFO-ULA-2.1 (the QNFO Unified License Agreement v2.1, CC BY-NC-SA 4.0 plus the QNFO Supplemental Terms): https://qnfo.org/legal/license. It is the licence of the papers quoted here. The licence text governs any use for training a model; read it before you rely on this record for that.
## How to cite
Quni-Gudzinas, R. B. (2026). QNFO record of how AI-written papers fail, version 2026-11-09 [dataset]. https://pub-426be63219f54be3932467bcbaf805f5.r2.dev/atlas/ai-science-failures/ (no DOI yet).
## Updates
Regenerated once a day by the qnfo-infra worker (REACH-COMMONS-1). `manifest.json` carries the version, the time and the sha256 of each file. A correction to a record arrives in the next version; earlier versions are not kept here.