Science, or slop?
A paper can read well sentence by sentence while its science does not connect. Unlike token-level AI detectors, the Science Slop Index reads the scientific reasoning: whether sections build on one another, claims and citations are argued, and the method and evidence can be inspected.
Link, PDF or LaTeX .zip. Reports stay private unless you list them.
0papers checked by researchers so farSpot slop the index missed? Flag it on the report. Contributors become co-authors of SciSlop v2.
Papers analyzed so far
· Leaderboard · Gallery
News
- Oct 2026Paper, benchmark, code, and this site are out. Science or Slop? is on arXiv (2610.00531), SciSlopBench on Hugging Face, the code on GitHub.
- ComingA Science Slop Index for every ICLR submission. We are preparing to run the six measures over this year's ~60K ICLR submissions and publish the venue-wide picture: how much scientific slop is in the pool, which patterns, and how it relates to the outcomes.
- Looking forAPI credit sponsorship. The four rule-based measures run on our own machines; the two model-based ones (Argument graph, Figure exposition) need roughly a million model calls for 60K papers. If your organization can sponsor API credits, write to yerim.oh@vision.snu.ac.kr. Sponsors are acknowledged in the release.
Team
The people behind Science or Slop? and this site.
Cite
If you use the Science Slop Index, SciSlopBench, or the code, please cite the paper.
@article{oh2026scislop,
title = {Science or Slop?: Benchmarking and Mitigating Scientific Slop in AI-Generated Papers},
author = {Oh, Yerim and Lee, Young-Jun and Ahn, Jaewoo and Kim, Gunhee and Kang, Dongyeop},
journal = {arXiv preprint arXiv:2610.00531},
year = {2026},
eprint = {2610.00531},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.00531}
}How it works
Six measures, three planes, one number
Each measure counts the share of a paper's units that show one pattern. The index averages the measures within each plane, then averages the planes, and reports the result out of 100. Higher means more of the patterns that set AI-generated papers apart from matched human papers.
Reads the paper, not the prose
AI text detectors score sentences: how predictable each token is, how a model would have phrased it. A paper written or revised by a language model can pass that test and still fail as science. Each paragraph looks fine on its own while the connections between paragraphs do not hold: a figure that no section ever refers to, a claim that arrives before anything leads up to it, a related-work paragraph that lists citations without relating them, a method diagram crowded with hyperparameters, a results table with no concrete example anywhere in the paper.
The Science Slop Index counts these breakdowns. It reads the whole paper, groups six patterns into three planes, and asks three questions. Below, each plane is illustrated with real findings from the papers analyzed on this site; use the arrows to browse them, and click a finding to open it on the paper.
What happens to a paper
Every report receives a key that reopens it later. Uploaded files are deleted after analysis; only the report and rendered figures stay.
1Read
LaTeX source is read directly; arXiv links fetch the source and the PDF. PDFs are rebuilt into sections, captions, references, and citations.
2Measure
Four measures are exact counting rules. Argument graph and Figure exposition ask a language model to label sentences and read the method figure.
3Highlight
Every flagged unit is placed back on the PDF, colored by plane, with a graph per measure. Export as highlighted PDF, JSON, or CSV.
The six measures
After Table 1 of the paper. Each score is a share of units: the numerator over the denominator. Pair accuracy is how often that measure alone ranks the human-written paper above its AI-generated counterpart on SciSlopBench.
One number
The index averages the measures within each plane, then averages the three planes. A measure that does not apply to a paper is skipped, not counted as zero.
The four bands are descriptive cut-points on the share of flagged units, not a probability of AI authorship.
Reported pair accuracy
Table 2 of the paper: on SciSlopBench, 390 AI-generated papers each matched with a human-written paper on the same problem. 0.5 is chance.
Leaderboard ↗
Papers analyzed on this site, ranked by Science Slop Index. Higher means more scientific slop. Click a row to see its six measures; open the report for every finding on the paper.
Gallery
Papers analyzed so far
Papers analyzed from public links, and uploads whose owners chose to list them. Ranked by Science Slop Index.
View report
Open a report
Enter the report key you received when you submitted your paper.
Propose a pattern
Propose a scientific slop pattern
Seen a recurring way AI-generated papers break as science that the six measures miss? Name it and say what a reader notices. We do the rest: implement it, run it on SciSlopBench, and adopt it if it separates AI from human papers, with your name on it.