Your Code Review Triage Runs on a Signal That No Longer Means What It Used To

For as long as most engineers have been doing code review, a PR description that reads as confident, well-organized, and technically fluent has been treated as weak evidence that the change behind it is also competent. That correlation was never formalized as policy anywhere, but it shaped real behavior: a clearly written summary got a lighter read, a vague or garbled one got more scrutiny. A new measurement project tracking GitHub pull request text gives the first dated, reproducible look at when that correlation started breaking, and the mechanism is not “AI is writing the code.” It’s that AI can now produce the specific prose style reviewers have been using as a shortcut, independent of whether anyone understands the change underneath it.

What the study actually measures, and what it doesn’t

The project, built and maintained by a single researcher (Louis Abraham, github.com/louisabraham/load-bearing), samples GitHub pull request descriptions daily via GitHub’s search API and groups them by an unsupervised clustering method: each description is assigned to whichever of ten “ways of writing” its word-frequency pattern is closest to, using KL divergence as the distance measure instead of the more familiar Euclidean one. No description is told what it’s about or who wrote it. The clustering only sees word frequencies, and it sorts the corpus (461,121 descriptions collected across 85 complete weeks between January 2025 and mid-August 2026) into ten groups based purely on how they’re written.

One of those ten groups is what the project’s name refers to: a cluster whose defining vocabulary includes the phrase “load-bearing,” the word “seam,” and a sharply rising rate of em dash usage — a punctuation habit widely associated with large language model output, and one the project tracks explicitly, rising from roughly 0.2 appearances per 10,000 words in the corpus’s earliest weeks to about 123 per 10,000 words in its most recent ones. As of this writing, that cluster has grown from 0.7% of the sampled corpus at the start of 2025 to about 39% of it by mid-2026, and the project’s own trend line (a least-squares fit over the most recent 12 weeks) puts current growth at roughly 1.2 percentage points per week. Those exact figures will already be slightly different by the time you read this, because the corpus updates daily and the numbers are the point, not a snapshot of them.

It’s worth being precise about what that growth curve is and isn’t a measurement of, because the study’s author is unusually careful on this point and the piece would be dishonest if it weren’t equally careful. The clustering identifies a style of writing that has become dramatically more common in a specific, dated, reproducible way. It does not, and by its own methodology cannot, identify whether any individual PR description was written by a model, assisted by one, or written by a human who happens to write that way. The author’s own documentation states this as a hard boundary, not a caveat buried in a footnote. There’s a real reason for this limit: the median PR description in the corpus is around 65 words, short enough to fall into the range where separate published research on linguistic AI-detection (Pudasaini et al., 2026) finds that style-based detection degrades and becomes unreliable on short or out-of-domain text generally. The study doesn’t try to detect authorship at that resolution. It measures something coarser and more defensible: how the overall population of descriptions is written, in aggregate, over time.

That distinction matters for how to read the headline number. This is a corpus-level distributional claim. It is not a claim that “39% of PRs are written by AI,” and treating it as one would be exactly the kind of loosely sourced statistic this piece is trying to avoid repeating.

The number moves, and the study says so out loud

The 39% figure itself is sensitive to methodology choices the author documents openly rather than hiding: the number of clusters (ten, chosen from within a window where the analysis reliably identifies a single, coherent arriving group rather than a mixture of several) and the random seed used to initialize the clustering algorithm both shift where the boundary of the cluster ends up. The author’s own robustness check runs the fit repeatedly from different starting seeds without any of the safeguards that make the daily published number more reliable, and reports that 31 of 32 of those unconditioned single fits identify the same cluster as a clean, coherent group; in the remaining 1 of 32, the leading component comes out mixed with another. That’s a high but not perfect rate of agreement on the shape of the finding, run without the retry logic that makes any single day’s published figure more dependable than an individual seed would be on its own.

What that robustness check is really establishing isn’t a precise number, it’s a floor: that a specific, identifiable style of writing exists as a coherent, separable pattern in this corpus, and that it went from vanishingly rare to a large minority of a continuously sampled body of GitHub PR descriptions in under two years, and both of those findings hold up almost every time the clustering is rerun independently. The exact percentage the daily pipeline reports is one specific fit’s answer, subject to the choices that went into it. That the pattern is real, coherent, and growing fast is not.

The project’s own history adds a second reason to take its caveats seriously rather than treat them as boilerplate hedging. An earlier version of this exact measurement, built on the public GH Archive event feed instead of GitHub’s search API, reported the same word appearing in only 17 documents — wrong by a factor of 158, because GitHub’s Events API had quietly stopped carrying comment text in its payloads and the archive-based pipeline had no way to know the data it was reading had gone missing rather than gone quiet. The author found this, diagnosed it, rewrote the entire collection method around the search API instead, and documented the failure in the open rather than issuing a quiet revision. That’s a stronger signal of methodological seriousness than a clean result with no visible mistakes would have been. A parallel, peer-reviewed line of work — Kobak et al.’s “excess vocabulary” analysis of biomedical abstracts, using a comparable word-frequency-drift technique — independently found that at least 13.5% of 2024 PubMed abstracts showed signs of LLM involvement, rising to 40% in some subfields. The general technique of measuring vocabulary drift against a pre-LLM baseline has scholarly grounding outside the GitHub context specifically.

Why this is a review-process problem, not a writing-style curiosity

Here is the actual mechanism, and it’s an inference on top of the data rather than something the study itself claims: engineering organizations built their code review triage on an implicit heuristic that treats prose quality as a weak proxy for engineering quality. A well-structured description that clearly explains what changed and why used to correlate, reasonably well, with an author who understood the change deeply enough to explain it well. Reviewers calibrated trust and depth-of-read against that signal without ever writing it down as a rule, because it worked often enough to become habit.

That heuristic depends on the two things staying correlated. AI-assisted drafting breaks the correlation without breaking the appearance of the signal: a model can produce confident, well-organized, technically fluent prose regardless of whether the human submitting the PR fully understands the change, and it can do so at the exact volume this study is now measuring. The individual PR still looks fine in isolation, every time. Nothing about a single instance of this drift is alarming. What’s alarming, if the mechanism argument holds, is that the signal reviewers have been unconsciously trained on for years is drifting away from what it used to indicate, gradually enough that no one updates their calibration, because there’s no single moment that forces the update.

This is where the piece has to be honest about its own limits, because the strongest counterargument here is a real one, not a strawman. Vocabulary convergence toward a “load-bearing”-style corpus could simply reflect engineers increasingly using an LLM to draft a clean, well-organized description for changes they wrote and understood entirely themselves — using a model to write your PR summary is a productivity habit, not evidence the underlying work is unreviewed or poorly understood. Under that reading, the vocabulary shift is real but not actually informative about review risk: the thing that should determine review depth is diff complexity, test coverage, and security surface, independent of who or what phrased the summary, and conflating writing style with engineering risk risks becoming a moral panic about prose rather than a focus on what actually matters. That’s a legitimate position, and this piece doesn’t have primary evidence that review quality has actually degraded as a direct result of this vocabulary shift. The study doesn’t measure review outcomes. It measures writing.

What the counterargument doesn’t fully dissolve, though, is the original mechanism claim, which isn’t “AI-drafted descriptions are dangerous.” It’s narrower: if reviewers have historically used prose quality as an implicit triage signal at all — even as one input among several, even unconsciously — then a rising share of descriptions produced in a style that’s decoupled from the writer’s own understanding of the change makes that particular signal less informative than it used to be, regardless of whether any individual instance of it is fine. The right response to that isn’t panic about AI-assisted writing. It’s making review depth depend on things that don’t drift with prose style at all — diff size and blast radius, test coverage delta, whether the change touches a security-sensitive path — rather than on how the description reads.

What this doesn’t tell you, and what to do regardless

This measurement says nothing about the quality of the code changes underneath these descriptions, nothing about whether any specific team’s review process is currently working or failing, and nothing about the true share of GitHub activity that involves AI assistance in any broader sense — the GitHub Octoverse 2025 report puts overall PR volume at roughly 43.2 million a month, up 23% year over year, which is useful context for how large a base rate this sampled corpus sits against, but this piece isn’t relying on any of Octoverse’s more specific and less independently verified claims about AI involvement in that volume. What the study gives you is something narrower and more useful than another “X% of code is AI-written” statistic: a dated, methodologically transparent curve showing that one specific, identifiable style of writing has gone from rare to common in the artifact reviewers read before they decide how carefully to read the diff.

If your team’s review triage has ever implicitly rewarded a well-written PR description with a lighter read, the actionable move isn’t to start treating fluent prose with suspicion, which just replaces one bad heuristic with another equally unreliable one. It’s to stop using prose quality as a triage signal at all, and make sure whatever does drive triage — diff size, test coverage, which paths a change touches — is written down as the actual policy rather than left to whatever a reviewer’s gut has been trained on. That heuristic worked passably for years because writing quality and engineering understanding used to move together closely enough not to matter. This study is a rare, well-documented look at exactly when and how fast they stopped.

---

Sources:

• Louis Abraham, “The load-bearing vocabulary of Claude” — https://github.com/louisabraham/load-bearing (full methodology, live data)
• Dmitry Kobak, Rita González-Márquez, Emőke-Ágnes Horvát, Jan Lause, “Delving into LLM-assisted writing in biomedical publications through excess vocabulary,” arXiv:2406.07016
• GitHub Octoverse 2025 report (cited only for the independently stated PR-volume base rate, 43.2M PRs/month, +23% YoY — not for its more specific and less independently verified AI-involvement figures)