← All posts

An audit that can't change the result is a ceremony

Two of the most celebrated election audits in recent American history could not change a single result. That's not a feature — it's a confession.

It is the morning of December 8, 2007, and a report has just landed on the desk of Ohio Secretary of State Jennifer Brunner. Nine weeks of work by teams from Pennsylvania State University, the University of Pennsylvania, and WebWise Security, Inc. — teams who were handed the actual source code and equipment for the voting systems used across Ohio. Their conclusion, applied to every system they studied, is not ambiguous.

All of the studied systems possess critical security failures that render their technical controls insufficient to guarantee a trustworthy election.

Not some. All. The EVEREST report — Evaluation and Validation of Election-Related Equipment, Standards and Testing — documented shared failures: insider attacks the software could not stop, cryptography applied incorrectly, audit logs that could be forged or erased by the very people they were meant to monitor. In the case of the ES&S system, the report noted it had never previously received a detailed source-code and red-team security review at all.

Ohio's elections had been running on these machines. And the audits meant to catch problems with those machines? In every county across the state, they were non-binding. The count stood regardless of what they found.

That is the gap this post is about.

What "non-binding audit" actually means

An audit that cannot change the result is not a safeguard. It is a ceremony.

The word "audit" carries enormous weight. It implies scrutiny, accountability, the possibility of correction. We use it in finance because a failed audit stops something — it can void a set of accounts, trigger a restatement, end a career. When election officials say "we conducted a post-election audit," most people assume they mean the same thing. Usually, they don't.

A non-binding audit is an observation of the count with no teeth attached. It may find a discrepancy. It will file a report. The certified result remains certified.

In most U.S. jurisdictions — and many democracies worldwide — there is no automatic, pre-defined threshold at which an audit discrepancy triggers a correction, let alone a re-run. The finding goes into a document. The document sits in a file. The winner is inaugurated.

This is not a hypothetical problem. The Ohio EVEREST review — an independent, source-code-level, adversarial security analysis — found critical failures across every system it examined. It was released to the public under the Ohio Public Records Act. Ohio continued using those systems. The EVEREST final report makes clear that trustworthy auditing was itself compromised, with logs "commonly forgeable or erasable by the people they were meant to monitor." An audit built on forgeable logs is not an audit. It is paperwork.

We track this category of structural gap across jurisdictions here — /gaps/non-binding-audit.

The two audits worth studying side by side

If you want to understand what a real audit does differently, start with two American examples that are genuinely worth examining — not because they prove elections are stolen, but because they illustrate, precisely, what the word "audit" can mean when it has teeth, and what it still cannot prove on its own.

Colorado, 2017. Following its November election, Colorado became the first U.S. state to complete a statewide risk-limiting audit (RLA). The design is mathematically specific: rather than checking a fixed percentage of ballots, an RLA sizes its sample to the closeness of the race. A race decided by one percent requires far more hand-checked ballots to reach statistical confidence than one decided by twenty. Colorado started at a nine percent risk limit and later tightened it to three percent, meaning there was at most a three-percent chance the audit would certify a wrong winner. The Colorado Secretary of State announced the completion on November 22, 2017. It was not a recount. It was a statistical test with a defined confidence level, grounded in hand-checked paper ballots.

Georgia, 2020. When the presidential margin in Georgia came in at roughly 11,000 votes out of nearly five million cast, the state's first statewide RLA had to become something larger. Because the margin was so narrow, the statistical requirements meant auditing essentially every ballot. So Georgia did exactly that: 159 counties, 41,881 batches, roughly five million presidential ballots hand-tallied by human eyes in under six days. The result differed from the machine tally by about a tenth of one percent — well within the range of human counting variability. Georgia Public Broadcasting reported the confirmation on November 19, 2020.

Both of these are, by any measure, more serious than a spot-check. The question worth asking is what they still cannot do — and what happens in the dozens of states that do neither.

The gap between "we checked" and "you can verify"

Here is what the Colorado and Georgia audits share with every other post-election audit: they are conducted by the same officials who ran the election. Outside observers can watch. Candidates can send representatives. But the fundamental act — pulling the ballots, running the count, reporting the totals — happens inside a government process, controlled by government employees, documented by government records.

This is not a conspiracy. It is a structural observation about how trust flows.

The German Federal Constitutional Court put it plainly in 2009 when it ruled that electronic voting machines failed the standard of democratic legitimacy: the essential steps of voting and determining results must be examinable by the citizen without specialist knowledge. The Bundesverfassungsgericht's ruling in cases 2 BvC 3/07 and 2 BvC 4/07 is about computers — but its logic extends directly to audits. A process you cannot independently verify is not a verifiable process, whatever it is called.

The Dutch came to a similar place in 2007, when the government's own advisory commission published Stemmen met vertrouwen ("Voting with confidence") and concluded that transparency and checkability — controleerbaarheid — were non-negotiable foundations of a trustworthy election. The commission's answer was not better machines or better audits by officials; it was a human-readable paper record that any observer could confirm. The Netherlands dropped its voting computers entirely. The Kiesraad holds the full report.

An audit conducted by officials, confirmed by officials, reported to officials, and filed in official records is better than nothing. It is not the same as independent verifiability.

What "confirmed" leaves open

Return to Georgia 2020 for a moment, because it is the strongest case for hand-count audits — and even it carries a caveat worth naming.

A tenth of one percent variation between the machine count and the hand count across five million ballots sounds like noise. It probably is. But a tenth of one percent of five million is five thousand votes. The winning margin was approximately eleven thousand. That means the variation between the two methods was roughly half the margin of victory — in a race where the loser demanded a recount and the result was bitterly contested nationwide.

"The hand count confirmed the machine count" and "the hand count was identical to the machine count" are different statements. Georgia's hand count confirmed the outcome — the same candidate won by a margin large enough to survive the variation. That is exactly what an RLA is designed to do: confirm the winner with statistical confidence, not achieve zero-error precision. Colorado's RLA does the same thing.

But now scale the problem down. Antrim County, Michigan, 2020: a small county where human error produced obviously wrong unofficial results. When corrected and hand-audited, the final hand tally differed from the machine tabulation by about a dozen votes out of roughly 15,700 cast. Twelve votes. In a race decided by a handful of votes, a hand count that is itself off by a dozen votes is not a proof of accuracy. It is a measurement with a margin of error — and the margin of error must be smaller than the margin of victory before a method can "confirm" anything.

The honest lesson is not that hand counts are unreliable or that anything is fraudulent. The honest lesson is that even the gold standard of manual recounts carries irreducible human error — and error without a systematic, independent check is just error we have agreed to accept.

A non-binding audit of a paperless system cannot even reach this level of honest uncertainty. It is simply counting electrons and filing a report.

The audit that actually changed something — because it was adversarial

The Ohio EVEREST review changed something. Not because it was binding on results — it wasn't — but because it was genuinely independent and adversarial, with access to actual source code and equipment, run by researchers who had no interest in defending the existing system.

California did something similar in 2007. Secretary of State Debra Bowen commissioned a top-to-bottom review by University of California computer scientists who were given source code for the voting systems certified in the state. The Diebold report found that technological controls "did not provide sufficient security to guarantee a trustworthy election" and identified vulnerabilities to viruses that could spread between machines. The Sequoia report found "significant security weaknesses throughout the system." The Hart InterCivic report found components that "largely trust one another" with unsecured network interfaces.

These findings produced enforceable conditions — strengthened security requirements announced August 3, 2007 — because the reviews were independent, the results were published, and the Secretary of State had to respond publicly to what public scrutiny produced.

This is structurally different from an election authority reviewing its own count and declaring it confirmed. And it points directly toward the model that works: not audits by insiders, but inspectability by outsiders — cryptographic, public, machine-readable, and ongoing.

Brazil runs a formal public adversarial security test before every general election, inviting any qualified researcher to try to break the urna eletrônica, with findings used to improve the system before it is deployed. The Tribunal Superior Eleitoral's program is the institutional version of what Ohio and California commissioned as one-off reviews — except it is routine, expected, and built into the election cycle.

The design question audits cannot answer

Here is what no post-election audit, binding or not, can answer: Was the ballot I marked the ballot that was counted?

In Curling v. Raffensperger, decided October 11, 2020, the U.S. District Court for the Northern District of Georgia recorded expert testimony about Georgia's Dominion ballot-marking devices that makes this vivid. The court found the system "does not provide a verifiable and auditable ballot record because it relies on the QR code for vote tabulation and that code itself cannot be read and verified by the voter." The court's opinion also quoted the National Academies' conclusion that no technical mechanism currently exists to ensure a vote-counting application produces accurate results and that testing alone cannot ensure systems have not been compromised.

A hand recount in Georgia in 2020 counted the printed text on those ballot-marking-device output sheets. But the machines tabulated from the QR codes. If a machine printed text saying "Candidate A" while encoding a vote for "Candidate B" in the barcode — a scenario the court treated as theoretically feasible — a hand recount of the human-readable text would confirm the manipulation, not catch it.

This is not an allegation that this happened. It is a description of an architectural problem that no post-election audit can fully resolve if the artifact being audited is not identical to the artifact that was tabulated. The Windham, New Hampshire forensic audit in 2021 illustrated the inverse: scanners misread folded ballots because a fold crease in the wrong place looked like a marked oval to the sensor. The SB 43 forensic audit report found no fraud, no malware — just physics. And the error was only discoverable because durable paper existed and a hand count could recover voter intent. Without paper, there was nothing to recover from.

A system that tabulates from a code the voter cannot read, audited by a process the voter cannot replicate, confirmed by officials the voter must trust — that is not a verifiable election. That is a performance of verification.

What a real audit would require

A binding audit is one that can change the result. That means, before the election begins, defining in statute: what discrepancy threshold triggers a hand recount; what hand recount threshold triggers an investigation; what investigation finding triggers a fresh election. Not "we will look into it." A specific, automatic, pre-committed escalation path.

A meaningful audit also requires:

A trustworthy artifact to audit against — a voter-verified paper record that the voter physically confirmed, not a barcode they cannot read, produced by a machine they cannot inspect.

Independence from the officials being audited — not observers who can watch, but auditors who can compel access, run their own counts, and publish their own findings.

Public, machine-readable results at the precinct level — so that any citizen with a spreadsheet can replicate the top-line totals themselves, without needing a government intermediary to tell them the count was correct.

And, eventually, cryptographic end-to-end verifiability — the kind of system where a voter can check, using public data alone, that their ballot was included in the final tally without learning anyone else's vote. The Switzerland iVote failure in 2019 — where researchers Sarah Jamie Lewis, Olivier Pereira, and Vanessa Teague found a cryptographic trapdoor that could make altered votes produce a passing verification proof — shows how hard this is to get right. But it also shows exactly why source-code inspectability by independent cryptographers is the only way to know whether the verifiability claim is real.

An audit that cannot change the result is a ceremony. It may be a well-intentioned ceremony, conducted by honest officials with nothing to hide. That does not make it verification. Verification is what happens when the answer would look different if the count were wrong — and when anyone can run the check, not just the people who certified the result.

What is still not verifiable — and what would make it checkable

In most U.S. jurisdictions today, here is what an independent member of the public cannot do:

  • Confirm that the ballot they marked matches what was tabulated, without trusting the machine.
  • Replicate the county-level total from precinct-level machine-readable data the same night results are released.
  • Verify that the audit sample was drawn randomly and not cherry-picked.
  • Know, before the election, what discrepancy level automatically triggers a binding correction.

What would make each of these checkable: voter-verified paper records with human-readable marks; instant, precinct-level machine-readable publication of results; cryptographically verifiable audit-sample selection; and statutory escalation thresholds written into law before any election begins — not negotiated after a surprising result.

Colorado's RLA is a step toward the second and fourth. Georgia's 2020 hand count is a step toward the second. Neither is complete verification. Neither is a ceremony. The gap between them and the states that do neither is documented in our gap tracker.

The goal is not to distrust every election official. It is to build systems where their honesty is irrelevant to the question — because the answer is checkable by anyone.

Explore how this gap appears across jurisdictions →

See the full landscape of election-integrity gaps →

Two-minute summary →

Sources

See all postsExplore the gaps