← All posts

An audit nobody is allowed to watch

An audit conducted without public observers, behind locked doors, produces one thing with certainty: a result you have to take on faith.

It is the morning of December 26, 2004, and in Kyiv, Ukraine, voters are filing into polling stations for the third time in two months. The first round of their presidential election was flawed. The declared second-round winner was invalidated by the Supreme Court on December 3 — the OSCE's observers concluded the original runoff had "failed to meet a considerable number of OSCE commitments" for democratic elections. Now, under tighter international scrutiny, with observers watching the count at every table, Ukraine is trying again.

What made the repeat vote different was not the people. It was not a new law. It was transparency: immediate publication of results, open counting, cameras on the process. The OSCE found the December 26 vote "substantially closer" to international standards, with improvement "most clearly shown in… the transparency of the Commission's work, including immediate publication of results."

That phrase — immediate publication of results — is the whole argument in four words.


The audit is not the problem. The closed door is.

Election audits are, in principle, one of the best ideas in democratic governance. You count the votes electronically, then you independently check that count against a physical record no machine touched. Done right, an audit can tell you whether the reported result matches the ballots voters actually marked. It can catch errors, distinguish honest mechanical failure from deliberate manipulation, and give every losing candidate — and every voter — a reason to accept the outcome.

Done behind closed doors, it produces none of that.

An audit conducted without public observers, behind locked doors, produces one thing with certainty: a result you have to take on faith.

This is the gap we track at /gaps/audits-behind-closed-doors. It is not exotic. It is not confined to fragile democracies. It happens routinely, in wealthy countries with well-staffed election offices, run by officials who are not corrupt and are genuinely trying to get it right. The problem is structural, not personal — and understanding why requires looking at two cases where the door was left open.


Windham, New Hampshire: what an open audit actually finds

The morning of November 4, 2020. A clerk's office in Windham, New Hampshire — population about 14,000 — is staring at a gap. Four Republican candidates for State Representative won their races the night before, but their machine totals were significantly lower than what a subsequent hand recount produced. One candidate's hand-count total was more than 300 votes higher than the machine tally.

The New Hampshire legislature responded with something rare: a genuine forensic audit, conducted under SB 43 by an independent team — Harri Hursti, Mark Lindeman, and Philip B. Stark — with the process open to observation and the findings published in full.

What they found was completely mundane, and completely unsettling. The town had leased a folding machine for mailing purposes. That machine did not fold ballots along the printed score lines — it folded them through the vote targets. When those absentee ballots passed through the AccuVote optical scanners, the fold creases were interpreted as marked ovals. The team found that roughly 44 percent of folds through targets were read as votes, with rates ranging from about 20 to more than 72 percent depending on conditions. White powder that had accumulated inside the scanners — obstructing the lenses — may have made it worse.

No malware. No fraud. No tampering. Just a folding machine and some dust, silently miscounting hundreds of validly marked ballots, invisible to anyone who was only looking at the totals.

The audit found this because durable paper ballots existed and because the team could examine them physically, compare them to scanner output, test the machines, and publish every finding. The full report runs to dozens of pages of methodology, test results, and conclusions — the kind of document that makes an audit reproducible and checkable, not just asserted.

A closed audit of the same discrepancy would have produced a press release. An open forensic audit produced an answer.

The distinction matters because Windham's discrepancy was visible. The hand recount and machine totals differed substantially enough to trigger the investigation in the first place. The scarier question the report itself raises: what about the mechanical miscounts that don't produce a detectable gap? A race won by 2,000 votes, where a fold-line error shaved 150 from one candidate's total — that would never surface. The machine would report. Officials would certify. And the closed audit, checking the machine's output against itself, would confirm everything matched.


Georgia, 2020: five million ballots, in public, in six days

Now consider scale.

After the November 2020 presidential election, Georgia — with roughly five million ballots cast — conducted a statewide risk-limiting audit of the presidential contest. Because the margin was narrow, the audit escalated to a full manual hand tally: 159 counties, 41,881 batches, examined over less than six days.

The hand count affirmed the machine-tabulated outcome. The variation between the original count and the hand count was about a tenth of one percent.

That tenth of a percent — across five million ballots, counted by humans, under time pressure, in the middle of a bitterly contested national election — is what a genuinely rigorous audit under real-world conditions produces. It is not zero. It reflects the irreducible reality of human counting: a ballot placed in the wrong batch, a tally sheet transposed, a voter intent that two different humans read differently.

That matters. It means even the gold-standard manual audit carries error. And in a race decided by a margin smaller than the audit's own variance — say, a few hundred votes in a jurisdiction where the hand count itself diverges by that much — "the hand count confirmed it" is a reassurance, not a proof. As we've noted before on this blog, the reassurance is not worthless, but it cannot settle the question on its own. What Georgia's 2020 audit actually demonstrated was something narrower and more valuable: paper ballots, plus an open process, plus statistical rigor, plus independent observation, produce a result that anyone can scrutinize. The confidence comes from the architecture, not from the officials.

What it did not demonstrate is that the same confidence is achievable when any one of those elements is removed.


What a closed audit actually checks

Here is the mechanism. Most post-election audits work by sampling paper ballots and comparing them to what the tabulating machine recorded. If the sample matches, the audit is said to confirm the result. If you are running a risk-limiting audit, you can even calculate how statistically confident you should be in that confirmation.

But every link in that chain depends on something you cannot verify from inside the chain:

  1. The paper ballots must be the original, unmodified ballots. If the sealed bags containing them were opened between election night and the audit — by anyone, for any reason, with or without witnesses — you are auditing whatever is in the bags now.

  2. The chain of custody must be documented and independently verifiable. Austria's Constitutional Court annulled a presidential election in 2016 — finding no fraud, but finding that absentee-ballot envelopes had been opened by unauthorized people before the required time, in roughly 77,000 cases, exceeding the winning margin of 30,000. The court's point was explicit: the safeguards against manipulation had been violated in numbers capable of affecting the result. You cannot certify a clean count if you cannot certify the chain.

  3. The comparison must be made against ballots the voter could actually verify. In Curling v. Raffensperger, the U.S. District Court for the Northern District of Georgia found that Georgia's Dominion ballot-marking device system "does not provide a verifiable and auditable ballot record because it relies on the QR code for vote tabulation and that [QR] code itself cannot be read and verified by the voter." A hand recount of those ballots does not audit the voter's intent — it audits the QR code's output. If the machine encoded the wrong choice into the barcode, the hand recount faithfully reproduces the error. The court's own words make the problem precise.

  4. The process must be observable. An audit conducted in a room that credentialed observers cannot enter produces a report, not evidence. The report might be accurate. You cannot tell.

Closing the door does not make step four impossible. It makes steps one through three unverifiable from outside the room.


Why officials close the door — and why that is not enough

It is worth being fair about why audits are sometimes conducted without meaningful public access. Election offices are small. Staffing is thin. The rooms where ballots are stored are genuinely sensitive from a security standpoint. Some officials worry, reasonably, that fully open access creates opportunities for disruption. Others are following procedures written before anyone thought systematically about public verifiability.

None of those reasons are corrupt. All of them are insufficient.

The purpose of a public election is not to produce a result. It is to produce a result that citizens who did not conduct the count can independently confirm.

That is what Germany's Federal Constitutional Court held in 2009 when it banned voting computers: "the essential steps of the voting and of the determination of the result can be examined by the citizen reliably and without any specialist knowledge." Not checked by officials. Not attested to by the vendor. Examined by citizens. Without specialist knowledge. That is a standard most closed audits fail on its face.

The Dutch government's 2007 commission on election reform — which ultimately led the Netherlands to abandon voting computers entirely — set the same bar: the process must be transparent, and "questions must be answerable and the answers checkable and verifiable." Checkable by whom? By anyone who wants to look.

The 2005 GAO report on electronic voting security found that federal efforts to improve these systems were under way but that key activities remained incomplete. That was 2005. The pattern since — Windham's folded ballots, Georgia's unreadable QR codes, Coffee County's copied system software — suggests the incomplete activities have remained incomplete in ways that matter.


The two-step test for a real audit

If you want to know whether an audit is genuine or ceremonial, two questions cut through every official statement.

First: could an independent person who was not part of running the election, and who does not trust the officials or the machines, have watched every step and checked every figure?

If the answer is "no" — if the ballot bags were handled without witness signatures, if the counting room had access restrictions that excluded credentialed independent observers, if the results were announced without the underlying batch data being published — then the audit confirmed only that the people running it believe it was correct. That is not an audit. It is a self-report.

Second: is the artifact being audited the one the voter marked?

If the count was run against a QR code the voter cannot read, or against a database rather than physical paper, or against paper that passed through an unmonitored room between election night and audit day, the answer is "not necessarily." And "not necessarily" is not good enough to settle a close race.

Windham, New Hampshire passed both tests because the legislature forced the question and the forensic team was allowed to work in the open. Georgia in 2020 came close because the scale of the race demanded it and the paper ballots were real, human-readable, and physically present.

Most audits do not get either condition.


What would actually change this

The fix is not more trust in officials. Officials can be honest and still produce an unverifiable audit, because the problem is structural.

The fix is building the verifiability into the process before the audit begins, so that by the time you get to the audit, independent confirmation is automatic rather than aspirational.

That means three things in practice:

Precinct-level results published the moment they are final — in machine-readable formats anyone can download, reconcile against tapes, and check for internal consistency. The U.S. Election Assistance Commission already urges this. Some jurisdictions already do it. The gap between "some" and "all" is the gap that lets unofficial totals go unchecked overnight.

Paper ballots that voters can actually read — where the human-readable text, not a barcode, is what gets counted and audited, so a hand recount audits the voter's intent rather than the machine's encoding of it.

Open, structured, witnessed audit procedures — where the chain of custody is documented in real time, the ballot bags are opened in front of credentialed observers, the counts are entered into public tally sheets as they happen, and the full batch-level data is published before the results are certified.

None of this requires trusting a new kind of machine. None of it requires cryptography. It requires treating the audit as a public act — which is what it is, legally and democratically — rather than an internal administrative review that the public is later told about.


The question nobody has answered yet

Here is what is still not verifiable, in most jurisdictions, today.

If a folding machine crept a fold line across vote targets in thousands of absentee ballots — but the race was won by enough that no one demanded a hand recount — would the routine audit catch it?

Almost certainly not. Because routine audits in most places check a small sample of ballots against machine totals. If the machine miscounted consistently, the sample matches the machine. The audit passes. The report says "confirmed."

The only thing that stops that scenario from being invisible is a paper ballot anyone can inspect, an audit procedure anyone can watch, and batch-level results anyone can download and check against the printed tapes. Without all three, you are not auditing the election. You are auditing the story the machines tell about the election.

That is the difference between a closed audit and a verifiable one. One produces a result. The other produces evidence.

See how common this gap is across the world →

Read the 2-minute version →

Track this specific gap →


Sources