How journalists can evaluate claims of election fraud
What are Media Briefings?
Research shows that election fraud is extremely rare even as public concern about fraud remains high. This difference in realized vs. perceived fraud is especially important as election-night vote totals often shift as votes are tabulated, counting can take days, and administrative discrepancies can emerge, all without indicating fraud. For reporters covering elections, understanding what processes are normal vs. which ones warrant scrutiny is essential. SciLine’s media briefing examined how election officials prevent fraud, detect problems, and verify that reported outcomes accurately reflect the votes cast. Three panelists participated in a moderated discussion, then took questions from reporters, all on the record:
- Dr. Michael Alvarez, California Institute of Technology
- Dr. Marc Meredith, University of Pennsylvania
- Dr. Philip Stark, University of California, Berkeley
- Matt DeRienzo, director of SciLine, will moderate the briefing
Journalists: video free for use in your stories
High definition (mp4, 1920x1080)
Introductions
[00:00:26]
MATT DERIENZO: Hello. This is Matt DeRienzo, Director of SciLine, and welcome to a briefing that we’re doing today with three great experts on how journalists can evaluate claims of voter fraud.
In my career in journalism, I know that in years of covering various proposed changes to how we vote, especially since 2020 or so, claims of widespread voter fraud or elections’ vulnerability to it have been kind of really at the heart of the rationale for some of those decisions. So it’s something that we encounter all the time as journalists. And today we have three experts who have done independent, nonpartisan research on that: Dr. Mike Alvarez, Dr. Marc Meredith, and Dr. Philip Stark, who we’ll hear from in a second.
They’re here to talk about their research, explain the parameters of how we detect and prevent voter fraud, and how prevalent it is. And after some initial presentations and back-and-forth that I’ll guide, they’re here to answer your questions. So if any of you are new to SciLine, I’ll tell you a little bit about the organization first.
We’re a philanthropically supported nonprofit, and we have one sole mission, and that is to help journalists access, understand, and incorporate more evidence-based research and scientific expertise into their work. We do that for free in a variety of ways, and we help reporters with it on deadline. Basically, any story that you’re working on that somebody studies, we can help you step back from the political back-and-forth and talk to a subject matter expert, talk to somebody who’s using the scientific method to understand that topic better. We do that through briefings like this, and then we have all kinds of in-depth reporting guides on how to access that kind of research on specific topics at sciLine.org. And that can range from extreme weather to data centers to flock cameras, a huge issue that we’ve done stuff on recently, to affordable housing, mental health.
This particular briefing, and one that will be—stay tuned for around this time next week—that we’ll be doing on what to expect on Election Day in the aftermath, are part of a months-long project that we’re doing on the midterm elections. You can go to sciLine.org and find a whole bunch of reporting guides there on the science behind, the research behind all kinds of voting process issues, in addition to those issues that are being debated state by state.
So accompanying this briefing, we have a reporting guide on voter fraud and understanding the extent of it and how it works. We have one on voting machine technology, voting by mail, gerrymandering, and one that we just published on the barriers that young people face to vote. I think you’ll find a lot of good stories, ideas there, and also some really important context for your coverage.
And then finally, the unique thing about SciLine, if you haven’t experienced it already, is that in addition to those resources, you can go on our site, and there’s a button that says, “I need an expert.” You click on that button, and you can get help on, really, literally any story that you’re working on that’s just individual to you, your community. And when you click on that button and say, “I need an expert on data centers,” or “I need an expert on this debate over voting machine technology in my community.”. Our team, which is part scientists and part journalism, background people like me- work together on a deadline, literally human beings to find an expert who is independent, nonpartisan, vetted, and has published research really relevant to what you’re doing, and connect you with them.
So for this briefing, we’re going to start with some introductions of our panelists, and then I’ll get into some back and forth with each of them explaining a little bit about their work and delving into some of the central issues to this. And then we’ll be ready for your questions. So feel free throughout to add your questions in the Q&A box there. First, Dr. Alvarez, if you could introduce yourself and your work.
[00:04:41]
MICHAEL ALVAREZ: Hi, Mike Alvarez. I’m a professor of political and computational social science at Caltech out in Pasadena, California, where I’m also the founding co-director of the Caltech Linde Center for Science, Society, and Policy. I’m also the co-director of the Caltech/MIT Voting Technology Project, which has been studying a lot of these issues since the 2000 election. I’ve been studying, again, election administration, election fraud, voting behavior, all sorts of election-related topics, you know, for my entire 30-plus-year career. And certainly, I’m happy to be here today with this group to answer all questions. And you can also reach me at rma@caltech.edu; I’m very simple to find.
[00:05:26]
MATT DERIENZO: Excellent. Dr. Meredith?
[00:05:29]
MARC MEREDITH: Hi, everyone. My name is Marc Meredith. I’m a professor of political science at the University of Pennsylvania in Philadelphia. I do a lot of research similar to Mike’s on election administration and elections. I also have experience in the media, having worked many times on the NBC Decision Desk on election night, so I also have a sense of what it’s like to be on the other side, and I edit the Election Law Journal, so I get exposed to a lot of those types of issues through that role. And it’s great to be here. Thanks for having me.
[00:05:56]
MATT DERIENZO: Dr. Stark?
[00:05:57]
PHILIP STARK: Hi, Philip Stark. I’m a professor of statistics at the University of California at Berkeley. My serious work in elections started around 2007, working for then-california secretary of state Debra Bowen. I’ve worked with secretaries of state of California, Colorado, and New Hampshire on various election integrity-related things. I testified as an expert witness in Curling v. Raffensperger over election technology in Georgia. I was on the board of advisors of the U.S. Election Assistance Commission for about nine years, appointed by Pelosi, invented something called “risk-limiting audits” that are now a thing. Thanks. Glad to be here.
Q&A
How do you define voter fraud, and where is the line between an honest mistake and intentional fraud?
[00:06:39]
MATT DERIENZO: We’re going to get into that when we get to your part of it. Absolutely. Thank you, guys, so much for being here and taking your time, volunteering your time to help reporters out today. One of the things before we get into some of this, I just want to stress, and I think you guys will hear it as a caveat a couple of times throughout this conversation, is what most of us know is that elections are controlled locally. And so the answer to some of our questions here is, it depends on where you live, right? And so we’ll get into that. So first, I’d like to start with you, Mike. Zooming out. How do you define voter fraud? And what’s the difference between where is the line between someone making an error and accidentally doing something versus fraud that has intent? And what’s the difference between that and, like, you know, just aggressive politics?
[00:07:33]
MICHAEL ALVAREZ: The first thing to start with is that voter fraud, what you just asked me about, is really a subset of the larger classification of election fraud. And so, starting with election fraud, that’s any kind of crime that’s really aimed at corrupting the election process. And oftentimes that’s usually from a campaign, a candidate, a party, or some other kind of insider. Voter fraud is really the narrower set of crimes that are committed by or in the name of individual voters. We can talk about both. You usually use the terms interchangeably, but it is good to try to make that distinction.
A key thing here when we’re talking about fraud is always separating fraud from error, and what separates fraud from error is intent. And in terms of the question about the candidates in the campaigns, what separates fraud from aggressive politics is usually just the law itself. So in any case, election fraud, like I said earlier, is a crime that’s committed to corrupt the political process. A lot of times, that’s things like vote-buying operations or schemes to collect and fill in discarded absentee ballots or tampering with the count, intimidating voters, coordinated activities conducted usually with larger groups of people.
Election fraud is things like voting twice, voting in someone else’s name, or voting when you know that you’re not eligible. Now, what’s the bottom line between fraud and error? And as I said earlier, the key answer here is intent. So fraud is something that’s knowing and willfully conducted. So someone like a poll worker who gives the wrong ballot to a voter; or a county that somehow enters data incorrectly for its precinct tabulations; voters who moved and still appear in the rules in their old counties- those are mistakes. Those kinds of mistakes matter, and they accumulate in ways that matter, but they’re not intentional acts to try to change the outcome in a legal way. Aggressive tactics and fraud, and again, I said earlier, the answer is: there’s a law. Campaigns can do a lot of activities that are really hardball politics, but they’re all legal. Filing lots of challenges to registrations and polling places, aggressive litigation, running misleading and erroneous ads, pushing hard on get out the vote. Those kinds of things can be seen as objectionable by people. But that’s politics. And in an election like we’re seeing right now, that kind of hardball politics is to be expected because of the competitiveness of the process.
So my basic advice for reporters is always to ask first: intent- who did it; ask whether it breaks the law; if you ever have any doubt, you get in contact with folks like the three folks here, myself, Philip, Marc, other academics who study the process.
How difficult is it to measure how often voter fraud occurs, and at what scale?
[00:10:36]
MATT DERIENZO: One of the underpinning rationales for sweeping changes to how we vote, or minor changes to how we vote, depending on how you look at it, is that this happens in a way that could change an election. Or that it happens in a somewhat systemic way. And I’ve never been able to find a really clear thing that tells us how often it happens. Can you get into a little bit about how difficult it is to get our arms around that?
[00:11:05]
MICHAEL ALVAREZ: It’s a difficult question, and we wrote an entire book on it at one point in 2010. So how often does fraud occur? Every real serious attempt to measure the incidence rate of election fraud, or voter fraud, finds that it’s very rare. In 2014, my colleague Justin Levitt, who’s just across the way here in Los Angeles, did a study where he looked for in-person voter impersonation fraud, and he found that there were 31 credible incidents out of 1 billion ballots or so that were cast from 2000 to 2014.
If you read the New York Times, they just had a study that shows that since January 2025, there have been 131 people charged with illegally registering or voting as non-citizens at the federal level, and there have been 25 convictions. So those are incredibly low-incidence types of behavior. That indicates that voting fraud itself, as an act, is usually very rare. But I will say that we have lots of processes in place to detect it. Most of the time, these get detected when they occur, and they get prosecuted because many of them are felonies.
So what I tell my students at Caltech is we’ll talk a lot about election fraud, but don’t try to commit it because you’ll get caught and you’ll get charged with a felony. Now, the other problem with studying fraud is that there’s no- and this goes back to what you said earlier about elections in the US- there’s no national database of election fraud. Nobody collects that data. In fact, there are really no state databases that are reliable that really track election or voting fraud data. So it’s difficult to find the data, and the best kind of data comes from things like I just cited: academic researchers, journalists, and advocacy groups on both sides.
Now, does it matter? It matters; it matters in particular because what happens when fraud allegations are made: people get upset, voters lose confidence in the process. And losing confidence in the process has many downstream consequences that we can talk about later. Does it affect the outcomes of elections? Probably not. First off, because it’s so rare. Secondly, because in general, most elections in the US are not very close. It can only matter in situations where we have very close elections, and usually very close elections at very small jurisdiction levels in the United States, like, say, a mayor’s election or things like that. So why don’t we see it matter anymore? Well, again, it’s rare, and it usually doesn’t affect the outcome of, say, a house election.
[00:13:41]
PHILIP STARK: Can I jump in briefly, please? I want to agree in part and disagree in part with what Mike said. I think that what he said is absolutely true with regard to voter fraud, and I think the distinction that he drew earlier on between voter fraud and election fraud is incredibly important. The level of voter fraud: all of the investigations that have happened have shown that it’s incredibly rare; really, it’s a handful of votes nationally over many, many years. Election fraud more broadly, we’re actually not set up in a way to detect it. So things that would involve messing with databases, messing with cybersecurity aspects, etc. We don’t actually run our elections in a way that would reliably detect problems that were on that kind of a scale. But individual voters, bussing voters across state lines to vote illegally in another state, that just doesn’t- there’s very strong evidence that that essentially never happens.
Does the verification process differ between mail-in voting and in-person voting in ways that affect fraud detection?
[00:14:41]
MATT DERIENZO: Dr. Alvarez, there’s been a lot of focus on voting by mail, over the past year, at least in the run-up to the midterm election. Is there anything different about the verification process for voting by mail versus voting in a local polling place that would have implications for detecting fraud?
[00:15:03]
MICHAEL ALVAREZ: Well, so when you vote in person, or you vote by mail, from an election official’s perspective, the process is identical. You’ve got to confirm that the voter is eligible to vote. You’ve got to confirm their identity, and you’ve got to make sure that they only get to vote once. So those are the key things that happen whether you vote in person or whether you vote by mail. When you vote by mail, obviously, you get a ballot sent to you by mail that’s been vetted against the registration record. And in general, you’re only supposed to get one ballot sent to you, one ballot for each voter. And then when it gets sent back, the ballots are marked as having been voted.
So then, like in a vote center in California where we have early voting, electronic poll books are directly connected to the election database, the voter registration database. And so if those mail ballots have already been returned and you show up in person, the poll workers are going to see that, and they’re going to issue you a provisional ballot, so that’ll be checked before it’s counted. So we have safeguards in place to make sure that people don’t vote twice. You know, with mail, the key thing that happens is that the ballots are closely scrutinized when they’re returned before they’re opened. When you’re returning your vote by mail ballot, you have to sign it. And what happens when those ballots are returned, those envelopes are scanned, or they’re examined by human beings, and the signature is verified and validated against the signature that’s on file. And only once that verification has happened is the ballot removed from the envelope and included in the tabulation process. So the key thing that happens with votes by mail is that the signature is verified against the signature that’s on file.
[00:16:51]
PHILIP STARK: So jumping on that as well, forgive me for complicating things over and over. There are jurisdictions where there is no signature matching taking place where, in fact, there’s consent decrees saying they’re not allowed to check the signatures. That signature checking is noisy. There are false negatives and false positives for sure. But if there were a lot of people voting using somebody else’s ballot, there would be collisions. You would see someone showing up to vote, and they’re being told, “No, you’ve already voted by mail,” and some level of complaints would happen, and I think we would see that. So I agree with the conclusion, but I think the mechanisms are a little different depending on the jurisdiction.
Why doesn’t the U.S. have a national voter registration list, and what are the implications of comparing federal data to state voter rolls?
[00:17:30]
MATT DERIENZO: The point that we made earlier is that there’s numerous rules in different states. So those names are checked against state-level lists. And there’s been a lot of talk this year coming from the administration’s attempts to take, whether it’s Social Security data or other things, and compare it to these state-level voting lists. Why don’t we have a national voter list?
[00:18:02]
MICHAEL ALVAREZ: Well, that goes back to what we just said just a second ago: elections are run at the state and local level. The Elections Clause in the Constitution largely delegates to states the authority to run elections. There are some exceptions to that, obviously, but in general, states run elections. And so in the aftermath of the passage of the Help America Vote Act, right after the 2000 election, that was when states really developed centralized state files. Most states didn’t have them before that. And that’s where we get this notion of a statewide database. States, in some sense, have kind of centralized the files, and they have data-sets that list every registered voter and have other information about them, including some identity information. But again, this all goes back to the Constitution and states and counties in the U.S. running elections.
What checkpoints exist between casting a ballot and that vote appearing in election results to ensure it hasn’t been tampered with?
[00:18:57]
MATT DERIENZO: Dr. Meredith, I want to zoom out again a little bit and say, when I cast my ballot and that vote between then and that vote showing up in the results on election night, what checkpoints, if any, exist to make sure that my vote hasn’t been tampered with, that I am who I am or whatever, or that it should count.
[00:19:23]
MARC MEREDITH: There are lots of things along the way that help ensure the integrity of the process. I like this distinction that Mike made between errors and fraud because I think what we see, I think, in practice, is hopefully these processes deter fraud and identify errors. I think that’s largely how this ends up working in practice. One thing that all election officials are doing, they’ve already done or are about to do, is the logic and accuracy testing on their voting machines. And that’s a really important part of the process to ensure that all these pieces of technology that we have accumulated to help cast ballots and tabulate ballots are configured well and that they’re going to give you accurate reports of voters’ intent.
On election day itself, there’s this process of reconciliation that’s happening at most polling places. We’re getting as an input the number of people who have checked in to vote, and as an export, we’re getting counts of how many ballots are in the system. And election officials and poll workers are checking those two numbers against each other and trying to identify any discrepancies that might exist between those two and figure out why and fix that.
But arguably, I think the biggest safeguard that we have in the system is what you just referred to as election night reporting, and those are not the final totals. And a lot of what we do in American elections is use intermediate reports as a way to put out to the general public: Here are the results we have. Do you see anything notable that you need to bring to our attention that could be wrong? I think that’s a really important safeguard in the process: that we have this intermediate reporting stage where, if there is something wrong, we had kind of a high-profile example. I remember the 2020 presidential election where something went wrong with the accuracy and testing in Antrim County, Michigan. And they had reversed the votes for Trump and Biden, and we quickly saw that and were able to fix it, and that wasn’t part of the final results. So I think this intermediate stage is a big part of why we can feel good about results because if there’s an issue, there’s so many eyes on these things, especially in high-profile elections, and we have enough time before we finalize the results to correct them.
Why and how do election officials reconcile the number of voters checked in with the number of ballots counted?
[00:21:34]
MATT DERIENZO: So as I understand it, in an election, a very common part of counting the vote is to reconcile how many voters have been checked in to vote and how many ballots are counted. Why do they do that? I’m sorry, that’s a dumb question. And how do they do that?
[00:21:51]
PHILIP STARK: Some states don’t.
[00:21:56]
MARC MEREDITH: But many do. And the ones that are largely keeping track, as people come in to vote, are tracking how many people have been checked in to vote. In Philadelphia, we still use paper poll books largely. And so it comes across as the poll worker sort of telling me a number, which they’re writing down in their poll book. As I vote, they’ll say, like, “Marc you’re voter 130, and that number will get attached to my name in the poll book.” And then at the end of the day, when the final voter comes through, let’s say there’s 1,000 people who have numbers who have been assigned; they’re going to go back and try to reconcile that number with the number of ballots that are tabulated on the machine.
And there are reasons why you want to do this. One is you don’t want poll workers entering ballots into the system. And this is a way to check that election fraud has occurred by people entering ballots that don’t get accounted for by a voter. It also will help identify cases if poll workers are doing the opposite. They’ll start discarding validly cast ballots so the number looks too low. And so, we’re looking for large discrepancies in those two numbers. Small discrepancies are often a result of something like a voter maybe forgot to put their in the bin. Maybe they spoiled a ballot, made an error, and got a new one; the poll worker didn’t do that. Small discrepancies happen all the time. At least in my home state of Pennsylvania, at the end of the day, they’re supposed to release a report that documents any discrepancies that are left unfound.
I would say, from my time covering elections, one of the things that this reconciliation is really helpful for is if ballots got scanned multiple times or failed to be scanned. I think that’s probably the common explanation for why you’d see a large discrepancy. And again, that’s an error, not fraud. But it’s also really helpful for those errors. So I think those are the primary reasons why we want to engage in the reconciliation process.
What mechanisms prevent voters from casting a ballot more than once, whether in person or by mail?
[00:23:55]
MATT DERIENZO: So what you talked about specific mechanisms or parts of the safeguards that prevent more people from voting twice, whether it’s in person or by mail?
[00:24:06]
MARC MEREDITH: I think maybe compared to non-citizen voting. There may be a little bit more of this that happens. It’s still quite rare, but I don’t think it’s quite as rare as non-citizen voting in terms of where voter-level fraud happens. I’ve done some research on this, and it probably accumulates to at least in the thousands in a given election. But if we think about thousands spread out over the millions and tens of millions of ballots, it’s still, I don’t think enough to register as something that could change the outcome of an election. But we have lots of safeguards in place. Mike hit on some of them in his answers. If we’re going to have mail balloting, we have to have a way for people to get replacement mail ballots. People are going to spill coffee on their ballot. They’re going to lose their ballot. And so anyone who’s running a mail balloting system has thought about ways that we can have voters exchange out their mail ballots. And so one of the common ways that jurisdictions do this is by having bar-codes on mail ballots that you can sort of turn signals on as these ballots are being processed that would note: Hey, this is someone who got two ballots. This is a replacement ballot. Let’s pull this one out of the pile for further testing. These things get more complicated, as we do in most states where you really have both an in-person and mail voting component, where now we have to worry when someone’s showing up to cast an in-person vote, we have to have a way to check: have they cast a mail ballot? States use different processes, but the thing that all states have because of the Help America Vote Act, all states have provisional ballots. And that’s a good thing because we can be conservative if we have any questions about whether someone has already voted using a mail ballot. We can say to this in-person voter: Look, we’re going to make you fill out this provisional ballot. We’re going to adjudicate after the election whether you’ve already voted, and if you haven’t, count this ballot, and if you have, we won’t. I think the process of provisional balloting is a really important safeguard in this process.
[00:26:17]
PHILIP STARK: I was just going to throw one wrinkle in there, which is the safeguards you’re describing are all around people voting twice in the same state. But some fraction of this is snowbirds who have residences in two states, and they’re voting in both. And that happens with some frequency.
[00:26:33]
MARC MEREDITH: Yes, I agree. That’s exactly what I was about to say. It is exactly that. I think where we’d be more worried is people have trouble looking across jurisdictions. Within the same state, it’s challenging. Cross-state is even more challenging. So I think we do worry about that. I do think that speaks to the importance of good voter registration list maintenance. You do have organizations out there like ERIC that help support cross-state registration checks to see if people are on the rolls in multiple states. There’s some states that have individual-level partnerships with each other to do this, as well. So an important part of having secure elections is regular maintenance of your voter registrations and making sure that people don’t have to register, or at least limit that to the extent that’s practical. And it helps you think about election administration as a system where these different portions of the system come together to help keep our elections with integrity.
What questions can reporters ask local election officials to understand the safeguards in their community or state?
[00:27:28]
MATT DERIENZO: There’s reporters here from all kinds of different states. So with that in mind, are there common questions that a reporter can ask their local officials to understand kind of what the safeguards are in their community or in their state?
[00:27:44]
MARC MEREDITH: Absolutely. I think some of the things we’ve already hit on that vary. Do you do signature verification or not? Do you have post-election audits? Do those elections happen, or do the audits happen before or after certification? How do you think about updating your poll books in real time to account for mail ballots that have come in? These are things that are going to differ across states. They’re going to have a system that’s usually going to sync up well together, but they are going to differ. And so I do think it’s really helpful to do this, I think, before the election, in part because you’re going to just have a better relationship with your local election official if you’ve made contact with them before- you’re coming to them at 3:00 a.m. on election night, being like, “Well, explain this discrepancy between your check-ins and your ballots counted.” And so I think getting in touch now, getting the groundwork for some of these things that do vary across states and even counties that help ensure election integrity is really important.
I think the other thing that’s important to remember is I’ve seen people get into this habit of interpreting election data far too literally. They want to interpret real-time check-ins right now on social media as a measure of voter enthusiasm or how many people have voted. And I think there’s just taking data far too literally. These things are not designed to be real-time updates of how many people have voted. They’re designed, I think, as I highlighted earlier, to be safeguards. We have public release of people who have voted in part so that people can see for themselves whether they see something, anything, that looks problematic and bring that to people’s attention.
So I think the other important thing that reporters can do is when they do see something that’s out of line, or seems odd, maybe, go to the election official and highlight to them, but ask what the normal explanation is for why this is happening. Don’t assume just because something looks strange that fraud or error has occurred. I guess the last piece of advice I would give is just try to give a sense of scale, like I think Mike was doing earlier when describing the amount of non-citizen voting that happens. We don’t want to negate the fact that things happen, but I think it’s also important to put errors and fraud in the broader context of the number of ballots that election officials are dealing with. And when you say maybe 100 people voted twice in the election, that sounds big. If you say 100 people out of 3 million voted twice, that sounds maybe less big. So I think giving that sense of scale is really important as you’re reporting on these things.
Why do election audits exist, how do they differ from recounts or administrative reviews, and what makes an audit credible?
[00:30:19]
MATT DERIENZO: Two things out of that stuck out to me. One is some great old editor’s advice, which is, get to know your local election officials before election day. Bring them a box of donuts now. You’ve only got about ten days here. And it’s also a great segue to Dr. Stark. You mentioned election audits. Can you talk to us a little bit about why election audits exist, how they’re different than, say, a recount or administrative review? Are there uniform standards? How do we know what’s a good audit?
[00:31:00]
PHILIP STARK: There’s a lot to unpack there. We need audits because errors happen, fraud happens, equipment gets misconfigured. As Marc mentioned, people forget to scan a box of ballots or scan it twice or three times, in the case of some of the stuff that happened in Georgia in 2020. People mean various things by audits. They range from just looking at error logs that’s generated by software to actually physically inspecting some number of ballots selected at random and looking at the votes and trying to use that to get some confidence or cast doubt on the reported results. It’s really, on some level, measure twice, cut once, just as in carpentry. Mistakes happen. If you’ve run your election in a good way, then a probity audit, typically a risk-limiting audit, is a way to tie a bow around a well-run election and say, whatever happened to go wrong, something always goes wrong, wasn’t material in the sense that it didn’t alter who appeared to win. So there’s always going to be something that goes wrong, but is what went wrong big enough to matter to change the reported winners?
The crucial thing in audits is to have a trustworthy audit trail to base the audit on. If the paper trail isn’t trustworthy, then it’s garbage in, garbage out. It’s security theater, and really, audits are a distraction from bigger problems in the election rather than a way to get confidence that the reported result is really right. And there are entities that market audits as a service; there are some state and local election officials who want to say, you know, we audited, and therefore, there’s nothing to see here. Everybody can go home. We science this thing, and therefore, it’s fine. But if the paper trail isn’t trustworthy, it’s not at all clear what you’re accomplishing other than distracting from bigger problems.
But what does it take for the paper trail to be trustworthy? Well, we talked about some of that. You need an accurate voter registration list. You need to know that the people who are registered really should be registered. Need some way to authenticate voters before they cast votes. So poll books, signatures on mailed ballots, things like that, and you need to know that that process doesn’t make too many mistakes in either direction, that you’re not disenfranchising people or enfranchising people who shouldn’t be. Then you need to generate the marked ballot in a way that is trustworthy. That generally means hand-marking paper ballots for the vast majority of voters, with assistive technology for voters who have dexterity issues that affect their ability to manipulate a piece of paper or visual issues that affect their ability to independently mark a ballot. Marking a ballot using a touchscreen ballot marking device is not a trustworthy way of marking a ballot. And there are entire states that require every in-person voter to use a touchscreen device to generate the paper trail. Georgia is an example. The whole county of Los Angeles, which is one is bigger than most states and one of the largest election districts in the largest in the country and larger than most states. So you want to generate the paper trail in a trustworthy way. Having done that, you need to keep track of it.
And Marc and Matt hinted at some of this, things like ballot accounting. Did the number of pieces of paper that went to a polling place blank come back equal the number that came back, voted, spoiled, or unused? The poll book reconciliation: does the number of ballots that we have equal, approximately, the number of people who checked in? There are wrinkles in how different jurisdictions do things, and some do an actual physical inventory of the pieces of paper to count the ballots there. Others rely on the software and the voting system to tell them how many pieces of paper there are. That’s trusting the voting system to audit itself. That’s not generally good practice. You need public reconciliation; ballot accounting. You need a really serious canvass. The canvass is this period of time where they make sure that every validly cast vote is included in the tally. This is the difference between election night results and the final result. You need a secure chain of custody of the ballots. You need good physical seal protocols. You need to be recording seals when you open things and making sure that’s the same number that should be there. There are all of these things, two-person chain-of-custody rules that really matter to ensure that at the end of the day, the paper trail can be relied on so that if you were to count the votes in that paper trail, perfectly, they would tell you who really won. If you don’t have that assurance, then an audit is not going to tell you anything. It’s just going to be a distraction from the fact that your paper trail isn’t trustworthy in the first place.
But what’s the difference between an audit and a recount? Well, first of all, audits are typically done on a sample basis. You don’t look at every ballot unless looking at a sample doesn’t give you sufficiently high confidence that if you did count every ballot, it would give you the same answer that you’ve already got. But you can think of a risk-limiting audit as an intelligent incremental recount that starts looking at ballots and doesn’t stop until it gets compelling evidence that the reported results are right. If it never finds compelling evidence that the reported results are right, it becomes a full hand count, which shows what the right answer ought to be. Here we’re talking about outcomes, not about exact counts. What we care is: did Lincoln win, and how many votes exactly did Lincoln get? It’s a different it’s a different and lower standard, but it ought to be somehow the minimal standard.
How does a risk-limiting audit work, and what would it take to reach a sufficient level of assurance?
[00:36:37]
MATT DERIENZO: When you say a risk-limiting, I was about to ask you about that, works until you find you have assurance. What would give that to you? Can you give us a little example? Yeah.
[00:36:52]
PHILIP STARK: So let’s just use a model for this: Somebody claims that more than half the ballots show a vote for Alice. We’ve got a two-candidate contest, Alice versus Bob; Alice is the reported winner. Alice is the reported winner, and if we pull a ballot at random out of all the ballots cast in this particular contest, it’s more likely to be a vote for Alice than a vote for Bob. So you can think of this as testing whether a coin is biased in favor of heads. A coin landing heads is like a vote for Alice; a coin landing tails is like a vote for Bob. So if you were to toss a coin twenty times and get twenty heads in a row, that would be pretty convincing evidence that the coin is biased in favor of heads. Similarly, if you were to reach into this pile of ballots at random, really technically at random, not just haphazardly, twenty times and pull out ballots and every time the ballot showed a vote for Alice, that’d be pretty strong evidence that Alice really got more votes than Bob in the entire pile; that’s one way. If you’re coming along more votes for Bob than Alice, or a whole bunch of votes for Bob and a whole bunch of votes for Alice, you might not be able to call it. You might need to look at more and more and more until the results would be really, really surprising if anyone other than Alice had actually won.
How long have risk-limiting audits been in practice, and what does their track record tell us about the integrity of U.S. elections?
[00:38:20]
MATT DERIENZO: So how long have risk-limiting audits been in practice? In terms of the body of them happening, what can you say about what that says about the integrity of U.S. elections and how often they find that things are not amiss?
[00:38:40]
PHILIP STARK: Let me throw one thing in before answering that question, which is the risk that a risk-limiting audit is supposed to limit is the risk of certifying the wrong winners. So what you’re trying to do is to make sure that if Alice didn’t really win but Alice was reported to win, you have a big chance of noticing that and correcting it before the outcome is final, before it’s legally binding. So that’s the risk you’re trying to control. Risk-limiting audits were invented in 2007 and first piloted in 2007 in California. They’ve been piloted in a bunch of states since then. Colorado passed a law requiring risk-limiting audits on the state level in about 2011 or ’12, but didn’t implement it statewide, I think, until about 2018, partly because of infighting between local jurisdictions and the Secretary of State, etc. Colorado has been doing them statewide in a legally binding way for the longest. There’s now, I think, about fifteen states that have laws that either mandate or authorize risk-limiting audits. Not all of the laws are equally good. Some use the words risk-limiting audit, but what they describe wouldn’t actually limit the risk of certifying an incorrect outcome. Some of those are for reasons that I think Marc or Mike mentioned, which is, for example, that the audit happens after the results are official. It’s too late to correct problems after the results have been certified. Some of them don’t worry about the integrity of the paper trail. What Georgia is doing right now is a sham risk-limiting audit. It does not limit the risk. So there are about fifteen states that do this. I don’t think a risk-limiting audit has overturned an election result, or it’s never been discovered that the reported result was wrong.
That said, some other kinds of audits have discovered problems that were big enough to alter election outcomes. There have been pilot projects in a number of states. California had, I think, two rounds of laws authorizing risk-limiting audit pilots. The most recent one was, I don’t know, five-ish. I was barely pre-COVID. Yes, it varies. There is no state that I’m aware of that routinely does a risk-limiting audit of every contest and every election. For example, Colorado’s risk-limiting audits are one statewide contest and one countywide contest in each election. And election officials are prone to say: We audited one contest; therefore, everything worked perfectly, no votes were flipped, etc.”That’s not a legitimate inference. You can have problems that affect one contest but not others.
How vulnerable are U.S. elections to organized hacking or manipulation, and how would we know if it occurred?
[00:41:24]
MATT DERIENZO: We’ve talked a lot, and actually, a lot of the discourse is focused on individual people voting illegally and that happening a lot and adding up to something, which we’ve talked a lot about here. You mentioned something at the beginning that I think probably perked up a lot of reporters’ ears, which is our possible susceptibility to a hack or manipulation that’s a little bit more organized. Can you say more? And how will we know?
[00:41:54]
PHILIP STARK: I mean, ballot box stuffing of physical ballot boxes has been a thing in the U.S. forever. Long before computers were involved in elections, there was large-scale election fraud. The difference when you have physical artifacts like paper ballots as the basis of what’s going on is that it’s difficult to alter or inject a lot of ballots into the system without some accomplices. And once you have accomplices, it gets more complicated.
The vulnerability, the leverage that happens as a result of using electronics to count votes, is that a single point of failure can alter an arbitrary number of votes. This can happen because of poorly designed software. It has happened that has happened, including voting machines that end up counting backwards, for example, because they overflow the amount of storage they have for numbers and then weird stuff happens. It can happen because of misconfiguration. Marc mentioned logic and accuracy testing that happens in most places, but it’s not perfect. The number of different combinations of vote patterns and things that people use is relatively limited, and it’s not protection against malware; it’s protection against obvious misconfiguration of the system. You can have more subtle problems, like somebody forgets to scan a box of ballots or scans it twice or uploads the memory sticks into the tabulator several times instead of only once because they don’t have an inventory of the memory cards that they’re using in the election. There’s insider threats, and the insider threats became really clear after 2020. There are jurisdictions that invited outsiders in, faked security credentials for them to come and get access to all of the software and all of the hardware, download databases, and so on.
A huge vulnerability is actually the election reporting that we’ve been talking about. If I wanted to cast fear, uncertainty, and doubt on US elections, I wouldn’t need to hack the voting equipment; I’d just hack the election reporting system that’s putting these results on the web. I could make it look like stuff was crazy, and nobody would trust the election results even if the final result was actually correct. So the cybersecurity of most of these devices is extremely poor, which should inform the roles that technology should be allowed to play in elections. So any situation where a failure of the technology will keep people from voting, or can’t be detected by some other means, is a problem. So one of the reasons that risk-limiting audits can be a backstop to electronic malfunctions in the tabulation and the aggregation and all of these other things depends on there being trustworthy physical artifacts.It really does depend on the physical security, chain of custody, hand-marked paper ballots, and so on. That provides a backstop for the states that do it for the contests that they actually audit, to say, if something went wrong in the tabulation and aggregation, I have a big chance of noticing it and fixing it before the result is final.
How should reporters weigh the rarity of election fraud against the difficulty of proving a negative, and how should they put conspiracy theories in perspective?
[00:45:03]
MATT DERIENZO: So that’s a great segue into a question I want to ask all of you. And as I do that, folks on the line, if you have questions, please put them in that Q&A box, and we’ll start asking your questions in 1 second. So that sounds scary the ability to manipulate people. And do you guys have any advice for reporters trying to weigh what you’ve said about the safeguards and how rare this stuff happens, with how hard it is to prove a negative doesn’t exist? And also, how to put conspiracy theories and other stuff in perspective given some of those vulnerabilities that Dr. Stark just talked about.
[00:45:52]
PHILIP STARK: I’m going to jump—I’m going to keep talking until you tell me to shut up—there’s a lot of misinformation on both sides of the political spectrum. On one side, we basically have vulnerabilities being equated with fraud. The fact that the vulnerability is there doesn’t mean it’s been exploited. In the other direction, we have out-and-out denial of well-documented vulnerabilities, including by members of Congress. Recently, I don’t remember the name of the senator; I believe he said voting machines aren’t connected to the Internet. It would be as hard to hack an election as it is to hack my toaster. It’s like, that’s just not true. Voting machines are connected to the Internet routinely, both directly and indirectly. That’s just not a true statement. Denying the vulnerabilities, I don’t think, is the way to get to secure, trustworthy elections where we’re not going to end up with blood on the streets if we disagree about what happened.
We really need to run elections in a way that starts with evidence, and elections should provide convincing evidence to all parties that the reported winners actually won. So a lot of the Excitement after 2020 was focused on things that I think were a little sideways to the real issues. For example, with Dominion voting systems in Georgia, some of them, the local election officials weren’t able to find all of the images of the ballots that should have been generated by the system, or a cryptographic hash of an image was missing, or the time stamp on it was wrong. Those are all probably procedural errors; there are signs of poorly designed software, etc. But ultimately, the evidence is the ballots, the physical ballots. And so even if the electronics are completely untrustworthy, even if you’re worried about supply chain problems, if you’re worried about the fact that the computer boards came from China or that the manufacturer of the voting system is extremely political in one direction or another. On some of it, it doesn’t matter if you have trustworthy ballots that you can go and check the results again, and you check the results against the paper. So I would kind of discount that here and just say that.
[00:48:15]
MICHAEL ALVAREZ: I think a lot of what Philip was talking about, we have to be careful here. Vulnerabilities are not fraud. Vulnerabilities are important; they need to be studied and addressed. But I think the question was about fraud. And what happens in many situations for reporters is that they’re going to hear a story from someone about some sort of anomaly: something weird happened, something weird in the data, something weird is being told about an election, about a story. I think the key thing that reporters need to do in those kinds of situations when they hear about an anomaly is investigate it to see if there’s not a perfectly legitimate explanation, a counter-explanation for that anomaly. That’s not fraud because, as Marc, Philip, and I have talked about a number of times, many times these anomalies that arise in elections- and we do see lots of anomalies. Are just mistakes that people are making. They’re errors in the process, and luckily, many of these anomalies get detected through these post-election procedures that Marc and Philip have been talking about.
[00:49:16]
PHILIP STARK: So to add to what Mike just said, the kinds of things people have gotten exercised over in the past are things like the Edison feed count for some candidate going down. Well, somebody probably just corrected an error because that’s something that’s generated manually typed in from other electronic sources, and it’s expected for that kind of thing to happen. Another kind of anomaly is people seeing fraud in the fact that, as absentee ballots are counted, the shares for different candidates change. That has to do with political preferences among who casts ballots how, who votes by mail, and who doesn’t, certainly. I mean, I’m not saying that it couldn’t be fraud, but it has a completely innocent explanation, as Mike mentioned. Numerical coincidences and results for different pieces of geography, they’re going to happen when you have a large number of things. Coincidences happen. There’s a lot of things that people get exercised over that seem completely benign.
[00:50:19]
MICHAEL ALVAREZ: And a lot of times these things happen just because they’re elections. One of the things about elections is that they’re exciting, they’re competitive, they’re dynamic. And if they weren’t, I wouldn’t be here talking. So things happen in elections like, a lot of times candidates will spend huge amounts of money trying to mobilize voters in very specific geographies. That’s going to happen in elections, and that will look like an anomaly because suddenly we’ll see that in certain geographies, turnout may just bump up in ways that we haven’t seen in the past. Again, a perfectly legitimate explanation for that kind of phenomenon, not fraud.
[00:50:57]
PHILIP STARK: Something that happened, I think it was in Pennsylvania, if I recall correctly, was surprising uniformity in the election results, which turned out to have a political explanation. There was a religious community that was all instructed to vote one way and did.
Given the severe penalties for voter fraud and the minimal personal benefit, do journalists have an obligation to convey how unlikely individual voter fraud is?
[00:51:11]
MATT DERIENZO: I’d like to move on to reporter questions. So we have a question from Random Lengths News, an independent newspaper in Los Angeles. Given that individual voter fraud is illegal under state and federal law, it carries severe penalties, and that individuals receive minuscule personal benefit to zero benefit from voting illegally, and that it’s often a struggle to get legal voters to vote in the first place. Do you feel journalists have an obligation to make clear how unlikely it is for anyone to be motivated to commit voter fraud?
[00:51:52]
MICHAEL ALVAREZ: Yes. It would be great if you would help us get the word out. Yes. Not only is it really difficult, but the incentives are minuscule. It’s incredibly difficult to commit, and when we talk about election fraud, not just voter fraud; talk about election fraud. It’s very difficult to commit election fraud at scale. Because one of the other things is you have to get people to coordinate, and it’s very hard to get lots of people to coordinate in illegal activity, first off, and also to do so in ways that aren’t detected. So yes, please help us get the word out.
[00:52:24]
PHILIP STARK: I think what may help is that there have been a number of studies sponsored by the federal government, generally by Republican administrations, all of which have found that the numbers are a handful. And so having those references at your fingertips might be helpful.
What recourse do voters have if their registration is not processed in time due to a state error?
[00:52:42]
MATT DERIENZO: We have a question from a freelance reporter in Austin about a recent thing that I’m sure all of you guys are familiar with in the news, and that was when the DMV in Texas didn’t send thousands of voter registrations to be counted. This is maybe on the flip side of fraud, but what recourse do voters have if their registration doesn’t get updated in time due to that state’s error? What is the flip side of this, where the safeguards and other things in the process prevent people?
[00:53:14]
MARC MEREDITH: I think this is one of the important use cases of the provisional ballots that we talked about earlier, which is that we can do a lot of things after the fact if people record their vote intentions on election day, and then we can sort out afterwards whether those votes should be counted or not, and there can be litigation. There are things about that. So I think there’s very little downside to casting a provisional ballot other than, you know, you’re using up some bandwidth of poll workers and stuff like that. But I think getting the word out that if you show up to a polling place for whatever reason, and you think that’s the right place for you to be voting, and there’s some question about that. Make sure you fill out that provisional ballot. And I do think given what I know about the story, at least, if people have filled out the paperwork by the registration deadline, even if they’re not showing up on the poll books that day, those ballots will ultimately be counted. And so I think that’s one of the clearest examples we can come up with of why establishing provisional balloting. Help America Vote Act was so good for election administration.
[00:54:22]
PHILIP STARK: Marc, you’re very optimistic about what the local election officials would do. I can completely see legal arguments over whether, even if you submitted your registration, if it hadn’t been processed, you were not legally registered at the time that you did that, and so your provisional ballot shouldn’t be counted, even though you could have been legally registered if they had done their job. And it’s just hard to know how that would play out in practice. But I agree that the best thing to do is fill out the provisional ballot.
How are electronic voting machines set up and verified, and how can officials confirm they haven’t been altered ahead of Election Day?
[00:54:52]
MATT DERIENZO: Here’s a question from The East Hartford Gazette in Connecticut. It says: I date back to the time when the machines were gears and tabs and levers. A mechanic played a vital role behind the scenes, setting up the machines. Mistakes would happen when the gears and tabs were not set up properly. And in those cases, the machines would be set aside. So how does all that work, and who’s doing that with electronic voting machines? How can elected officials know if the machines are set up properly, as they can be changed easily and perhaps even remotely right up to Election Day? I will say that, in addition to the response you’re about to hear, SciLine has a whole reporting guide on voting machines and some of the history with Dominion and stuff like that that you can find on our site. Does anyone want to take that?
[00:55:43]
MICHAEL ALVAREZ: I think Marc talked about this already, to some extent, with logic and accuracy testing. I mean, it’s not—and I’m sure Philip will talk about this, too—it’s not a fail-safe, but it’s one of the procedures that’s in place for trying to make sure, pre-election, that at least some set of the voting systems are working as expected. I’d say that the other safeguard here is that on election day, there’re going to be people in the polling places watching the operation of these machines. Voters are going to interact with them. And as problems arise, observers, poll workers, and others oftentimes will flag those problems, and if the polling place is working or the vote center is working as anticipated, those systems that are flaking out or not functioning correctly will be shut down.
[00:56:31]
PHILIP STARK: So that didn’t happen in Northampton, Pennsylvania, in a relatively recent election where it was, first of all, the equipment passed logic and accuracy tests, but then on election day was flipping selections in a judicial retention contest. Voters complained that their printed selections on these BMDs were being reversed. They were told conflicting things about whether to cast the ballot anyway or go back and re-vote deliberately the wrong way so that the printout would match what they did. At the end of the day, they didn’t rerun the election even though they knew the equipment was malfunctioning. A judge just ordered counter the way it is. So the right thing doesn’t always happen. Logic and accuracy testing has big holes, and they’re bigger in some jurisdictions than others because they use fewer combinations of vote patterns and things when they’re doing the testing. You know, there are safeguards that voters can help participate in. For example, photographing poll tapes that are supposed to be posted after the close of polls on election day and checking those results against what ends up getting posted on the web later to make sure that this precinct count is the same in both places. Audits of trustworthy, generated and kept trustworthy, you know, paper ballots are really kind of the best backstop we have for tabulation, but again, you need to establish that the paper trail is trustworthy, and a lot of that is really shoe leather. I mean, it really is making sure that the eligibility determinations were made accurately; no ballots fell off a truck, no ballots fell on a truck.
[00:58:11]
MARC MEREDITH: So I still think it’s the case on this front. It’s not perfect, but logic and accuracy testing is going to make it better than the world without it. Having workers—
[00:58:20]
PHILIP STARK: Oh yes. No, it’s better than nothing. It’s not protection against malware.
[00:58:24]
MARC MEREDITH: No, I mean, malware is different, but I think accuracy logic testing gets it right 99 percent of the time, arguably. And then when it doesn’t, workers are going to fix it with a high probability. It leaves some small probability left that we have this Northampton case, but I do think, you know, we can make things a lot better. We’re probably never going to make things perfect, but we also don’t want to make the perfect enemy of the pretty good.
What role do bipartisan canvassing boards and partisan poll watchers play as safeguards in the election process?
[00:58:49]
MATT DERIENZO: Marc, that made me think of something that I wanted to ask you earlier, which is we also hear a lot of talk, and this has been happening for many, many decades, too, where you’ll have canvassing boards where there’s representatives from both political parties and independents; in some cases, you’ll have partisan poll watchers. Do they play any role at all in being a safeguard against some of this stuff? Are they part of the problem, or I think they’re probably more often part of the solution than part of the problem.
[00:59:22]
MARC MEREDITH: I think there are people who will have reasons to disagree with someone being an eligible registrant, and they can lodge a registration challenge. You will have cases where there’s genuine disagreement about what the law says you should do in the case of a handwritten ballot; voter intent can be a subjective thing. And so I think there are times when poll watchers can certainly do things that are less than helpful. But I think, again, more often than not, having the poll watchers there, I do think probably that is a good thing, and having bipartisan canvassing boards, I think, at least makes sure that both sides’ opinions get registered. And again, we’re going to always have some disagreements on things that have the importance of elections. But you know, I’d rather have a world where you have poll watchers and bipartisan canvassing boards, perhaps disagreeing. Again, because we get more time after Election Day. Having this period from election night to canvassing gives people the opportunity to consider more facts than you have time to do in real time. And I think that’s a real advantage of the system: when we have more time, oftentimes we can reach better choices. When you were talking about Butter intent, I had a vivid flashback to 26 years ago and hanging chads in Florida. We have a question from a freelance reporter in Portland, Maine.
How expensive is it to implement risk-limiting audits, and how do under-resourced jurisdictions manage without them?
[01:00:58]
MATT DERIENZO: How expensive is it to set up a jurisdiction to conduct a risk-limiting audit, and is expense part of what’s holding people back from doing that more often with proposed cuts to the Election Assistance Commission’s budget? How do jurisdictions that just can’t afford it?
[01:01:21]
PHILIP STARK: The EAC has never been involved in funding the routine audits or anything like that. There were a couple of pilot grants. I think California got one, Colorado got one on trying to figure out how to implement them statewide.
The answer is that the vast majority of work is in making sure the paper trail is trustworthy. And you don’t need much additional technology at all. Some of the methods for risk-omitting audits, you can really do the calculations with a pencil and paper. Addition and multiplication are all you need to do. If you want to audit efficiently so that you touch as few pieces of paper as possible, the number of pieces of paper you need to touch depends on a lot of things, including the margin. If the margin is small, it doesn’t take much error to change who won. If the margin is big, it would have taken a lot of error to produce an apparent margin that large. The workload really does depend on the results. It depends on how the ballots are organized. Can you efficiently go in and sample? It depends on the data that you can export from the voting system. But the very simplest method really is very much like what I mentioned before on checking whether a coin is biased in favor of heads. It’s kind of, you know, numbering your ballots from one to however many there are, having software say, go give me ballot 17. OK, what does it say? It’s a vote for Alice, and you keep doing that until you have so many more votes for Alice than for Bob and the sample you’ve got that you’ve got convincing evidence.
In Orange County, California, which I think is the nation’s fifth-largest election jurisdiction, if I’m recalling correctly, we did some calculations for what sample sizes would be needed for the latest methods. And in 2020, I want to say that there were roughly 200 contests on the ballot in Orange County, not all on any single ballot, but if you look at all the different smaller contests and add up how many there were, it was about 200. They could have audited them all at five percent, meaning that if any of them were wrong, there was a 95 percent chance they would do a full recount to set the record straight before things were final. They could have done that for all 200 contests on the ballot, including some that were tied and required a full recount of those contests by looking at something like 0.7 percent of all the ballots cast, less than a one percent sample size.
Are ballot tabulation machines typically connected to the internet, and what questions should reporters ask to understand how they’re secured?
[01:03:51]
MATT DERIENZO: So we have a question from the Bellingham Herald in Washington State. And it’s about voting machines. So when you talk about that, they’re trying to draw a distinction between voting machines in states that have a lot of in-person voting and Washington State, where voting is by mail. And it says they say that we are told repeatedly by the county auditors and Secretary of State that these tabulation machines are not networked. Can you clarify whether the tabulating machines for paper ballots are typically connected to the Internet, or maybe help them with what questions they should be asking?
[01:04:29]
PHILIP STARK: You really want to know whether they’re on the same internal network at all as anything else, because very often there are public-facing systems that are on the same network as the tabulators, even though they think they’ve got firewalls or this or that. And the relatively recent CISA report documented that. There are a lot of voting systems that are persistently on the Internet. There was some lovely research by Kevin Skoglund about seven or eight years ago demonstrating that he was able to find connected voting systems by just doing port scans. Even if things are not directly connected to the Internet, very often the ballot definition files that say ballot style, blah, blah, blah, have the following contests on it. The first contest is this. The place to look for the vote target for this candidate in this contest is at this XY coordinate on the page, etc. The devices that are used to configure that information are connected to the Internet, and then you carry the memory stick over to another thing by Sneakernet, but it’s very easy to transmit malware that way, and there’s examples of that happening. There’s examples that Alex Halderman from the University of Michigan has demonstrated of passing malware back from voting systems where the actual physical device is accessible to the voter back to the central tabulator; that’s less of an issue in Washington state where the voters aren’t directly interacting with the voting system.
But it’s all completely open to insider threats and threats from the manufacturer, the software provider, external contractors who are doing various things. A lot of voting systems are configured to allow remote desktop access for technicians from the voting system vendor. Really, really bad idea but widespread. A lot of booting sorry, a lot of voting systems have WiFi, Bluetooth and cellular modems in them, which they should have none of those things.
[01:06:27]
MICHAEL ALVAREZ: I think, for the reporter, I think two key questions that we ask when we go and do these kinds of observation efforts. I’ve been doing these for 30 years. You go to the office, and you should do it now, like Marc said; don’t wait until election day, and look for two things. One: just look to see if the things are connected to the Internet; just look and see if there’s a physical connection. Step two: walk through what Philip just talked about. Walk through all the steps that will get information or materials on or off of that ballot tabulating system. That’s going to help you decide whether or not they’re using WiFi to move materials on and off, or whether, as Philip said, there’s a physical process that’s going to have some sort of device like a thumb drive or something, a card that’s going to be connected to another system that’s connected to the Internet. Just have them walk through the process and ask questions about how that information is getting on that system.
How can reporters make the day-to-day work of preventing voter fraud more visible, and who should they talk to?
[01:07:23]
MATT DERIENZO: So this is a really good segue into another question I really like. It seems like it touches upon a lot of what Marc and other people have said. It seems like a lot of election officials work to identify and prevent voter fraud. It occurs kind of out of the public view, and maybe that contributes to kind of uncertainty and concerns. How can reporters make this work more visible? What types of experts should they ask to interview, or are there particular processes that they can observe and report on, like you mentioned?
[01:07:51]
MARC MEREDITH: I got three right here. Where else?
[01:07:55]
MATT DERIENZO: And you can just click on “Ask an Expert” on SciLine.
[01:08:05]
MICHAEL ALVAREZ: But I think, first off, go and visit yourself. I think that the most important thing you can do is just go and watch it yourself. Many election officials in larger jurisdictions now will have webcams and web feeds. So inquire to see if they have those. And in many cases, there’s lots of opportunities for public observation, right? Sometimes it’s limited, but there’s still opportunities, and I think you should encourage people to go and watch it themselves and ask questions.
[01:08:33]
MARC MEREDITH: I agree with that. And a couple of things to add. One is there’s a really nice community of former election officials who often can speak a little bit more of the nuts and bolts of what they’re doing and maybe can provide more tangible examples than we academics can. So I think I encourage you, maybe not so credible, to talk to the people who are running the election right now as to talk to the people who ran it before, and I think they’ll almost universally be able to praise the stuff that election officials do to maintain electoral integrity. I think the other thing for journalists, I think it was the Pennsylvania vote beat. I might be getting wrong, wrote a story after a recent primary that said, “Oh, what a good job Pennsylvania did running this election because I think there’s this asymmetry in journalism.” It leads to a mentality of, like: Oh, we’re going to talk about things when they go bad and then just ignore the world when it goes good. I think a challenge for journalists is if you’re watching your local election officials run this upcoming election, and everything seems like it’s going great. That story and tell people about everything they’re doing because I do worry that we only hear about the problems, and that’s going to skew how people understand what election administration is all about.
How should reporters cover vote count shifts after Election Day, and how can they help the public understand that changing tallies are not evidence of fraud?
[01:09:42]
MATT DERIENZO: I’m so glad you mentioned Votebeat because I had intended to bring them up. If people on the line haven’t heard of Votebeat, it has been around, I don’t know, four or five years now. The editor is a wonderful person, Carrie Levine, whom I used to work with on voting rights projects or investigations. And they have a slack channel that reporters can join that have experts where you can ask them on demand, as a supplemental things a lot of good information that they’re putting out. I have one more question, and then we’ll ask you guys, to kind of just give us any kind of takeaways or final advice. But it’s kind of two-part. And also, it’s a preview of a briefing that we’re going to do around this time next week. So stay tuned if you’re on the line for that information coming out in the next couple of days, which will be about what to expect on election day in the aftermath. So the question is: We’ve all seen in the past where, especially in places like California, the early results will say one thing in terms of a political party winning or whatever. And then by the time the morning comes around or a week comes around, it shifts based on who votes by mail or other things or how things are counted or what towns in. Any advice you can give reporters on how to, like, cover that in a way that helps the public understand that that is not fraud, right? And then any advice you have in terms of all of these issues we talked about in the aftermath of the election when people are making claims.
[01:11:24]
PHILIP STARK: It’s almost like confusing what the radio announcer says with what the true score is in a ball game. The stuff that comes out early is early. In jurisdictions that have a large fraction of people voting by mail, it could be the minority of ballots that have been counted as of election night, but the vast majority of the work still to happen. So it is a process. The canvassing periods vary enormously by state. I think the shortest is seven or nine days and the longest, I think, is California at about 20 or 30 days. And a lot happens during that time. And so it’s hard work. It takes a while to get stuff right. But the fact that results changed during the canvas is not at all surprising.
[01:12:11]
MICHAEL ALVAREZ: I just posted a link to a paper we did on this, which provides a lot of systematic evidence and also links to some other papers that have been done to study this. I think, first off, ask the election officials why that’s happening. They’re going to be the people who are going to be able to tell you which types of ballot groups have been run through their system already, and that’s going to tell you a lot about what’s driving these blue or red shifts, as we call them in the business. But also reach out to experts. Marc’s done a lot of work on this. I have reach out to us, reach out to Charles Stewart at MIT, who’s done a lot of work on this as well, and we’ll help walk through.
[01:12:49]
MARC MEREDITH: And one thing I’ll add is, I do think among a sophisticated political community, they’ve gotten used to this idea at this point, but they assume that the past is going to predict the future. I think one thing we’re seeing right now in ballot returns is that there may be a little bit of shifts happening out there where some of this, the differences where Democrats are voting by mail and Republicans are voting in person, might not be as focal this time around. And so I do think one general thing you have to remember in elections is each one’s a new one, and when you assume too much, the past is going to predict the future, you can get yourself into trouble. And so, especially encouraged because of that, you know, reach out to the types of people who Mike is highlighting because I do think they may have some insights for how you can talk not just about how these things generally go, but also some of the things that are idiosyncratic to this specific election.
What is one take-home message you have for reporters?
[01:13:40]
MATT DERIENZO: Excellent. So I’m going to go in reverse order and just ask you to take maybe, like, 30 seconds to, like, leave us with something. Leave us with some wisdom. Dr. Stark?
[01:13:55]
PHILIP STARK: Hand-marked paper is a remarkably good technology for recording and counting votes, but it does have to be kept track of, and you have to make sure that whoever contributed to the paper pile was eligible to vote. I think that both sides of the political spectrum would agree that knowing that the paper record is trustworthy is incredibly important. And the arguments seem to be around why might it not be trustworthy, with one side putting a lot of emphasis on voting by non-citizens and fraud at that level. The other side largely denies that there are any vulnerabilities at all right now, despite the well-documented cyber vulnerabilities and other things. So the truth is somewhere in the middle.
[01:14:38]
MATT DERIENZO: Dr. Meredith?
[01:14:40]
MARC MEREDITH: I would just say to remind people in the media that the purpose of this system is that we have some preliminary results on election night, and then we have a final canvass, which is really part of the reason that’s there is to catch and correct errors that might be in the system. I do emphasize errors because I think more often than not, the types of things that we see don’t fit Mike’s definition of fraud, but just things that inevitably happen when you’re trying to tabulate the votes of millions of people. And so be careful not to let the process designed to make it work better undermine its legitimacy.
[01:15:13]
MATT DERIENZO: Great. Dr. Alvarez.
[01:15:16]
MICHAEL ALVAREZ: I’ll really, in some sense, echo what my colleagues have already said, and that is to say that the election process is really many layers, and I think the layers are what provide a lot of security and a lot of insurance that in the end, the results are as accurate as we can possibly make them. But I think from a reporter’s perspective, you have to be patient, let the process work its way out, especially for those of us in California; be patient with us. It takes time. But in general, we get it right.
[01:15:46]
MATT DERIENZO: Thank you, guys, so much. On behalf of SciLine and the reporters here, we really, really appreciate you volunteering your time to answer these questions and giving the message that reporters can reach out and the feature to get these stories right. Reporters on the line, please let us know at SciLine how we can help you, and stay tuned for future briefings. Thanks so much, guys.