A Williams College professor has apologized for a “lack of clarity and due diligence” after his statistical analysis of Pennsylvania mail-in votes was used by conservative lawmakers to push unsubstantiated claims of voter fraud.
The analysis by Steven Miller, a professor of mathematics at Williams, has drawn criticism from statisticians for failing to meet basic standards for a statistical analysis. Academic peers called the analysis “irresponsible” and “naive” for ignoring the shortcomings of the underlying data.
In a signed affidavit, Miller claimed to show that more than 89,000 ballots requested by Pennsylvania Republicans either were not counted by the state or requested by someone other than the registered Republican. He used data provided by former Donald Trump campaign staffer Matt Braynard.
Election officials have called the voting process secure and said they found no evidence of widespread voter fraud. Former Vice President Joe Biden won Pennsylvania by 80,000 votes, a victory that was certified Tuesday.
Miller told The Eagle that he made a mistake separating his analysis of the data from questions about the reliability of the data itself.
“Especially as the consequences are so important, I should have made a greater effort to go deeply into and share how the data was collected and not treat this solely as an independent calculation,” he wrote in a statement Monday night.
He said he was not claiming that voter fraud had occurred but maintained that “the extrapolated numbers here are significant” and deserve more attention.
According to his analysis, Miller used phone survey results from several thousand voters to draw conclusions about trends among about 165,000 Republican mail-in ballots in Pennsylvania that were requested but not returned to the state.
In other words, his study took up the question of whether Republican voters who requested ballots had been disenfranchised, either because their ballots were returned but not counted by local election officials, or because their ballots were sent to someone else in the first place.
The data Miller used came from Braynard’s “Voter Integrity Fund,” a group led by former Trump campaign staffers and government employees, Miller confirmed to The Eagle. He said he was not paid for his work.
Braynard collected the data by contracting call centers to get in touch with Republican voters across six swing states, in an apparent attempt to root out election fraud, according to reporting from several outlets. He told the Pittsburgh Post-Gazette that his operators were asking voters three questions: “Did a person with your name vote? Did you request a mail-in ballot? Did you return a mail-in ballot?”
In his analysis, Miller wrote that the group called 20,000 Republican voters in Pennsylvania who, according to state records, had requested but not returned ballots. In all, 2,684 agreed to answer questions, he said. Of the respondents, 463 reported that they actually had mailed in a ballot and 556 reported that they had not requested a ballot in the first place, according to the report.
Miller extrapolated from those numbers that 44,892 to 48,522 Republican voters on the state’s list of “requested but not returned” mail-in ballots had, in fact, returned their ballot, only to see it excluded from the state’s final vote count. He also estimated that an additional 40,875 to 53,909 ballots were requested in the name of a registered Republican by another person.
Right-wing news outlets and politicians, including Federal Election Commission Chairman Trey Trainor, a Trump appointee, and Republican U.S. Sen. Rand Paul, seized on the analysis as confirmation of President Donald Trump’s claims of voter fraud.
‘Irresponsible and unethical’
But, Miller’s affidavit was met with sharp criticism from his peers, who agreed with his later statement that it was wrong to separate the basic mathematical analysis from questions about the validity of the data.
Statisticians, mathematicians and other academics who work with large data sets told The Eagle they were deeply concerned that the data collection was flawed and therefore the results meaningless.
In his statement, Miller said he should have included a discussion of questions about the data collection in his report, including how Braynard’s group chose which people to contact and whether the high nonresponse rate might have created bias “in this particular issue.”
He stopped short of directly critiquing the data that Braynard collected. But, fellow academics said they hoped he would acknowledge that he had drawn conclusions from flimsy evidence.
Richard De Veaux, vice president of the American Statistical Association and a colleague of Miller’s in the Williams College Department of Mathematics and Statistics, called the estimates “completely without merit” and agreed that Miller erred in publishing his results without addressing issues in the underlying data.
“To apply naïve statistical formulas to biased data and publish this is both irresponsible and unethical,” De Veaux wrote in a statement to The Eagle. “It is the statistician’s responsibility to verify the data, or to provide disclaimers if that can’t be done.”
Carina Curto, a professor of mathematics at Pennsylvania State University, said Miller’s numbers “are almost surely wrong” because they rest on the key assumption that the people who answered the phone are actually a representative sample of all the Pennsylvania Republican voters who requested but did not return a ballot.
“This small sample from the phone survey almost surely has large sampling biases and systematic errors,” Curto said. “There is absolutely no reason to believe it is representative of the larger population.”
Lior Pachter, a computational biologist at the California Institute of Technology, said that simple issues, such as incorrect phone numbers, could have accounted for some of the concerning patterns that Miller saw.
“There’s no guarantee that the people they talked to were the actual people they meant to call,” he said.
He criticized Braynard’s group for not putting the same questions to Pennsylvania Democrats as a control group. More than 230,000 Democrats in the state did not return mail-in ballots that they had requested, according to the U.S Elections Project.
“What happened here is somebody had a question, to which they had an answer they had decided on ahead of time,” he said. “At a minimum, if they actually wanted to figure out if there was election fraud, they would have also polled Democrats.”
Pachter also expressed concern that Miller’s academic credentials are giving validity to claims about election misconduct that Trump has pushed for weeks. “He has a degree from Yale, he immediately looks like an expert, so I think this is a very dangerous piece.”
Stands by concerns
Even as he acknowledged his missteps, Miller defended his decision to carry out the analysis. He said he agreed to make the calculations because he felt it was “worthwhile to know how well these systems worked.”
“I am not concluding fraud happened, or that state outcomes should be changed,” he said. “What I said was, assuming the accuracy of these numbers, then we have a large number of people who had a ballot requested in their name but say they did not request it, and we have a large number of people who said they mailed their ballot back but it was not counted.”
The results of his work, he said, “should encourage more analysis of this important problem so we can see how well things worked and, if there are issues, how we can do better in future elections.”
He also said Braynard later answered key questions about the data collection “but this should have been explicitly in the report so that people can see for themselves whether or not the data is reasonable.”
In interviews with The Williams Record and the public radio station WAMC, Miller has gone on the record about being a conservative. But, he told The Eagle he came to the analysis without a partisan bent, pointing to work he did this month to debunk a separate claim of voter fraud.
In a statement to The Eagle, Williams College Chief Communications Officer Jim Reische wrote that “Williams faculty are free to pursue political and other personal interests outside of work without college interference. Steven Miller doesn’t claim college endorsement of his election analysis, nor would we grant it.”