Bruce McPherson, California Secretary of State, issued the "Volume Reliability Testing Protocol" (available at the Secretary of State's website) on Jan. 31, 2006 to define the procedure for performing volume tests on electronic voting machines:
"Prior to certification, all voting systems will be subject to volume testing in accordance with standards defined by the California Secretary of State. The purpose of this volume testing is to verify that proposed voting equipment will operate reliably in conditions approximating normal use in a polling place on Election Day...
The volume test shall be deemed successful if no more than 1% of the machines experience a failure that affects the record of the vote on the DRE or the VVPAT [voter verified paper audit trail] and if no more than 3% of the machines experience a substantive failure. Substantive failures will include, but not be limited to: errors that require the equipment to be powered off for resolution; errors that require a pollworker to take the DRE out of election mode to resolve; paper jams that are not automatically managed gracefully by the operating system; failures that require equipment to be taken out of service or replaced; errors, other than operator error, that require a ballot to be canceled and restarted; errors that prevent the paper audit trail from being fully displayed to the voter; errors resulting in an error message (not including voter warnings regarding improperly cast ballots); errors resulting in improper display of the ballot or the ballot image on the paper audit record; and battery failure at a point in time that is more than twenty percent less than published specifications for battery life."
Alfred DuPlessis, an electronic systems reliability engineer, offered the following definitions in a Dec. 2, 2002 email to the Institute for Electrical and Electronic Engineers (IEEE) Project 1583 Voting Machine Reliability Task Group (available at the IEEE Project 1583 website):
"Equipment Reliability: Reliability is traditionally the probability that a piece of equipment will perform its intended function over its intended operation time. When equipment does not complete its intended functions, it is said to fail. Reliability or more accurately unreliability is always a measure of the failures of equipment over time...
Equipment Failure: Let us define failure as the inability of the voting equipment to collect the votes or to transmit votes to another location for longer than 10 seconds. most of these failures are total outages of the equipment...
There are a number of potential ways to measure equipment reliability. Voting equipment needs to function for 16 consecutive hours. Because of this, the recommended way of quantifying reliability will take into consideration this length of time."
Are Electronic Voting Machines Mechanically Reliable??
Diebold Election Systems, a manufacturer of electronic voting machines, stated in a Jan. 23, 2006 press release titled "California Tests Find Diebold Touch-Screen Voting 100 Percent Accurate During November 2005 Election" (available on the company's website):
"The performance of Diebold's touch-screen technology was also verified during the extensive system volume testing conducted in September in California, when more than 11,000 votes were cast on 100 touch-screen stations with voter-verifiable paper audit trail printers with 100 percent accurate election results and very reliable system operation. The California Elections Division determined that the volume test would be deemed successful if no more than 1 percent of the machines experienced a failure that affects the record of the vote on the AccuVote-TSX or on the VVPAT paper trail. Diebold surpassed these test criteria with flying colors."
The Election Technology Council explained in their Oct. 2005 "Frequently Asked Questions" document (available on their website):
"How often do electronic voting machines break down?
DRE systems are relatively new, generally in the marketplace for less than two years. As a result, statistics in this area are limited. Vendor reported experiences with DRE break downs suggests that the rate is nominal. All vendors must submit their equipment to rigorous testing which includes tests to determine the ruggedness of the hardware. These tests assure that DREs can be expected to have a useful life well into the future."
Harris Miller, former President of the Information Technology Association of America (ITAA), stated in a Nov. 2, 2004 press release "E-Voting Machines Have Succeeded Today" (available at the ITAA website):
"Reports from electronic voting machine vendors and media coverage today indicate that electronic voting machines are recording Americans' votes efficiently and effectively. Media reports are focused primarily on long lines and other problems related to high voter turnout. Electronic voting machine issues that have been cited are related to human error, process missteps or unsubstantiated reports. In all cases, the issues related to electronic voting machine performance have been quickly resolved. Electronic voting machine vendors in constant contact with state and local election officials across the country are reporting no widespread problems."
Michael Shamos, PhD, JD, Distinguished Career Professor of Computer Science at Carnegie Mellon University, wrote in his paper "Paper v. Electronic Voting Records - An Assessment," published in the Proceedings of the 14th ACM Conference on Computers, Freedom and Privacy, 2004:
"By far the most justifiable criticism of DRE machines is that they fail during service or in some cases cannot even be brought into service on election day. There are numerous documented instances of such failures. These incidents are real. They are intolerable when they interfere with the act of voting...
They may 'hang up,' refusing to accept any more votes. The mechanical components, particularly the touchscreens, may develop dead spots or fail to register at all. Switches and buttons wear out. Circuits exhibit erratic behavior. These situations can result in sever voter inconvenience and loss of confidence in the process."
Stanley Klein, a software systems and technology expert, explained in his June 29, 2005 presentation "Statement for Election Assessment Hearing" (available at the Election Assessment website) before the election administration project Election Assessment:
"TrueVoteMD placed observers in 6% of Maryland polling places and observed 201 machine 'problems,' some of which included issues such as lack of proper help in using the machines. By my analysis, these 'problems' included 111 voting day machine malfunctions, or an 11.4% machine failure rate...
The 'unofficial' data is confirmed by evidence from official sources. The Maryland Board of Elections secretly placed all Maryland voting machines on lockdown following the 2004 General Election as a result of extensive machine failures experienced... The Maryland Board of Elections believes that a 7% failure rate of voting machines is normal. They may have determined this by comparison of their failure experience with that of other election administrators. A 7% rate may be a normal failure rate for voting machines that are designed and tested to a deeply inadequate specification, but it is seriously abnormal for properly specified, designed, and tested electronic equipment used in mission-critical applications."
John Gideon and Howard Stanislevic, both from Vote TrustUSA, stated in their Mar. 15, 2006 website article "Voting Systems Batch Test Results - Reliability":
"The 2002 (and 1990) voting systems standards require all voting systems to have a reliability, or Mean Time Between Failures (MTBF), of 163 hours, or a 9.2% failure rate in a 15-hour election day. We also learn that this is a woefully inadequate standard especially when compared to everyday items like an incandescent light bulb, which has an MTBF of 1,000 hours, a standard PC, which has an MTBF of 30,000 hours, and even New York City's aged mechanical lever voting machines.
With this in mind we looked at the results from the recently completed 'batch testing' of voting systems manufactured by Diebold, Hart InterCivic, and Sequoia Voting Systems, and put the information from those tests, provided by the California Secretary of State's office, into the MTBF formulae... These results are stunning. Even the best of the DRE systems tested is only 30% as reliable as an incandescent light bulb and this is the only machine to meet the federal Reliability standard. Also stunning is that all of these systems except the Diebold TSx tested in July, 2005 and the Hart eScan [an optical scan voting system] were recommended for certification."