What is the difference between qualification testing and certification testing?
General Reference (not clearly pro or con)
The Ad Hoc Touch Screen Task Force, commissioned by former California Secretary of State Kevin Shelley, released a July 1, 2003 report titled "Report to the Secretary of State" which included the following descriptions:
"All voting equipment and systems used in elections in California are required to be tested by the federal and state governments. Initial qualification testing is done by an 'Independent Testing Authority' (ITA) and uses guidelines adopted by the federal government for voting system performance and security. Both the hardware and software of voting systems are analyzed and tested...
Once voting equipment has received federal qualification, it is eligible to apply for certification by the state for use in California elections. This certification process requires further testing by an internationally renowned voting systems consultant on contract with the state. This consultant conducts performance tests to ensure that the equipment is accurate and secure and can conduct elections according to California law. In addition, the applicant must demonstrate the equipment to election officials, interest groups (such as persons who are blind or visually impaired), and others. The applicant is currently required to place the source code that operates the voting system in an escrow facility and to produce an extensive manual of procedures for the use of the equipment. The voting system is considered for certification at a public meeting of the state's Voting Systems and Procedures Panel."
Ellen Theisen, CEO of the Vote-PAD Company, stated in her 2005 report "Myth Breakers: Facts About Electronic Elections":
"Most states have laws or administrative rules that require election systems to be 'qualified' by NASED [National Association of State Election Directors] before they can be certified for use in the state. In the qualification process, an Independent Testing Authority (ITA), approved by NASED, tests voting equipment against the voluntary Federal Voting System Standards approved by the Federal Election Commission and more recently the Election Assistance Commission established by HAVA.
If the system meets or exceeds the standards, the system is placed on the list of NASED 'Qualified' machines and is assigned a NASED qualification number. Hardware and software are tested separately by different ITAs and are assigned separate NASED numbers...
Certification is performed by the states and involves checking the functionality to make sure that it meets the state's needs, for example, the ability to do candidate rotation on the ballot, to all cross-over voting, or to perform other functions required by state law."
Britain Williams, PhD, voting machine examiner for the State of Georgia, offered the following explanation in his May 5, 2004 testimony before the U.S. Election Assistance Commission (EAC):
"Before a voting
systems can be considered for use in Georgia, it must be examined by
the ITAs for compliance with the EAC Voting System Standards. Georgia
considers a voting system to consist of a specific version of each of
the system components: hardware, voting system software, and operating
the system successfully completes ITA qualification testing and is
issued a NASED qualification number, it can be brought into Georgia for
State Certification Testing. The system to be tested is not obtained
from the vendor but is transmitted to the KSU [Kennesaw State
University] Center for Election Systems directly from the ITAs.
KSU Center for Election Systems conducts a series of tests on the
system. Some tests examine the level of difficulty associated with
operating the system. Another tests the capacity of the system to
accommodate the maximum number of ballots that might be cast in a large
precinct or at an in-person absentee voting location. One test is
specifically designed by the KSU Center for Information Security,
Education, and Awareness to detect fraudulent or malicious code that
might be present in the system. This test is designed to wake up any,
so called, Trojan horse that might be present. In all of these tests a
known pattern of votes is cast and then compared with the output of the
Deirdre Mulligan, JD, Director of Samuelson Law Technology and Public Policy Clinic at University of California Berkeley School of Law, and Joseph Lorenzo Hall, PhD candidate, stated UC Berkeley, in their 2004 white paper "Preliminary Analysis of e-Voting Problems Highlights Need for Heightened Standards and Testing," submitted to the National Research Council of the National Academy of Sciences' Committee on Electronic Voting:
"Voting systems are tested against the VSS [Voting System Standards] by independent Testing Authorities (ITAs) that are certified to conduct these tests by the National Association of State Election Directors (NASED)... These ITAs conduct manual and automated source code review, documentation review, environmental 'shake-and-bake' testing, and some systems-level testing of the full voting system... Each voting system, that does not predate the VSS themselves, must pass both hardware and software testing by an ITA before it is considered 'federally qualified' and given a NASED identification number...
Some states may or may not require voting machines they purchase and use to have federal qualification. Some states further regulate and test voting systems to ensure that they meet specific local requirements absent from the federal qualification process or they may require additional testing where the state has found federal qualification to be deficient. Specifically, 35 states require both federal qualification and additional state certification, 9 require only federal qualification, 5 require only state certification (Arizona, New Hampshire, New Jersey, North Carolina, and Vermont) and 2 require neither federal qualification nor state certification (Mississippi and Oklahoma)."