CERTIFICATION TESTING: STILL WAITING FOR HOME DELIVERY.

ASSOCIATION EXECUTIVES ARE LOOKING-BUT NOT leaping-into browser-based Internet testing for their certification programs. They love the idea's apparent simplicity: By taking advantage of the tremendous advances in technology, associations could allow candidates to take their certification tests anytime, anywhere via the Internet.

No more proctors. No more paper-and-pencil tests. That's the theory. If only the reality were that easy.

Despite rapidly advancing technology, your association's certification program still requires a proctor for security, which effectively eliminates home delivery.., for now.

What's the reality?

The advantages of offering certification candidates the ability to take browser-based Internet tests delivered to their homes or offices are readily apparent. Two come quickly to mind.

Convenience. Your candidates could log onto their home or office computers and take their certification tests anytime, anywhere, when they're ready. They wouldn't have to wait for the designated testing date in the designated testing city. This flexibility is particularly important to associations with a large international contingency, explains Mike Littman, vice president, certifications, Brainbench, Chantilly, Virginia.

Immediate response. Upon completion of their Internet tests, your candidates could find out their scores with no waiting. Compare this instantaneous response to the current lag time between test delivery and test results, which may range from several weeks to several months.

Despite these advantages, associations are not yet using browser-based testing for their certification programs. The primary reason is security--or lack of it. A secondary reason is system reliability.

Imagine this scenario: Mary Smith decides to take your association's certification exam. She logs onto your association's Web site, enters a code, obtains a test, and passes with flying colors. Now, how can your association ascertain that Mary--instead of Tom, Dick, or Pat--took the test?

The reality is, you can't. Not yet anyway. "Everybody wants to be using the Web for certification," says Wade Delk, executive director, National Organization for Competency Assurance, Washington, D.C. "It's a great idea, but how do we do it? Security is the hot button. Because of the public safety and trust involved, certifying organizations must be able to guarantee that the person taking the test is supposed to be taking the test. Otherwise, we would water down certifications. We certainly want to make sure you can continue to trust certifications."

Michael E. Gallery, CAE, deputy executive director and chief operating officer, American College of Emergency Physicians, Dallas, agrees.

"Security is the number one issue," he says. "You have to know, 'Is the person taking the exam the proper person?' You can obviously do that by requiring identification, which means your candidates have to show up at a testing center and then verify their identities."

He jests that technology does offer one solution, such as retina scanning, but questions whether it's appropriate for this use. "That's almost science fiction," he says. "It may be technically feasible but would be cost prohibitive."

Another potential solution could be using electronic signatures, Delk says, which he describes as an encrypted code that you would electronically apply to documents and would be as valid as your handwritten signature is today. "Eventually, we'll get to the point where the government recognizes electronic signatures," he says. "As this kind of technology comes online, the security issues will become fewer and fewer. Until then, you have to have procedures in place to verify who the person is.

This fear of security breaches is a sad reflection on society, he continues. "If everyone were honest and on the up and up, certifications would be available on the Internet right now. You could just study at home, take the test, and send it in. Obviously, that would be a perfect world. We don't have that."

Candidates want reliability

The secondary problem associated with browser-based, home-delivery testing is system reliability. "We call our Web-based, browser-based testing system a low-stakes testing system because it's in an unproctored environment, says Don Wagner, executive director, business development at Vue, an NCS Pearson Business in Minneapolis. "It's also less reliable because you're dependent on the performance of the Internet. The reliability is not there in terms of engine performance. Your connection can get dropped."

He thinks browser-based Internet tests are just dandy for low-stakes testing, such as continuing education tests and practice tests, because a candidate's certification doesn't depend on system performance. "If you are sitting at a computer taking a certification exam and all of a sudden the Internet slows down, you're not going to be very happy," he says. "Whereas if you're taking a practice test or are in a continuing education environment, you will have a higher tolerance for performance degradation. When you're taking a timed test and it means whether or not you get a credential, your tolerance for performance problems is not very high."

For association certification programs, he recommends a "high-stakes delivery system." For example, his company delivers candidate and certification test information via the Internet to 2,600 testing centers and retrieves the results upon test completion. "For high-stakes tests, we use the Internet like a network," Wagner explains. "We connect to our testing center via the Internet, download information, upload information, then we disconnect. Our testing centers are not connected while they are delivering tests."

At what cost?

Whether you choose a testing center environment or someday take advantage of a browser-based delivery system, you will face three additional issues if your association is moving from a paper-and-pencil certification exam.

1. Culture shock. Taking a certification exam terrifies some people--even if they're experts in their field who know their subject matter like the back of their hand, according to Ralph J. Nappi, CAE, president, American Machine Tool Distributors Association (AMTDA), Rockville, Maryland. For those folks, changing to a computer test "puts the fear of God in them."

2. Increased item bank. Internet or computer-based testing typically requires a large item bank so that each test can be unique. This reduces question exposure. "If your candidates are able to take their certification tests whenever they want, on demand, your association must have the ability to scramble questions," Nappi says. "Which means you have to have a large item bank--much larger than most associations have today."

Don Wagner, of Vue, warns association executives not to go overboard by requiring an excessive item bank. "There are things you can do from a computer-delivery perspective that will reduce the impact of exposure," he says. "I can tell you there are global organizations that are delivering tests that don't have thousands of items in their banks. They simply monitor the performance of their tests to see if the performance of items changes."

3. Sticker shock. "Let's face it," Nappi says. "To put a certification program online initially is going to cost a fair amount of money. That is the only thing that has prevented AMTDA from making the leap. But I will also tell you to keep an eye on costs. They are coming down."

Nappi explains that associations with a large pool of candidates will find the unit costs more competitive. Associations with a limited certification pool, such as AMTDA, face higher per-person price tags. "My candidate population is only a few thousand. We only give our exam two times a year to anywhere from 40 to 60 people. To put our test online is a substantial cost at this point."

Francine Butler, CAE, president, Resource Center for Associations, Wheat Ridge, Colorado, agrees. "I've just gone through trying to get information from three companies that offer online testing. For our organization (which offers a variety of industry-related certification programs), it's not going to be possible for a while because it's just too expensive for us to implement right now. For associations that offer an exam and can meet the minimum number of people, I just don't see why everyone doesn't jump on this. I wish we could."

Wagner believes that candidates are willing to pay more for computer testing as long as you allow them to test on demand and receive immediate results.

"A candidate cares about computer-based testing for two reasons," he observes. "`I don't have to wait until the second Saturday in May and the second Saturday in November....I can test when I want.... I can test on demand' and 'I get my results quickly.' Association executives take away the benefits for candidates and then tell me my price is too high. My point is if you allow the candidates to realize the benefits, they are willing to pay more. It's really hard to say, 'OK, candidates. I'm going to double your test price from $100 to $200, and there's no real tangible benefit to you.' That's a hard sell."

A natural progression

Timothy D. Kent, CAE, executive vice president, North Carolina Association of Realtors, Greensboro, and a member of ASAE's CAE Commission, believes that the proliferation of computerization and usage will prompt more and more associations to proceed with online certification testing--using proctored testing centers until security and systems issues can be resolved for home or office delivery. "Members are becoming more familiar with working and learning online," he says. "This is a natural progression for a forward-thinking organization to provide to its membership."

In his business dealings, Wagner finds that most association executives are already convinced of the value of delivering their certification tests online. "They wonder whether they can afford it, but they want to do it," he says. "In society today, everybody wants everything to happen now. Candidates don't like the whole idea of doing this paper-and-pencil thing twice a year. They are saying, 'There has got to be a better way.' Especially candidates who fail. They don't want to have to wait another six months to take the test. They want alternatives. They want satisfaction now."

Margo Vanover Porter is a freelance writer based in Locust Grove, Virginia.

GETTING CONNECTED

The National Organization for Competency Assurance, Washington, D.C., recently released Certification Testing on the Internet, a 20-page overview covering such issues as registration, payment, delivery, and scoring Internet certification exams.

The monograph explains that tests offered on the Internet typically fall into one of two categories:

DISCONNECTED MODEL. Before an examination begins, test files--such as test taker information and the entire test--are sent via the Internet to a test center or test station. A live Internet connection is used to download the test and to return results.

"This model is advantageous if the test includes large graphical or multimedia files," the monograph reports. "A second advantage of using a disconnected model is that it protects the test taker from interruption due to Internet disruption."

CONTINUOUS CONNECTION MODEL. Using a continuous connection examination, Internet connectivity is maintained throughout the test. "Questions and results are continuously relayed back and forth from test taker to test delivery vendor or the test sponsor," authors say. "This may involve sending each item, one at a time, to the computer. If connectivity is lost during test administration, the test cannot be completed until connectivity is restored."

The study concludes that "continuous online testing is not a viable option for high-stakes certification examinations that require verification that the registrant and test taker are, in fact, the same person. This may be possible in the near future. If this seems overly optimistic, consider that the Internet as it is currently used did not exist 10 years ago."

For a copy of the study, contact NOCA at 202-367-1165. The price is $25 for NOCA members and $35 for nonmembers.

INTERNET CAE TESTS? MAYBE SOMEDAY

Should ASAE's CAE exam move online? Not yet, agree members of the CAE Commission, who have been debating the question. Security--or lack of it--ranks as the main reason to postpone an Internet solution for the certification program.

"The members of the CAE Commission would like to be on the leading edge of technology," says Timothy D. Kent, CAE, executive vice president, North Carolina Association of Realtors, Greensboro, and a member of the commission. "But we do not want to compromise the integrity of the examination. Obviously, that's very important."

Michael E. Gallery, CAE, deputy executive director and chief operating officer, American College of Emergency Physicians, Dallas, points out that the CAE item bank needs bolstering as well. "You really need a large pool of items, and those items have to be of fairly equal value to ensure the same level of difficulty on each test. To create parallel examinations, you need a pretty robust question pool, and we're not there yet."

Ralph J. Nappi, CAE, president, American Machine Tool Distributors Association, Rockville, Maryland, believes these two issues can be worked out in the not-too-distant future. "The way things are moving, you have to start to develop the process," says Nappi, a past chair of the CAE Commission.

His advice: "Take care of any cultural issues and beef up the item bank now, so when our technology brethren cure the security issue, we're ready to rock," he says. "The security problem will be solved. There's not a doubt in my mind."

ALONG THOSE LINES...

The Internet has forever changed association certification and testing programs. Need proof? Here are three examples:

1. ASSESSMENTS AT A DISCOUNT. Don McLaurin, CEO, National Association of Computer Consultant Businesses, Alexandria, Virginia, recently signed a strategic partnership agreement with Brainbench, a company headquartered in Chantilly, Virginia, that offers Internet assessments and certifications. NACCB members, who supply information technology workers on a contract basis, receive a discount on Brainbench tests. The association receives a royalty for each assessment taken by members.

The program, in effect for about six months, is very popular, McLaurin says, and helps determine a candidate's competency in different technology areas. It is not a certification program, he explains, and tests are not proctored.

2. WEB SITE INFORMATION. Robert A. Booth, Jr., executive director, American Board of Examiners in Clinical Social Work, Salem, Massachusetts, uses the association's Web site to its full advantage. Social workers who are contemplating certification can find out all of the requirements online. "Anyone who wants to apply for our credential does not have to approach us at all. They can get all that they need through our Web site."

In addition, consumers who are looking for board-certified clinical social workers with a particular specialty can, by entering a few parameters, search a database and come up with a list of names. "This is an important information tool for us," Booth says. "It is highly desirable for us to make that available to the world of online end users."

3. CERTIFYING SOFTWARE. ASTD, Alexandria, Virginia, recently launched a program that certifies software for e-learning. Dubbed ELC--for e-learning courseware--the program evaluates the instruction design and usability of off-the-shelf asynchronous multimedia and Web-based courses.

"We're not certifying content," points out Anne Blouin, CAE, ASTD's director of e-line certification. "It is very subjective to determine whether the content is accurate....Since it needed to be legally defensible, we decided not to evaluate content."

Blouin hopes the electronic certification will improve the overall quality of e-learning courseware, as well as help trainers make informed buying decisions. "There are no standards that exist today. As a result, there is a lot of poorly designed courseware out there. Dependable quality standards are vital to the health and credibility of the new e-learning industry."

ASTD established a subsidiary, the ASTD Certification Institute, to handle the program. Reviewers will evaluate courseware against a predefined set of 32 certification standards, Courses meeting the standards will be posted on the association's and institute's Web sites.

INSURANCE ISSUES

If organization is considering an online certification program or has one in place already, you're probably aware that your insurance options are limited. As organizations move their certification programs online, data become more vulnerable. But since compromised data is not an insurable risk, what can you do to protect your content?

COPYRIGHT YOUR CERTIFICATION EXAM. This gives your association recourse in the event that the exam is stolen and distributed, explains Amy Doherty, assistant vice president, Aon Association Services, Washington, D.C. This is critical whether the exam is provided online or offline.

TAKE EXTRA STEPS TO PROTECT THE CONTENT OF THE EXAM QUESTIONS. "Most, if not all, insurance carriers will be unwilling to pay to re-create a new pool of examination questions in the event that the questions are compromised," Doherty says. "Most property and casualty insurers will provide coverage to restore and replace an association's existing documents. There are limitations to the extent of this coverage, however, and possible exclusions would be accounting or programming errors; computer deficiency; and unauthorized viewing or copying."

CHECK WITH YOUR LOCAL AGENT OR BROKER TO REVIEW POLICY LANGUAGE. "The insurance industry is grappling with the issue of cyberliability, and many insurance companies are in the process of developing policy language to address this unique and evolving exposure."