Testing, testing: IHEs are trying to ensure the success of students by testing their abilities using the latest online assessment software.

In order to produce learned, well-spoken alumni, IHEs nowadays need to assess the skills and abilities of students when they first walk onto campus. "No college wants to churn out people who don't know their subject," one software company president has said. IHEs already know the GPA and SAT score of a student, as well as what his or her teachers thought of them through letters of recommendation, but sometimes, freshman arrive with reading, writing, and math skills that are in need of remedial education.

That's where those pen-and-paper placement tests usually enter the equation. Now, however, paper tests are ceding to online assessments designed and powered by a handful of software companies.

Tom Ewing, spokesperson for Educational Testing Service (ETS), knows that college faculty members, who in recent years have been relegated to teaching essay-writing instead of delving into the world of ideas, decry students' writing abilities.

But, he says, "It's a problem not only in higher ed, but in business as well. Employers complain that a person out of college can't write, or perform even basic math. They are finding students are tacking in skills central to 'success."

PUTTING STUDENTS TO THE TEST

At Penn State University, 12,000 freshmen are tested each fall in English and math, and sometimes even in chemistry. Students take online assessments in an unproctored environment using LXR*TEST, created by logic extension Resources, a Georgetown, S.C., company that specializes in developing software tools for testing, certification, and licensure.

"The benefits are mostly for students," says Ralph Locklin, senior measurement specialist at the Schreyer Institute for Innovation in Learning at Penn State. "They can complete it at a place and time that they choose. In the past, tests were proctored and all students would be crammed together. Now, students can choose time or day. We take great pains to tell them to get a good fix on their ability."

Kirkwood Community College (Iowa), the state's third-largest college system behind Iowa State and the University of Iowa, cut class time by five minutes down to 50-minute class periods, so instructors welcomed Questionmark's Perception, according to Rich Edwards, professor of learning services. Questionmark, based in Stamford, Conn., created "Perception for Web," which is used to administer assessments using the internet or a school's intranet.

"A lot of faculty find an advantage in online testing," says Edwards. "Tests are not given in class, so there is more time for instructors to teach."

He says online testing is even more beneficial for students. "Students get a week's window to go anytime to the testing center, and they get an immediate score."

Kirkwood, which serves 15,000 students in a seven-county area in eastern Iowa, found many advantages in using Perception, a system Edwards considers extremely secure and convenient. He says that students attend classes at several different campuses, so taking a day out of their schedule to congregate at the main campus for an assessment is not easy. Instead, students go to the main campus for testing at their convenience.

The community college system has used the untimed Perception tests since 1998. Kirkwood administered more than 7,000 Perception tests on its main campus in 2004.

Edwards listed a number of benefits for using Perception: It produces a more efficient and effective testing environment, creates a high degree of security, removes the fear of Losing paper, obtains immediate scores, and provides a degree of quality control. Plus, "students like accessibility," he says.

Kirkwood also uses ACT's Compass to test incoming students on reading, writing, and math skills.

WebCT, a Lynnfield, Mass.-based provider of e-learning systems, helped the University of Alberta, Edmonton (Canada) pull off an efficient way to test incoming freshmen.

"They will have their name added to the course List, but will not be officially added until the placement test confirms their ability for math or English," says Barb Ross, chief operating officer of WebCT.

Ross says the system "saves wear and tear on the student and institution." It is beneficial for the students, because they will be placed in the appropriate math or English class. It is also good for the university, which can take a look at the Level of achievement of incoming freshmen, and continually assess and improve its programs as they evaluate those results, she says.

John Hughes, vice president for academic affairs at The Master's College (Calif.), a Christian school, says that freshmen assessments help gather information on its 1,000 undergraduates.

However, size matters when it comes to testing, as revealed by some schools.

"Having large numbers of students creates different kinds of problems for administrators that smaller schools don't have," says Penn State's Locklin. Penn State tests more than 12,000 incoming freshmen each year, but, he says, "We're trying to be student-centered in everything we do."

However, in a community college setting like Kirkwood, with its generally open-door policy, Edwards says, "It becomes extremely important that students know what they need to improve on."

Although Kirkwood continually assesses students, Edwards considers it "more critical at the front end," when students are first admitted.

"You typically, philosophically, keep your doors open," he says.

WHEN ASSESSMENTS GO WRONG

Some software companies agreed that assessing students can create unwanted test anxiety. "You can worry students," says Questionmark President Eric Shepherd. "Ultimately, you want students to perform in a work environment. Some schools might overstress students."

And although incoming students are typically more web-savvy than ever, Melissa Anderson, a pedagogical advisor at Blackboard, a software company specializing in "e-education," says, "If [students] don't have experience doing online tests, they may be at a loss."

"The placement testing provides valuable information because it's standardized," says ETS' Ewing. "But the information is only useful when it's combined with other factors like a good AP score, SAT score, GPA, the personal essay, and the recommendation."

"We do have pitfalls," admits TMC's Hughes. "You add a whole extra layer of assessment that is not complementary to what students are studying and have no motivation to do it."

WebCT's Ross says the results of tests are not helpful to a university or to students if the test framework is inadequate. "As always, assessment is one of those issues that come down to the design of the assessment itself. The quality of the student's answer is only as good as the test's design."

"Student assessment tools are an integral part of the learning process, but it matters when or how you do it," says Kirkwood's Edwards. He adds that despite the licensing cost, assessments are necessary. "In the long run, it is a cost-efficient testing tool."

On the other hand, Penn State's Locklin doesn't think there is a particular downside to student assessments. "It's just another opportunity for us to find out a tittle more information about who our students are," he says.

As might be expected, the companies give each other very harsh critiques. Some think of themselves as highly specialized, focusing on security and analyzing students' results, while others claim to offer a broader experience by featuring chat sessions and class schedules.

Ross says WebCT integrates third-party software, like Questionmark, but indicates that students are able to track their results and can participate in chat sessions, even if they have yet to matriculate. "[WebCT] take students down different paths, and only release a certain question if they get above this number or group or by a certain date."

However, Shepherd says what makes Questionmark different is its enhancements. "We offer security, we lockdown PCs--even with open-book tests--and include sounds and videos," he says. "We offer a different presentation of questions and a statistical analysis, but present it in a consumable way for the student to understand."

On the other hand, Blackboard's Anderson says her company's product offers a "more robust assessment experience and ease of use."

FUTURE OF ONLINE ASSESSMENT SOFTWARE

Penn State's Locklin raves, "The really exciting thing about testing is new tests."

Locklin praised Questionmark, calling it an "exciting development." Questionmark is known in the industry for testing students using pictures, not just words, since not everybody learns in the same way. "You may be missing students that really need to see a picture," says Locklin, of word-based assessments.

"Things are much different now than they were five or 10 years ago," says Questionmark's Shepherd. "There are more tests, and people see the advantage of being tested. As our society matures, we see the benefits."

Shepherd anticipates plenty in the future for assessment software, such as better stimulus, videos, sounds, simulations, and better measurement of abilities.

"We don't want to show students a thousand statistics, and we don't want to merely show a pass or fail score," he says. "We want to show what the student did wrong and how they can improve on it. The student then sees the value of it."

RELATED ARTICLE: To fund or not to fund: debating remedial education.

State colleges and universities already had enough on their plates when newly appointed U.S. Secretary of Education Margaret Spellings announced that public IHEs, along with public elementary and secondary schools, would be required to prove they are doing their job: educating students.

Remedial education has spurred a debate among IHEs, faculty and even legislators, especially since more than 76 percent of students are enrolled in public institutions, and approximately two-thirds of those students are matriculated at four-year public schools, according to a 2001 report by the National Center for Education Statistics (nces.ed.gov).

However, the American Council on Education's Center for Policy Analysis (www.acenet.edu) reported that in 1999, 20 percent of freshmen and sophomores said they had taken a remedial course, white 32 percent of all students admitted to enrolling in a remedial course at some point.

And at university systems such as the University of Nevada, where 40 percent of incoming freshmen needed remedial education in the summer and fall of 2004, the stakes are even higher.

The state has notified the university system that it will stop paying for remedial courses at state universities by the 2006 school year, and now the system is moving all its remedial courses to the community colleges in that year, where it will be less expensive to administer the courses.

"It's one of the biggest issues out there for faculty, and every college, including the most selective Ivy Leagues," says Matthew Greene, educational director at Howard Greene & Associates (www.greenesguides.com), a Conn.-based education consulting firm. "Students don't know how to write or read critically."

Legislators argue that remedial courses drain states of funding that could easily go toward other education-related endeavors.

But Greene says that with remedial training, the student may make it through the freshman year, so "they're more likely to persist through years two, three, and four. And the longer they go, the likelihood of completion gets higher and higher," he says. "It's that huge freshman year drop-off and sophomore year drop-off that colleges are so worried about."

Tom Ewing, spokesperson for the Educational Testing Service (www.ets.org), which designs the SAT and GRE, but does not create traditional placement tests, thinks such tests "only give you a snapshot of skills. They don't measure creativity or ability in science."

Greene explains, "A lot of colleges don't call these classes remedial now. It's a sort of common denominator-level of writing or math. So it could be a quantitative course requirement, but if you get a 650 on the math section of the SAT, you don't need it."

"I think the issues will be different for different schools," says Greene. "For a community college system, that's largely seen as their mission, as part of their job. They want to offer all Levels of courses for students to come in.

"For the four-year state system, it's a resource issue for sure, but it's certainly a complaint that the college faculty has. The focus gels pushed down on helping the high schools. This is a big initiative out there. Colleges are now taking a more active role in helping high schools understand what the students need to get through to college."--JAV

RELATED ARTICLE: Getting past ABC and 123: ETS' groundbreaking exam.

Besides the traditional placement tests that assess reading, writing and math skirts, the Educational Testing Service has introduced the Information, Communication, and Technology (ICT) Literacy Test.

"It helps IHEs with the selection process to assess if students have the essential technology information processing ability to succeed," says Ewing.

"The ICT Literacy Test assesses students' ability to access information, and interpret and communicate their findings in a technological environment," he says. "It looks at their thinking process as they go through the test."

Students are asked scenario-based questions in real-time. One question might first describe a situation where a woman has been diagnosed with a hole in her heart, and her doctor recommends surgery. The student is asked to scour the web to find three sources of information to support the conclusion, and explain why the source is considered reliable.

"We created this test because universities wanted it, but didn't have the knowledge and ability to create it," Ewing says.

The ICT test was designed with advice from the University of California and University of Texas systems, among others. The pilot version of the test is being used at universities across the country. One such school is Portland State University (Ore.). Jennifer Dorner, social sciences Ubrarian at PSU, says that if students matriculated since their freshman year do worse on the test than transfer students, PSU will adjust its curriculum.

According to ETS, "In the future, the test may also be used as a pre- and post-program assessment for selected university programs."--JAV