Beware the writing assessment: Q & A with George Hillocks Jr.

Q Why is "teaching to the test" especially harmful in writing instruction ?

A Examinations of all state assessments, and interviews with more than 400 teachers and administrators in six school districts in five states, found that large-scale writing assessments in most states have a negative effect on what students learn. Teachers and administrators restrict their writing programs to what they hope will achieve high scores.

Most state tests ... are conducive to formulaic writing. [They] provide a prompt to which students must respond in a limited period of time with no chance for finding information that can be used in the writing. Many teachers resort to formulaic writing [assignments], assuming that students will be able to write if they have the structure.

In one district in Illinois, for example, it is district policy for teachers to teach file five-paragraph theme from third on up to 10th grade. One third-grade teacher said, "We pound it, pound it, pound it!" They pound it so much in most schools that directors of college writing programs in Texas and Illinois have told me that one of their most difficult tasks is to dislodge such writing from their students' psyches. The vast majority of writing assessments are like those in Texas and Illinois.

Q In what ways can teaching basic essay structures hurt student writing?

A In our research, the most frequently taught "basic essay structure" is the five-paragraph theme. The problem is that it does not represent any real essay I have ever seen, and it is not basic in any way. Everything about it indicates that it short-circuits thinking. In fact, I think it is taught so that students do not have to think. [The structure is] limited to five paragraphs, three of which are supposed to elaborate the three points that appear in the first paragraph. A writer cannot deal with counter arguments or even recognize them for fear of losing points.

Q What aspects of the judging process for standardized writing assessments concern you most?

A Most states send the writing to independent contractors who score the papers. Their effort is to score the papers at the rate of one per minute or faster. All the rater can look at is the outward structure of the piece and the apparent elaboration of points. Further, the [judging] criteria ... does not begin to deal with the relevance of the logic of the supposed support, permitting any kind of nonsense.

In scoring guides from both Texas and Illinois, we found pieces with perfect scores that were written as parodies of the assignment. One writer responding to a prompt asking that he nominate someone for the "best-relative-of-the-year award" uses the fiction of an unreliable narrator, naive about all of Grandpa Dulong's dishonesty and peccadilloes, suggesting they are worthy of great praise. The evidence, which we are meant to see through, cannot support the contention that Grandpa Dulong is generous, helpful and caring. On the contrary, it demonstrates that he is a rascal at best, not a person worthy of the award. The [judging] criteria have no way of evaluating evidence.

Q What types of writing assessments do you advocate?

A Some assessments are not plagued with these problems. Probably the best is one that encourages a great range of writing--the Kentucky portfolio assessment. While it has some problems, it is far superior to other assessments. Students don't have to learn to write without thinking in order to fill a page or two in 40 minutes. They have the year to produce several pieces of writing that go into a portfolio for judgment. The assessment is not conducive to formulaic writing.

If testing must be a matter of sitting down and writing in a set period, I recommend a test that provides some subject matter material for students to write about, as do the New York Regents exams. [They supply] literary passages or sets of data from social sciences.

Q What advice can you offer administrators on implementing writing assessments?

A Try to steer teachers away from formulaic writing ... and encourage breadth in the range of writing in your school curriculum. If you have the energy, examine your state's writing assessments. For ideas how to do that, see my article in English Journal (March 2003) called "Fighting Back: Assessing the Assessments."

George Hillocks Jr., author of The Testing Trap: How State Writing Assessments Control Learning (2002, Teachers College Press), is a professor of English and education at the University of Chicago.