The relationship between testing condition and student test scores.

Active student engagement with course materials, such as note-taking and cooperative learning, is associated with greater performance. This study investigated the relationship between scores achieved by students in an undergraduate course and active engagement during testing, whether using a cheat sheet or engaging in a form of cooperative testing. Subjects were 141 undergraduate students enrolled in a course required for admission to the teacher education program at a regional state university. Five multiple-choice tests were administered during the semester. Four testing conditions were sequentially implemented: (a) independent, (b) 'cheat sheet', (c) heterogeneous achievement group discussion, and (d) homogenous achievement group discussion along with a 'cheat sheet.' Wilcoxon signed-ranks test showed significant gains for all alternative testing conditions over the independent condition. Heterogeneous achievement group discussion had the largest effect size, overall, but responses to the other two testing conditions differed depending on students' prior achievement levels. Results suggest implications for the effective use of individual and cooperative testing procedures.

**********

Those challenged with facilitating student learning at all instructional levels recognize that student engagement is key to academic competence. Effective study skills are foundationally important to competence in both academic and non-academic settings (Gettinger & Seibert, 2002). These include competencies associated with acquiring, recording, organizing, synthesizing, remembering and using information (Hoover & Patton, 1995). Pressley & Afflerbach (1995) identified several key strategies that facilitate learning. These included (a) overview before reading, (b) attention to important information, (c) relating / connecting important points, (d) activation and use of prior knowledge, (e) changing strategies when appropriate, and (f) monitoring understanding and taking action to enhance comprehension. Ideally students who have progressed through formal education systems to the college level have acquired such tactics, strategies and self-regulative skills that direct and enhance their ability to learn. However, students may enter post-secondary level institutions without knowledge of and previous practice with these skills and strategies (Schumacker & Sayler, 1995).

Additionally, there seems to be a tradition of bias in the American educational system toward individual accountability and responsibility (Meinster & Rose, 1993). This would seem to inhibit adoption of and engagement in cooperative learning endeavors, even though the benefits have been repeatedly demonstrated. However, studies repeatedly show that cooperative and collaborative learning environments enhance both classroom climate and student performance (e.g., Aronson, Stephan, Blaney, Sikes, & Snapp, 1978; Webb, 1997).

The basis for much of the cooperative learning movement originates from the work of Lev Vygotsky. Vygotsky (1978) believed that social interactions are the foundation of cognitive development. Through the utilization of cultural sign systems, of which language is the most pervasive and efficient, students help each other learn conceptual material. Often they learn just as well or better during these peer interactions as they would if teachers work with students. Vigorous cooperative engagement improves learning (Duffy & Bednar, 1991; Kember & Murphy, 1990;Vygotsky, 1978). It follows that debate, negotiation, and discussion regarding conceptual material would be effective ways to increase student learning, improve reasoning and facilitate the development of improved learning strategies and new knowledge that they might not gain independently (Damon & Phelps, 1989; Gabbert, Johnson, & Johnson, 1987; Johnson, Johnson, & Smith, 1990). Research has shown that cooperative learning techniques are positively associated with (a) performance on academic tasks (Johnson, Johnson, & Smith, 1990), (b) student self-esteem and student attitudes toward school (Slavin, 1991), and (c) reduced test anxiety (Meinster & Rose, 1993). In addition, Webb (1997) found that students who actively participated in group problem-solving activities, either by asking for help or in leading the group, performed better on subsequent individual tests. However, students who did not participate performed poorly on subsequent individual tests. Webb concluded that if learning is the goal of the teacher and the classroom, then the learning processes should include "exchanging and discussing ideas, opinions, and knowledge; helping others; working together rather then separately; actively seeking help and clarification; encouraging others to participate; justifying and explaining one's own ideas; and elaborating on others' ideas" (1997, pp. 209-210).

It follows that, if tests are to be a learning experience as well as an assessment, and the teacher's goal is student learning and mastery of conceptual material, allowing cooperation during test-taking would be a 'natural' extension of Vygotsky's theory (Klecker, 2002). Research on cooperative test-taking reports improved student attitudes about themselves, about the subject or course, and a reduction in anxiety (Becker & Cardulla, 1995; Giraud & Enders, 1998; Klecker, 2002; Zimbardo, Butler, & Wolfe, 2003), as well as somewhat higher test scores than independent testing (Lambiotte et al., 1987; Zimbardo, et al., 2003). These results were found in studies conducted in chemistry, statistics, psychology, and education courses.

Reviewing of content materials, as well as course notes, has been recognized as an important study strategy (Loftman, 1975; Zimmerman & Pons, 1986). Instructors often encourage students to revise and rewrite individual course notes to increase total study time and familiarity with the examination materials. Loftman (1975) found that time spent reviewing was a better predictor of grades than total study time, assignment preparation, and time spent in study groups. Zimmerman and Pons (1986) found that student reviews of content materials and other self-regulated learning strategies discriminated among achievement tracks of high school students.

In the interest of improving scores on tests, students will sometimes spend time in the preparation of cheat sheets or crib cards, which may be considered as products derived from the process of advanced preparation. Of course, the intuitive intent, for some students, is to use such devices secretly to facilitate their test performance. Some instructors have implemented the 'legal' use of cheat sheets in testing situations (Dorsel & Cundiff, 1979; Hindman, 1980). Some studies show that testing situations that allow the use of "cheat sheets" or "crib cards" (i.e., notes the students have prepared ahead of time to use during an examination or test) have the effect of improving student attitudes and lowering anxiety (Cherim, 1981). More importantly, preparing cheat sheets or crib cards seems to help students to review, organize and clarify the main points of material that will most likely appear on a test. (Cherim, 1981; Hindman, 1980; Tips for beginners, 1992). Hindman (1980) found that students who did better academically were more likely to prepare and use "legal" crib cards, which was also associated with a slight increase in test scores. However, Dorsel and Cundiff (1979) found that cheat sheet preparation did not improve student performance if the student was depending upon it being available in a test situation. When students made out a cheat sheet but knew they were not permitted to use it during the test (i.e., non-dependent), they performed as well as when a cheat sheet was not made at all, or made and not used. It should be noted that construction of the cheat sheet followed the administration of a pretest and followed a prescribed text as a model (i.e., summary paragraph regarding Freud), and was not prepared before class time. This type of advanced preparation is in contrast to "open book" exams, where students who come into the test or exam situation unprepared do not benefit from the availability of notes and textbook use during the exam (Boniface, 1985).

Certainly encouraging and supporting energetic engagement with course material, either before or during a test, would seem appropriate. It follows from the research on advanced preparation and cooperative testing that combining both a cheat sheet and group discussion option in testing situations might help lower-achieving students achieve better test scores. Lower-achieving students tend not to make cheat sheets or crib cards (Hindman, 1980), and their analysis of multiple-choice question alternatives was found to be poorer than that of "A" students (McClain, 1983). However, whether or not any member of a group prepared a cheat sheet, it seems probable that a discussion format would benefit lower-achieving students. They might be more likely to read, discuss, and analyze all the alternatives presented for individual questions with the members of their group. This would result in better conceptual understanding and higher test scores. Preparation of a cheat sheet by even one member of the group might provide clarification of test items and response alternatives for all members of the group, thereby facilitating understanding of all members or the group.

Therefore, the purpose of this study was to investigate the relationship between testing condition (i.e., independent, cheat sheet, homogenous grouping with cheat sheet, and heterogeneous group without cheat sheet) and student test scores.

Method

One hundred forty-one undergraduate students enrolled in a course required for entry into the teacher education program at a regional state university in the Mid-South were participants in this study. All four sections of the course were taught by the same instructor and had identical multiple-choice tests and test conditions. Five tests were given over the course of the semester and none of them was comprehensive.

The first two tests were administered in the traditional fashion, with students working alone to complete the assessments. For Test 3, students were told in advance that they could prepare an 8 1/2 by 11 inch sheet of paper with whatever information they wanted handwritten on both sides. This "cheat sheet" could be brought to class and used during the test.

For Test 4, students were told that they would be allowed to discuss the test in groups of three or four prior to marking their answers. The groups were announced at the beginning of the class period during which the test was taken, but students were not told the basis on which groups were formed. Groups were assigned randomly (using a random number table), but stratified on the average performance on the first two tests. Thus each group contained (at a minimum) a student who previously had performed at the A or B level, one who had performed at the C level, and one who had performed at the D or F level. Students wishing to opt out of the group discussion were allowed to do so. The instructor rearranged group membership as necessary to adjust for those students opting out (n=12) or absent (n=11). Students choosing the discussion option met with their groups in the hallway outside the classroom, but they were not allowed to make any notes or mark on their exams in any way during the discussion. Members of each group returned to the classroom to mark their exams individually when their group was finished discussing the items.

For Test 5, students were told they could bring in a cheat sheet (as on Test 3) and they also would be allowed to discuss the test with a different group prior to marking their answers. As before, students were not told the basis of group assignment. For Test 5, the instructor assigned students into homogeneous grade groups, based on their average score on the first two individual exams. This meant that three to five students who had earned an A average individually were grouped together for discussion, as were students in the B range, and so on. Students could refer to their cheat sheets during discussion, but could not write on them or on the test papers during group time. Again, pre-assigned groups had to be rearranged somewhat because of students opting out (23) or being absent (7). (Table 1 displays the numbers of students from each grade group who opted out of the group discussion alternatives. The composition of the homogeneous grade groups is displayed in Table 2.)

Analysis

Descriptive and statistical data analyses were conducted. Wilcoxon signed-ranks tests (Siegel, 1956) were used to determine whether one of the alternative testing conditions resulted in students earning higher scores compared to the traditional format. The nonparametric Wilcoxon was chosen rather than a one-sample t-test because the study employed a convenience sample not randomly chosen from a specific population. Descriptive statistics and frequency distributions were generated for the whole sample and for the homogeneous grade groups. Individual data regarding the use or nonuse of cheat sheets were not collected, so, for the sake of consistency, statistics were calculated using the entire sample, even though some students did not make and use cheat sheets, or did not engage in discussion with classmates.

Results and Discussion

Descriptive statistics for all testing conditions are displayed in Table 3 and Figure 1. Heterogeneous group discussion resulted in the highest average test score (90%). The differences between test scores under alternative conditions and the highest score earned in the traditional condition were calculated and are displayed in Table 4, along with the Wilcoxon results. The difference between the two cheat sheet conditions (one combined with homogenous group discussion) is also included in Table 4. The probability of each Wilcoxon test was less than the Bonferroni-corrected alpha of 0.0127, so each difference was statistically significant. All alternative conditions resulted in higher scores than students earned in the traditional independent condition, despite including non-participant student scores in the analyses.

Inspection of the means clearly indicates that discussion in heterogeneous groups had the largest effect, which could have been the result of higher achieving students in the group giving answers to the lower achievers. This could not have been the case in the homogeneous grade groups discussion option, however, where the data indicate that discussion with peers at the same achievement level added value to having a cheat sheet alone.

The data in Table 5 and Figure 2 illustrate the different ways that the grade groups responded to the alternative testing conditions. The highest scores across all grade groups, except for the 'A' students, occurred in the heterogeneous group discussion condition. The 'A' students' highest score (by a small margin) was in the homogeneous discussion / cheat sheet combination condition, which was the second highest score for all other groups except the 'B' students. The 'B' students scored 5 points higher on the cheat sheet format than they did when it was combined with homogeneous group discussion.

Students in the 'C' and 'D' grade groups benefited substantially from a cheat sheet alone, although the 'F' students were not helped as much by this condition. Adding homogeneous group discussion to the cheat sheet condition resulted in 5-7 point gains for the 'C' and 'D' students, but nearly triple that gain (17 points) for the 'F' students. This means that when these lowest-achieving students, two-thirds of whom had not scored higher than 59% independently, talked in groups of three to five peers about the test, they raised their average test score by nearly two letter grades.

There was no control in the study design over the relative difficulty of the five tests. The differing responses of the grade groups to the alternatives, however, alleviated some of this concern. If the reason that 'F' students performed so well on the last test was simply because it was easier than some of the other exams, then the 'B' students also would be expected to increase their scores on it. This was not the case. (Perhaps the 'B' students argued each other out of the correct answer during homogeneous discussion.)

The lowest-achieving students might have been knowledgeable individually in different areas of the content being tested and the group discussion simply helped them fill in the gaps for each other. Alternatively, perhaps more of them decided to make cheat sheets for the final test, and the presence of those notes during the group discussion and individual marking of exams accounted for the difference in achievement. It also might be that for these students, thinking together is an effective scaffold for the task of dealing with the mechanics of testing, such as fully considering all response options before answering the test item, as well as scaffolding their conceptual understanding of the material.

Further study of this area is needed. The cause for the increased grades in the homogeneous group discussion condition requires more investigation, as noted above. A second related question is whether testing condition is related to length of retention of the content material.

Table 1
Number of Students in Particular Grade Groups Opting Out
of Discussion Formats

                         Grade Group

Test Format               A      B      C      D      F

Heterogeneous Groups      5      4      1      1      1
Homogenous Groups         6     10      5      2      0

Table 2
Highest Individual Scores of Students in Homogeneous Grade Groups

                Highest Grade Obtained Individually
               (Number of Students)

Grade Group      A      B      C      D      F     Total

'A' Group       16      2      0      0      0        18
'B' Group        4     13      1      0      0        18
'C' Group        1     20     17      1      0        39
'D' Group        0      0      5     14      0        19
'F' Group        0      0      0      6     11        17

Note: Homogeneous groups were formed based on an average of the
first two individually-taken tests. Only 111 students participated
in the homogenous group option.

Table 3
Test Score (%) Descriptive Statistics for All Alternative Formats

                                         Standard
Test Format              N      Mean     Deviation

Individual 1            141     73.7       15.6
Individual 2            141     71.6       13.8
Cheat Sheet             141     80.7       12.7
Heterogenous Groups     140     90.1        8.3
Homogenous Groups
  and Cheat Sheet       139     84.0        9.0

Note: Means include scores of all students, even those who did not
choose to use a cheat sheet or work in groups.

Table 4
Wilcoxan Signed Ranks Tests for Differences between Testing Formats

                               Mean of     Standard
Format Comparison        N    Difference   Deviation    S       Pr > S

Highest Individual
Test Compared to:
  Cheat Sheet Alone     141       2.7        11.0      1179     0.0069
  Heterogeneous Group   140      12.0        13.0      3571   < 0.0001
  Homogenous Group
    and Cheat Sheet     139       5.8        12.0      2209   < 0.0001
Cheat Sheet Alone
Compared to:
  Homogenous Group
    and Cheat Sheet     139       3.1        12.3      1188   < 0.0110

Note: Means include difference scores for all students, even those
who did not choose to use a cheat sheet or work in groups.

Table 5
All Test Score Descriptive Statistics for Each Homogeneous Grade Group

Test Format and     Mean     Standard      Minimum     Maximum
Grade Group                  Deviation

First Individual Test
  'A' Group         93.4        5.6          80          100
  'B' Group         79.7        9.6          58           94
  'C' Group         75.0        9.4          50           89
  'D' Group         61.5        9.0          45           79
  'F' Group         49.6        8.8          30           65

Second Individual Test
  'A' Group         86.9        5.8          77           97
  'B' Group         81.9        6.7          72           93
  'C' Group         70.6        9.2          49           94
  'D' Group         63.0        7.0          46           79
  'F' Group         51.9        8.3          42           69

Cheat Sheet
  'A' Group         91.6        7.2          74          100
  'B' Group         85.8        9.0          65           97
  'C' Group         80.8        7.2          67           96
  'D' Group         74.5        7.8          64           91
  'F' Group         60.5       11.6          44           82

Heterogenous Groups
  'A' Group         94.2        5.1          87          100
  'B' Group         90.7        6.1          77          100
  'C' Group         91.7        7.9          69          100
  'D' Group         89.6        6.4          71           96
  'F' Group         84.2       12.9          58           97

Homogenous Groups and Cheat Sheet
  'A' Group         95.0        4.9          86          100
  'B' Group         80.7        7.3          69           94
  'C' Group         86.4        7.4          68          100
  'D' Group         81.1        5.3          70           91
  'F' Group         76.8        8.8          60           89

Note: Means include only scores of the 111 students participating
in the homogeneous group discussion alternative.

References

Aronson, E., Stephan, C., Blaney, N., Sikes, J., & Snapp, M. (1978). The jigsaw classroom. Beverly Hills, CA: Sage.

Becker, R., & Cardulla, F. (1995). Final analysis--rethinking an age-old practice. Journal of Chemical Education, 72(9), 816-819.

Boniface, D. (1985). Candidates' use of notes and textbooks during an open-book examination. Educational Research, 27(3), 201-209.

Cherim, S. (1981, March). A philosophy of teaching preparatory chemistry. Paper presented at the Annual Two-Year College Chemistry Conference, Atlanta, GA.

Damon, W., & Phelps, E. (1989). Critical distinctions among three methods of peer education. International Journal of Educational Research, 13, 9-19.

Dorsal, T.N., & Cundiff, G.W. (1979). The cheat-sheet: Efficient coding device or indispensable crutch? Journal of Experimental Education, 48(1), 39-42.

Duffy, T.M., & Bednar, A.K. (1991). Attempting to come to grips with alternative perspectives. Educational Technology, 31, 12-15.

Gabbert, B., Johnson, D.W., & Johnson, R. T. (1987). Cooperative learning, group-to-individual transfer, process gain, and the acquisition of cognitive reasoning strategies. The Journal of Psychology, 120, 265-278.

Gettinger, M., & Seibert, J.K. (2002). Contributions of study skills to academic competence. School Psychology Review, 31, 350-365.

Giraud, G., & Enders, C. (1998, April). The effects of repeated cooperative testing in an introductory statistics course. Paper presented at the Annual Meeting of the Educational Research Association, San Diego, CA.

Hindman, C.D. (1980). Crib notes in the classroom: Cheaters never win. Teaching of Psychology, 7(3), 166-168.

Hoover, J.J., & Patton, P.R. (1995). Teaching students with learning problems to use study skills: A teacher's guide. Austin, TX: Pro-Ed.

Johnson, R.T., Johnson, D.W., & Smith, K. A. (1990). Cooperative learning: An active learning strategy for the college classroom. Baylor Education, 15, 11-16.

Kember, D., & Murphy, D. (1990). Alternative new directions for instructional design. Educational Technology, 30, 42-47.

Klecker, B.M. (2002, October). Formative classroom assessment using cooperative groups: Vygotsky and random assignment. Paper presented at the Annual Meeting of the Midwest Association of Teachers of Educational Psychology, Oxford, OH.

Lambiotte. J.G. Dansereau, D.F., Rocklin, T.R., Fletcher, B., Hythecker, V.I., Larson, C.O., & O'Donnell, A. M. (1987). Cooperative learning and test taking: Transfer of skills. Contemporary Educational Psychology, 12(1), 52-61.

Loftman, G.R. (1975). Study habits and their effectiveness in legal education. Journal of Legal Education, 27, 418-472.

McClain, L. (1983). Behavior during examinations: A comparison of "A," C," and "F" students. Teaching of Psychology, 10(2), 69-71.

Meinster, M.O., & Rose, K.C. (1993, March). Cooperative testing in introductory-level psychology courses. Teaching of psychology: Ideas and innovations. Proceedings of the annual conference on Undergraduate Teaching of Psychology, Ellenville, NY.

Pressley, M., & Afflerbach, P. (1995). Verbal reports of reading: The nature of constructively responsive reading. Hillsdale, NJ: Erlbaum.

Schumacker, R.E., & Sayler, M. (1995). Identifying at-risk gifted students in an early college entrance program. Roeper Review, 18(2), 126-129.

Siegel, S. (1956). Nonparametric statistics for the behavioral sciences. New York: McGraw-Hill.

Slavin, R.E. (1991). Synthesis of research on cooperative learning. Educational Leadership, 48(2), 71-82.

Tips for beginners. (1992). Mathematics Teacher, 85(2), 110-111.

Webb, N.M. (1997). Assessing students in collaborative groups. Theory into Practice. 36, 205-213.

Vygotsky, L. (1978). Mind and society. The development of higher mental processes. Cambridge, MA: Harvard University Press.

Zimbardo, P.G., Butler, L.D., & Wolfe, V.A. (2003). Cooperative college examinations: More gain, less pain when students share information and grades. The Journal of Experimental Education, 71(2), 101-125.

Zimmerman, B.J., & Pons, M. M. (1986). Development of a structured interview for assessing student use of self-regulated learning strategies. American Educational Research Journal, 23(4), 614-628.

Ronald L. Skidmore, Ph.D. and Lola Aargaard, Ph.D., Assistant Professor Department of Counseling. Leadership, Adult/High and Secondary Education, Morehead University.

Correspondence concerning this article should be addressed to Dr. Ronald L. Skidmore, Assistant Professor, 503 Ginger Hall, Morehead, KY 40351-1689: Email: [e-mail]