Student prepared testing aids: a low-tech method of encouraging student engagement.
The use of student-prepared testing aids, commonly referred to as cheat sheets or crib sheets, is a commonplace assessment procedure in education (Butler & Crouch, 2011; Larwin & Larwin, 2009). However, opposing hypotheses exist about the reasons for their impact on student performance and learning. Dorsel & Cundiff (1979), Dickson & Bauer (2008) and Funk and Dickson (2011) have concluded that student-prepared testing aids benefit students in that they serve as a source of information that students need for an exam, and students then rely on or depend on the aid, rather than learning the information and committing it to memory. According to this dependency hypothesis explanation, the act of creating a testing aid does not enhance student learning and memory; it amounts instead to a clerical exercise in creating a reference tool that is depended upon to provide the information the student needs during an exam.
However, there were a number of potential extraneous variables operating within the Dickson and Bauer (2008) study that qualifies this evidentiary challenge. In this repeated-measures study, Dickson and Bauer found that there was indeed significantly better performance when students prepared and used student-prepared testing aids on a multiple-choice psychology exam, relative to when those same students prepared but did not use testing aids on a pre-test subset composed of identical questions. As all the students had prepared testing aids before the pre-test-then-exam sequence, the fact that the students performed better on the exam when testing aid use was allowed caused the authors to conclude that constructing aids did not actually enhance student learning; they argued instead, that the use of testing aids simply enhanced student performance. Dickson and Bauer reasoned that had the act of constructing testing aids actually enhanced learning, all students should have performed similarly on the pre-test and the exam.
However, in the Dickson and Bauer (2008) study, students were aware that the pre-test would not count toward their grade and thus their performance quality on the pre-test was of little significance to them personally. This may have undermined some of their motivation to perform well. Secondly, students were under the impression that they would be able to use their student-prepared testing aids on their examinations. Thus, even though the pre-test didn't "count," having an unexpected pretest to take without the potential benefit of their student-prepared testing aids may have generated some test anxiety that could have worked to attenuate pre-test performance. Finally, as this study was a repeated-measures study where the pre-test items and exam items on which performance was assessed were identical, it is possible the improved scores on the exam, the second assessment, were influenced by a practice effect. Indeed, Dickson and Bauer presented evidence that the students performed significantly better on the specific exam questions that were identical to those in the pre-test relative to the exam questions that were not a part of the pre-test.
A similar pattern of results was obtained by Dorsel and Cundiff (1979) who found that psychology students who prepared and used student-prepared testing aids outperformed students who prepared and did not use them. However, in this study this effect was only observed when students were unexpectedly unable to use the student-prepared testing aids they had prepared. While Dorsel and Cundiff interpreted this as a result of a dependency effect--whereby students might have been relying too heavily on the testing aid they prepared and thought they were going to be able to use--it is also true that the performance deficit observed in their study might be an anxiety effect, produced by the anxiety associated with learning at the last moment that they wouldn't be able to use the testing aid they had prepared. Thus, Dorsel and Cundiff's (1979) pattern of results may reflect either a cognitive dependency, whereby students were relying too heavily on the student-prepared testing aids for recall and intellectual performance, or an emotional dependency, whereby students were relying on their student-prepared testing aids to help allay fears associated with the testing situation, or both.
The engagement hypothesis explanation of the beneficial effects of student-prepared testing aids is the notion that the preparation encourages students to review, organize, and clarify the main points of the course material that will most likely appear on a test (Cherim, 1981; Hindman, 1980; Skidmore & Aagaard, 2004; Wachsman, 2002). Loftman (1975) and Zimmerman & Pons (1986) investigated the impact of having students review course materials prior to exams. Both researchers asked students to rewrite, revise, and reorganize the course notes in an effort to increase overall engagement with course content, increase familiarity with materials (and thus potential exam questions), and increase study time. They found that students who participated in the reviews of the exam materials significantly outperformed students who did not. Since active student engagement with course materials and information--such as note taking, participation in cooperative learning groups, and activities-based learning--is associated with greater performance and learning creating and using testing aids might act as another opportunity to encourage student engagement with the course information (Larwin & Larwin, 2009; Skidmore & Aagaard, 2004).
However, another pattern of results obtained in Dorsel and Cundiff's (1979) study is more clearly a challenge to the engagement hypothesis of student-prepared testing aids construction and use. When the authors compared the group of students who had prepared and used student-prepared testing aids with another who neither prepared nor used them, there were no differences in test performance. According to the engagement hypothesis, the student-prepared testing aids group should have outperformed the control group that did not have the opportunity to make and use testing aids. Whatever the reason for this pattern of results, a consideration of these findings along with those of Dickson and Bauer (2008) discussed above, reveals that the precise impact of student-prepared testing aids on student learning and performance is varied--across experimental conditions and student populations--and still far from clear.
The current investigation seeks to shed some light on which one of these explanations, the dependency hypothesis or the engagement hypothesis, is a more viable explanation of the effects of student prepared testing aids. Specifically, when given the opportunity to prepare a testing aid, students prepared one of four: a handwritten testing aid, a computer-generated aid that was constructed from cutting and pasting information from electronic notes posted by the professor or supplementary textbook materials, a combination of handwritten notes and copy-and-paste notes, or no testing aid at all. It is theorized that students who handwrite their testing aids are potentially engaging in the information more so than a student who simply cuts-and-pastes information that is electronically available. As previous research has demonstrated, reviewing and re-writing of notes has been found to lead to better retention and performance on exams (Carder, 1983). Additionally, research has suggested that if student prepared testing aids are advantageous, that this advantage exists more so for lower order skill questions rather than higher order skill questions, such as conceptual or application type questions (Whitley, 1996). For this investigation, it is hypothesized that if no difference between the exam performances of three experimental groups is found, the results provide support for a dependency hypothesis explanation. However, if students in the handwritten group perform better than the other experimental groups, these results would provide support for the engagement hypothesis, assuming there is a beneficial effect.
Additionally, the current investigation examines students' performance across both lower- and higher-skill type questions. Particularly, if the students' performance on Part One of the exam (lower-skill, selected response questions) exceeds their performance on Part Two of the exam (higher-skill, constructed response to application-type questions), then this would provide support for the dependency hypothesis (Dickson & Bauer, 2008; Funk & Dickson, 2011). Contrarily, if students perform equally on the two sections of the exam, or perform significantly better of the Part Two of the exam, these results would suggest support for the engagement hypothesis, which maintains that students use the preparation of the testing aid to process, organize, code, and develop a deeper level of understanding of the information and concepts being studied than is suggested might occur within the dependency hypothesis (Larwin & Larwin, 2009).
Participants included n = 44 second and third year undergraduate students that ranged from 18 to 32 years of age (M= 21.82, SD = 3.08), including n = 17 male (38.6%) and n = 27 female (61.4%) students, who were enrolled in one section of applied statistics at a small Midwestern University. Students in these applied statistics courses were expected to develop the statistical tools used in research decision making, including but not limited to determination and interpretation of measures of central tendency, variance, probability, regression and correlation analysis, hypothesis testing, frequency and probability distributions, and sampling methods. Students also were introduced to graphical, tabular, and mathematical depictions of statistical information. This section incorporated activity-based cooperative learning groups and supported learning with the use of statistical computer packages (MiniTab and SPSS).
The final class exam was used to measure student performance. This exam was administered two weeks prior to the end of the semester; student groups were additionally required to submit an applied research project during final exam week as part of their final grades. All students completed the exams within the allotted period. This exam included two parts: Part One which included selected-response items (multiple choice questions), and Part Two, which included application problems appropriate for the material covered by the exam. The application portion of these exams accounted for thirty points, the selected response portion of the exam accounted for twenty points. Although the exams were not intended to be cumulative, the nature of the course material required that students understand foundational course material presented earlier in the semester in order to understand material presented later in the semester. The assessment used for this measure required students to be prepared with information that covered t-tests, f-tests, and correlation/regression analyses.
Students in the present study were instructed that for the last exam of the semester, they would be permitted to use one-side of an 11 by 8.5 piece of paper to create a sheet of notes which they could prepare for use on the final assessment. Students were also instructed that they would be turning in their student prepared test-notes with their exam. Students were not given any other instructions as to how to prepare these note-sheets except that all information had to be limited to one side of the paper. Since class notes were posted electronically for this class, it was possible that a student could use those posted notes as part of their note-sheets. These posted notes were developed to compliment the textbook and lecture.
A 2 (gender) by 4 (notes) factorial multivariate Analysis of Variance was conducted on the student exam scores for part one (selected response) and part two (application). For the "notes" factor, students' note-sheets were coded as: (1) handwritten notes, (2) copy-and paste notes, (3) mixed: meaning that students had a mix of copy and pasted information and handwritten information, and (4) no to minimal notes. For the "no to minimal notes" condition, the "minimal notes" condition included note sheets that had less than twenty words written on the note sheet.
A factorial MANOVA was conducted for the two exam scores (part one and part two) across the fixed factors of gender and note-sheet condition. Initial analyses indicate that all test assumptions were tenable for these analyses. The distribution of grades by note-taking condition is presented in Table One.
As indicated above, Part One of the Exam was based on a possible 20 points, whereas Part Two of the exam was based on a possible 30 points. Multivariate analyses reveal no differences across gender, however differences were revealed across the notes preparation condition, F(6, 70) = 3.362, p = 0.006, partial [[eta].sup.2] = 0.224 (based on Hotelling's Trace). The tests of between-subjects effects indicates that the differences in the note-condition existed for both Part One, F(3, 37) = 3.48,p = 0.025,partial [[eta].sup.2]= 0.220, and Part Two, F(3,37) = 4.95,p = 0.005,partial [[eta].sup.2] = 0.286 of the exam. Scheffe Post hoc analyses of the notes condition indicates that differences exist between handwritten and copy/paste notes conditions, [M.sub.d] = 2.69, SD = .733, p = .009, CI [0.544, 4.842], for Part One of the exam, and between handwritten and copy/paste notes conditions, [M.sub.d] = 3.31, SD = .796, p = .002, CI [0.987, 5.649], for Part Two. A within subjects analysis reveals that a significant difference exists between students performance on Part One (M = 14.02, SD = 2.29) relative to Part Two (M= 27.47, SD = 2.52), with students performing significantly better on Part Two, F (1,40) = 56.60,p < 0.001.
Discussion, Limitations, and Conclusions
The goal of the current investigation aimed to examine whether the preparation and use of student-prepared testing aids (a.k.a. cheat sheets, or crib cards) provide students simply with a "crutch" (Dickson & Bauer, 2008; Dorsal & Cundif, 1979; Funk & Dickson, 2011), or whether their preparation encourages students to further engage with the information being tested (Cherim, 1981; Hindman, 1980; Larwin & Larwin, 2009; Skidmore & Aagaard, 2004; Wachsman, 2002; Zimmerman & Pons, 1986). The analyses presented here suggest the latter.
Specifically, the results of the current study demonstrated that students who produced their testing aids by handwriting out the information performed significantly better than students who simply copied and pasted the information onto a piece of paper that they electronically printed for use during the examination. Interestingly, students who hand-wrote some of their notes in combination with copied and pasted information (mixed method) also performed better than students who simply copied and pasted all of the information for their testing aid, although these results were not statistically significant. This pattern of results is consistent across the selected response section (Part One) and the application section (Part Two) of the exam, however the greater impact was found for the application portion of the exam.
These results provide support for the engagement hypothesis in that all of the students prepared their own testing aid and included what information they deemed most necessary. Students self-determined how they went about constructing the testing aid, and all of the students were allowed to use the testing aid that they produced, as they expected. There were no surprises surrounding the testing conditions that might have generated anxiety and served as a confound clouding the interpretation of the dependent measure. The experimental condition for these findings represents a very realistic testing situation, thus provides a high level of external validity for these results. Unlike Funk and Dickson's (2011) position that having a testing aid actually reduced students' study time, the students who produced handwritten testing aids were likely spending more time selecting, organizing, and deciding what information might be most beneficially included on the limited space of a 8.5 by 11 inch sheet of paper, relative to the students using the cut and paste approach. At the same time, it is also possible that creating a testing aid through cutting and pasting of information does actually reduce the time that students spend engaging and studying the test information, relative to those students who take the time and write out these testing aids.
Additionally, the results of the current investigation suggest that the positive impact of preparing the testing aid resulted in significantly better results overall for the application portion (Part Two) of the exam (91.6% correct) relative to the selected response portion of the exam (70.1% correct).
The handwritten groups average performance on Part One (74.4%) and Part Two (95.9%) was higher than the average performance for the mixed method group on Part One (72.1%) and Part Two (90.4%), which was higher than the average performance of the copy/paste group on Part One (60.9%) and Part Two (83.9%). These results were not completely surprising in that students knew that they would be required to decide the appropriate analysis and hand-calculate the results from a small sample of data; many students across all groups included an example or two of some basic analysis (i.e. Standard deviation, t- test, f-test, etc) on their prepared testing aid. However, having the calculations available on a testing aid does not help students who do not know what analyses need to be conducted to answer different types of research questions. Potentially, writing out the formulas served as not only an opportunity to review, but also to practice for those students who choose this approach.
A potential limitation of this research is that it was based on one class of students for one testing, and students were not randomly assigned to their respective note taking conditions; they self-selected. This limitation is also strength in that these research findings have high external validity. Additionally, the current investigation lacks a true control group. Only two students choose not to prepare a testing aid (with more than 20 words) which did not provide a comparison group of appropriate sample size or power for comparison.
The findings of the current investigation suggest that the use of student prepared testing aids can be beneficial. However, it seems that even in today's world of high technology learning tools, the use of student prepared testing aids should be limited to handwritten materials. The process of writing out the information can reinforce students' memory of the information, and encourage more processing of the information as they write it out. Potentially, this process encourages students to engage and process the information more slowly, thus providing them with the time to see what they might not understand. As indicated by the number of students who decided to simply copy and paste information into their testing aid, there are always going to be those students that do not want to invest appropriate time for test preparation. Perhaps for these students the use of self-generated testing aids will not benefit them beyond a simple dependency effect.
Butler, D., & Crouch, N. (2011). Student experience of making and using cheat sheets in mathematical exams. Mathematics: Traditions and New Practices, 134-141.
Carrier, C.A. (1993). Note taking research: Implications for the classroom. Journal of Instructional Development, 6(3), 19-26.
Cherim, S. (1981). A philosophy of teaching preparatory chemistry. Paper presented at the Annual Two-Year College Chemistry Conference, Atlanta, Ga.
Dickson, K. L., & Bauer, J. J. (2008). Do students learn course material during crib sheet construction? Teaching of Psychology, 35(II), 117-120.
Dorsal, T., & Cundiff, G. (1979). The cheat-sheet: Efficient coding device or indispensable crutch? Journal of Experimental Education, 48, 39-42.
Funk, S.C., & Dickson, K.L. (2011). Crib card use during tests: Helpful or Crutch? Teaching of Psychology, 38(2). 114-117.
Hindman, C. D. (1980). Crib notes in the classroom: Cheaters never win. Teaching of Psychology, 7, 166-168.
Larwin, K.H., & Larwin, D.A. (2009, Nov). The Use of In-test Mnemonic Aids (a.k.a. Cheat-sheets) in Higher Education To Improve Student Learning and Performance. Presentation at the American Evaluation Association Annual Conference, Orlando, Florida.
Loftman, G. R. (1975). Study habits and their effectiveness in legal education. Journal of Legal Education, 27, 418-472.
Skidmore, R. L., & Aagaard, L. (2004). The relationship between testing condition and student test scores. Journal of Instructional Psychology, 31, 304-313.
Wachsman,Y. (2002). Should cheat sheets be used as study aids in economics tests? Economics Bulletin, 1, 1-11.
Whitley, B.E. (1996). Does 'cheating" help? The effect of using authorized crib notes during examinations. College Student Journal, 30, 489-493.
Zimmerman, B. J., & Pons, M. (1986). Development of a structured interview for assessing student use of self-regulated learning strategies. American Educational Research Journal, 23, 614-628.
Karen Larwin, Ph.D., Assistant Professor, Educational Foundations, Research, Technology, and Leadership, Beeghly College of Education, Youngstown State University, Youngstown, OH.
Correspondence concerning this article should be addressed to Dr. Karen Larwin at email@example.com.
Table 1 Mean and Standard Deviation of Part One and Part Two Exam Grades Part One Part Two n Mean Sd Mean Sd Handwritten 24 14.87 2.34 28.50 2.19 Copy/Paste 11 12.18 1.25 25.18 2.18 Mix Method 7 14.42 1.99 27.42 2.07 Minimal Notes 2 14.02 2.29 28.01 2.82
|Printer friendly Cite/link Email Feedback|
|Publication:||Journal of Instructional Psychology|
|Date:||Jun 1, 2012|
|Previous Article:||The qualities of physical education teachers based upon students' perceptions of physical appearance.|
|Next Article:||Collaborative relationships for general education teachers working with students with disabilities.|