The impact of formative feedback on student learning in an online classroom.This exploratory research Exploratory research is a type of research conducted because a problem has not been clearly defined. Exploratory research helps determine the best research design, data collection method and selection of subjects. study examined the impact of using feedback from weekly multiple-choice tests on final exam Noun 1. final exam - an examination administered at the end of an academic termfinal examination, final exam, examination, test - a set of questions or exercises evaluating skill or knowledge; "when the test was stolen the professor had to make a new set of scores and students' IDEA course ratings. The teacher researcher taught two sections (N=33; N=34) of a graduate-level, semester-long, online course in advanced human growth and development. Each section had identical course materials on separate electronic Blackboard (1) See Blackboard Learning System. (2) The traditional classroom presentation board that is written on with chalk and erased with a felt pad. Although originally black, "white" boards and colored chalks are also used. sites. The treatment, 20-item multiple-choice tests, was randomly assigned as·sign tr.v. as·signed, as·sign·ing, as·signs 1. To set apart for a particular purpose; designate: assigned a day for the inspection. 2. to one section. The other section had no weekly multiple-choice tests. Identical final examinations and IDEA course evaluations A course evaluation is a paper or electronic questionnaire, which requires a written or selected response answer to a series of questions in order to evaluate the instruction of a given course. were used to measure differences in learning and student course satisfaction. Students in the course with the weekly formative formative /for·ma·tive/ (for´mah-tiv) concerned in the origination and development of an organism, part, or tissue. tests had statistically significantly (p.<.05) higher final exam scores. There was no statistically significant difference in the students' course ratings. ********** The online classroom environment provides challenging, unique, and exciting opportunities for assessing student learning. Benson (2003) suggested: Two key benefits of online assessments are (1) the ability of every learner to respond to every question the instructor asks and (2) the ability of the instructor to provide immediate feedback to each learner. In a traditional course, when the instructor asks a question, the first student to answer is typically afforded the sole opportunity to provide an answer (p. 70). However, students' response to questions, particularly assessment questions, and especially test questions, varies widely. Ricketts and Wilks (2002) suggested the online methods of assessment may adversely influence student learning. They concluded that all online instructors should conduct action research studies to explore the impact of their assessment methods on measures of student learning and course satisfaction. The Purpose of the Study The researcher/instructor had developed the online class in the spring of 2004 and had taught at least one section of the class every semester se·mes·ter n. One of two divisions of 15 to 18 weeks each of an academic year. [German, from Latin (cursus) s (including two summer sessions) prior to the study. Each semester, student feedback from course grades, IDEA online student evaluations, and student evaluations using a departmental survey was used to improve the delivery of the course. The research questions for this study were formed when the researcher had the opportunity to teach two online sections of a graduate-level course in advanced human growth and development spring semester 2006. The two questions were: 1. Does the use of weekly multiple-choice formative tests with immediate feedback impact student learning in the course? 2. Does the use of weekly multiple-choice formative tests with immediate feedback impact student evaluations of the course/instructor? Theoretical Framework The importance of prompt feedback in online classroom assessment has been the focus of several recent research studies (e.g., Cashion & Palmieri, 2002; Greenberg, 1998; Shuey, 2002; Siew, 2003). Students in face-to-face classrooms expect graded work to be returned within the week. Because the time parameters are different in online classes, feedback to students can range from instant--for example in a Blackboard-graded multiple-choice exam--to weekly--as in an instructor-graded essay exam. Feedback to students serves as both an extrinsic EVIDENCE, EXTRINSIC. External evidence, or that which is not contained in the body of an agreement, contract, and the like. 2. It is a general rule that extrinsic evidence cannot be admitted to contradict, explain, vary or change the terms of a contract or of a motivator--when grades are involved--and an intrinsic intrinsic /in·trin·sic/ (in-trin´sik) situated entirely within or pertaining exclusively to a part. in·trin·sic adj. 1. Of or relating to the essential nature of a thing. 2. motivator--when self-correcting is the primary motivating force. Research on assessment strategies in face-to-face classrooms has found that varying the type and/or frequency of formative assessment Formative assessment is a self-reflective process that intends to promote student attainment [1]. Cowie and Bell [2] define it as the bidirectional process between teacher and student to enhance, recognise and respond to the learning. results in measurable differences in student learning as measured by summative Adj. 1. summative - of or relating to a summation or produced by summation summational additive - characterized or produced by addition; "an additive process" , final course examinations (e.g., Brookhart, 2000; Stiggins, 1997). There is little research in online classrooms to test the theoretical classroom assessment research in face-to-face classrooms. Method Research Design The plan for this action research study was reviewed and approved by the IRB IRB See: Industrial Revenue Bond at the university. The study was conducted to inform online teaching of a graduate course in advanced human growth and development. The research was limited to two graduate classes in a single semester at a large regional state university. Participants The participants in the study were graduate students in the college of education. Advanced human growth and development is a required course for many majors (e.g., counseling, masters in teaching in secondary, masters in special education). The researcher/ instructor was assigned two sections spring semester 2006. The participants self-enrolled in one of two sections of the online courses. Enrollment was limited to 25 participants per class, but because of high demand for the course, "override An arrangement whereby commissions are made by sales managers based upon the sales made by their subordinate sales representatives. A term found in an agreement between a real estate agent and a property owner whereby the agent keeps the right to receive a commission for the sale of enrollments" permitted by the researcher/instructor resulted in 34 students in Class A and 33 students in Class B. There was no assumption that the enrollment of participants in the classes was random. Procedure Random assignment of treatment. The course requirements for the two online sections (delivered entirely through electronic Blackboard) were identical with a single exception. The 'manipulated variable,' weekly 20-item multiple-choice formative exams, was randomly assigned to Class A (N=33). Class B (N=34) served as the control. Description of the treatment. In Class A, the students were required to complete 20 multiple-choice questions on the weekly assigned readings. The multiple-choice questions were made available to the participants each Thursday at noon in two formats (l) as a printable print·a·ble adj. 1. Capable of being printed or of producing a print: printable negatives. 2. Fit for publication: printable language. 'review copy' and (2) as an answer sheet that immediately scored the student's submitted responses. Each student was able to discuss the questions through an "Exam Discussion Board" (Klecker, 2003). Students could submit their responses for scoring only once. The weekly deadline for submitting responses for scoring was Sunday at 11:55 p.m. The feedback provided through Blackboard was (1) the total number of items correct, and (2) the item number of items not answered correctly. Students did not receive the correct answer as feedback as they took the test. Answers to the 20 items were provided on Monday morning at 7:00 a.m. through a posting on Blackboard's "Announcement" page. The weekly formative exams were used weekly for 15 weeks. The dependent variable. Question 1: A sixty-item, multiple-choice, final exam was used as the summative measure of 'student learning' to answer research question one. The final exam was cumulative, covering the material from all 16 weeks, and was administered as a timed two-hour exam through electronic Blackboard. Students choose their time to take the final exam during a Thursday noon through Sunday night Sunday Night, later named Michelob Presents Night Music, was an NBC late-night television show which aired for two seasons between 1988 and 1990 as a showcase for jazz and eclectic musical artists. period--but they were able to enter the Exam only once and had to complete the 60-items within the two-hour time frame. There were no questions that had appeared on the formative exams on the final examination. The dependent variable. Question 2: The norm-referenced Individual Development and Educational Assessment (IDEA) (2006) student evaluation instruments, (administered online) were used to measure 'student satisfaction' to answer research question two. The researcher selected identical course objectives on the preparation sheets for the IDEA ratings. Data Analysis After all student grades were submitted and the semester ended, the data were extracted from Blackboard in an Excel format and uploaded into a Statistical Analysis Software (SAS (1) (SAS Institute Inc., Cary, NC, www.sas.com) A software company that specializes in data warehousing and decision support software based on the SAS System. Founded in 1976, SAS is one of the world's largest privately held software companies. See SAS System. ) program for analysis. Results The means and standard deviations In statistics, the average amount a number varies from the average number in a series of numbers. (statistics) standard deviation - (SD) A measure of the range of values in a set of numbers. are presented in Table 1. The mean of Class A, the class that received the weekly formative-exam feedback (M=54.45, SD=4.26), was higher than the mean of Class B, the control group (M=51.56, SD=6.48). The larger standard deviation for Class B indicated that there was greater variation in the final exam scores for the control group, and that the t-test for unequal variance should be selected to test the observed differences for statistical significance {Table 2}. The results of the independent t-test (t(57)=2.166, p=.017) found that that the observed differences were statistically significant at the .05 level set a priori a priori In epistemology, knowledge that is independent of all particular experiences, as opposed to a posteriori (or empirical) knowledge, which derives from experience. . [Note, the adjusted degrees of freedom because of the unequal variance.] Table 3 presents the adjusted T scores for five summary categories created from student ratings on the Individual Development and Educational Assessment (IDEA) Ratings by class. The IDEA was administered online. Results were not received by the researcher/ instructor until after the final grades had been submitted to the Registrar's Office. The IDEA is used each semester. Non-tenured faculty are required to evaluate two classes per semester using this norm-referenced instrument. The T scores for the two classes appear to be remarkably similar. [T scores have a mean of 50 and a standard deviation of 10.] The non-parametric statistical test, Chi square chi square (kī), n a nonparametric statistic used with discrete data in the form of frequency count (nominal data) or percentages or proportions that can be reduced to frequencies. bivariate bi·var·i·ate adj. Mathematics Having two variables: bivariate binomial distribution. Adj. 1. tabular tab·u·lar adj. 1. Having a plane surface; flat. 2. Organized as a table or list. 3. Calculated by means of a table. tabular resembling a table. analysis (also known as crossbreaks) was chosen as appropriate for looking for Looking for In the context of general equities, this describing a buy interest in which a dealer is asked to offer stock, often involving a capital commitment. Antithesis of in touch with. statistically significant differences in the tabular data. This analysis seemed somewhat redundant to the conclusions drawn by simply looking at Table 3. Although the non-parametric test lacks the power of a parametric See parametric modeling, parametric symbol and PTC. test, with the two data columns as well-matched as these, it's not surprising that there was no statistically significant difference in the student evaluations by class ([X.sup.2](4) = 0.227, p.>,05) (Table 4). Conclusions The use of weekly formative 20-item, multiple-choice exams in the online teaching of Advanced Human Growth and Development led to higher scores on the final exam, the measure of 'student learning' for this action research study. Although students were not randomly selected for the classes, the treatment was randomly assigned to the class. It is of interest that although one class had the added assignment of the formative exams, the course evaluations were identical. The weekly 20-item, multiple-choice exams have been added to the online course. It should be noted that in addition to the final exam, the course grades included a review of research paper [graded and returned until students reached mastery] and a 10-page term paper that students could submit for instructor feedback throughout the semester. There was no statistically significant difference in students' grades in the two classes. References Benson, A. D. (2003). Assessing participant learning in online environments. New Directions for Adult and Continuing Education continuing education: see adult education. continuing education or adult education Any form of learning provided for adults. In the U.S. the University of Wisconsin was the first academic institution to offer such programs (1904). , 100, 69-77. Brookhart, S. M. (2000). The art and science of classroom assessment: The missing part of pedagogy. ERIC Digest Digest: see Corpus Juris Civilis. (1) A compilation of all the traffic on a news group or mailing list. Digests can be daily or weekly. (2) Any compilation or summary. . ERIC Clearinghouse on Higher Education higher education Study beyond the level of secondary education. Institutions of higher education include not only colleges and universities but also professional schools in such fields as law, theology, medicine, business, music, and art. , One Dupont Circle Dupont Circle is a traffic circle in the northwest quadrant of Washington, D.C., at the intersection of Massachusetts Avenue, Connecticut Avenue, New Hampshire Avenue, P Street and 19th Street. , Washington, DC (ERIC Document Reproduction Service No. ED432938) Cashion, J., & Palmieri, P. (2002). The secret is the teacher: The learners' view of online learning. Leabrook, Australia: National Center for Vocational Education vocational education, training designed to advance individuals' general proficiency, especially in relation to their present or future occupations. The term does not normally include training for the professions. Research (ERIC Document Reproduction Service No. ED475001) Greenberg, R. (1998). Online testing. Techniques: Making Education & Career Connections. 73, (3), 109-131. Individual Development and Educational Assessment (IDEA) (2006). IDEA online. Retrieved September 16, 2006 from http://www.idea.ksu. edu/pilot/index.html Klecker, B. (2003). Formative classroom assessment using cooperative groups: Vygotsky and random assignment. Journal of Instructional Psychology, 30, (1), 216-219. Klecker, B. (2005). Assessing learning online: The top ten list. Paper presented at the annual international meeting of The Society for Information Technology & Teacher Education (SITE), Phoenix, AZ. Ricketts, C., & Wilks, S.J. (2002). Improving student performance through computer-based assessment: Insights from recent research. Assessment & Evaluation in Higher Education, 27, (5), 475-479. Shuey, S. (2002). Assessing online learning in higher education. Journal of Instruction Delivery Systems, 16, (2), 13-18. Siew, P.F. (2003). Flexible online assessment and feedback for teaching linear algebra linear algebra Branch of algebra concerned with methods of solving systems of linear equations; more generally, the mathematics of linear transformations and vector spaces. . International Journal of Mathematical Education in Science and Technology, 34, (1), 43-51. Retrieved August 28, 2006. Stiggins, R. J. (1997). Student-centered classroom assessment (2nd. ed.) Columbus, OH: Merrill, an imprint im·print tr.v. im·print·ed, im·print·ing, im·prints 1. To produce (a mark or pattern) on a surface by pressure. 2. To produce a mark on (a surface) by pressure. 3. of Prentice Hall Prentice Hall is a leading educational publisher. It is an imprint of Pearson Education, Inc., based in Upper Saddle River, New Jersey, USA. Prentice Hall publishes print and digital content for the 6-12 and higher education market. History In 1913, law professor Dr. . Beverly M. Klecker, Morehead State University History Morehead State University was originally founded as a private teacher's college in 1887, The Morehead Normal School. It is said to have been comprised of 13 buildings with a layout in the shape of a crescent moon for some period prior to 1922. . Correspondence concerning this article should be addressed to Beverly M. Klecker at b.klecker@morehead-st.edu
Table 1
Means and Standard Deviations for Final Exam Scores by Class
Class N M SD
Class A 33 54.45 4.26
Class B 34 51.56 6.48
Table 2
Independent t-test of Means of Final Exam Scores
Class N M SD
Class A 33 54.45 4.26
Class B 34 51.56 6.48
t-test with unequal variance
DF=57
t=2.166
Table 3
Means for Individual Development and Educational Assessment (IDEA)
Ratings by Class
Class A (N=33) Class B (N=34)
Item M * M *
Progress on Relevant Objectives** 56 60
Overall Excellence of Teacher 56 56
Overall Excellence of Course 62 59
Average of B & C 59 58
Average of A & D 58 59
* Note: Adjusted T Score
** Same objectives were selected for each course
Table 4
Chi-Square Results of IDEA Student Ratings by Class
Rating Class A Class B Totals
1 56 60 116
2 56 56 112
3 62 59 121
4 59 58 117
5 58 59 117
Totals 91 92 583
Degrees of freedom: 4
Chi-square = 0.227
For significance at the .05 level, chi-square should be greater than or
equal to 9.49.
The distribution is not significant.
p is less than or equal to 1.
|
|
||||||||||||||
Printer friendly
Cite/link
Email
Feedback
Reader Opinion