Student response to grading incentives: evidence from college economics courses.

This paper aims to provide evidence on the response of various student types to the use of simple grading incentives. The results show that while the majority of students expend the effort to complete an assignment for credit, this response is not uniform. Using data from six semesters of college Economics classes, the study demonstrates that female students are more likely to submit assignments for credit, while nontraditional adult students and ones whose major field of study is not directly related to the course are less likely to respond to the opportunity of receiving course credit. This study highlights the degree of importance of extrinsic motivation for the above groups of students. For the student types that exhibit lower response to grades, alternative methods of motivation may be recommended.

Keywords: Grading, Incentives, Student Performance, Student Motivation

**********

Grades are used extensively at the university level to provide incentives for learning, however, there is surprisingly little evidence collected in economics of education on how various types of students respond to grading incentives. Meanwhile, the response of different groups of students to the incentives created by the grading system may not be uniform. While the extrinsic motivation provided by the grading system may stimulate greater effort by some students, others may be motivated by intrinsic factors. The goal of this study is to provide new evidence on the importance of grades in motivating students to complete coursework.

Oettinger (2002) analyzes how a grading standard that allows only a small number of distinct grades (that is "A" through "F") affects student behavior in an introductory college economics course. Evidence is presented that course performance of students is clustered slightly above the boundaries between grade levels, and that students that were slightly below a grade boundary going into the final exam exhibit more effort even controlling for pre-exam performance. Thus, students respond to the incentives created by the grading system.

Cullen et al. (1975) look at the decision of high-school students on whether to complete an assignment under the three conditions of positive grade for submission, negative grade (penalty) for non-submission, or no grade incentive. Results show that the use of both positive and negative grades increases student effort.

These are the few examples of empirical research that look specifically at student's response to the use of grades as incentives. Other aspects of grading have been the subject of several empirical studies. For instance, Figlio and Lucas (2004), Betts and Grogger (2003) and Jacob (2005) look at the link between grading standards and student performance. The evidence from these studies generally suggests that stricter grading standards lead to better student performance. On the other side of the student-educator relationship, Jacob also looks at the teachers' incentive to increase grading standard in response to accountability policies. Faculty incentives leading to grade inflation were also a topic of academic research (such as Germain and Scandura, 2005).

On the theoretical front, there has been significant debate on whether learning is driven by extrinsic or intrinsic incentives that is by grades or personal desire to learn (see for instance Covington, 2001). Wynne and Ayers (1994) present an overview of both propositions. The empirical studies reported above, in general, demonstrate that grades do influence student effort. However, the data used in these studies is not rich in demographical detail to provide evidence on the relative importance of extrinsic incentives for various groups in the population.

This study attempts to complement the existing literature on several accounts. Firstly, it will demonstrate how students respond to grading incentives for specific assignments. It is distinct from the study by Cullen et al. (1975) in that it looks at the university as opposed to the high-school level, and includes repeated assignments through the course of the semester, not a one-time exercise. The difference from the work of Oettinger (2002) lies in the focus of the study. Oettinger looked at the effort at the final exam of students just below a grade scale boundary. In the current study the focus is on the effort of students in obtaining grades throughout the semester, not just the performance on the final exam when they can estimate their final grade.

Secondly, the dataset to be used for this research is exceptionally rich in student demographic detail. Information is present on the student gender, major field of study, whether the student is traditional or adult, as well as the final grade achieved. Thus, the study can demonstrate the relative importance of grades as incentives for the different types of students.

Data and Methodology

The study is based on the analysis of a unique dataset that includes 189 students enrolled over six semesters in a required course--Introduction to Macroeconomics-taught by the same instructor. Grading in these classes was done using the same system.

There were several types of assignments--midterm and final exams, weekly quizzes, and homework. The grading system used was such that when a student received a low score on the weekly quiz--less than 60%--he or she had to prepare and submit homework to receive credit. Those with high quiz scores received homework credit automatically. Submission of the homework following the low quiz score was voluntary. Each homework was worth about 1 percent of total course grade and involved answering questions following the corresponding chapter in the textbook. There was no additional grading of the homework. If the student expended the effort to answer the questions, they received full credit.

Regarding the homework, there is a clear incentive structure. Students who read the book and write down the answers would receive credit equal to 1 percent of course grade for each homework submitted. There were a total of ten such opportunities during the semester, so the maximum possible credit is 10 percent of course grade, which would easily mean a difference of a full letter grade for the student. An average of 69 percent of students in the sample received at least one low quiz score, and were subject to the homework incentive. However, only 39 percent of all students actually chose to expend the effort and submit the homework.

The question to be answered in this study is whether a particular type of student is more likely to respond to the grading incentive and submit the homework. The dataset used is rich in demographic detail. In particular, it is possible to identify the gender of a student, whether the student is traditional or adult, whether the student's major is related to the course topic. Table 1 presents the descriptive statistics of the data.

Results

The first question to be answered by this study is whether the grading incentives matter for the students. Among those who scored low on the quiz and were subject to the homework incentive, 57% of the students actually chose to submit the homework to receive credit for it. This number is significant, however it is also important to understand whether the response is different among the various types of students present in the dataset. The decision to submit the homework or not is in essence binary, so the technique of Probit estimation is used to evaluate the impact of independent variables on the likelihood that a student expends the effort to prepare and submit a homework. Table 2 presents the results of the Probit estimation.

The dependent variable in the table is binary, and equal to one if the homework is submitted. The independent variables are also binary, and equal to one if the student is, respectively, female, classified as adult (nontraditional), has a major that is non-related to the course subject. A binary variable is included to control for the student's year of study--equal to one if the student is freshman or senior. Finally, binary dummy variables are included to control for the semester each student was enrolled in the course.

Coefficients reported in Table 2 represent the marginal effects on the dependent variable. A comparison to the mean homework submission rate makes the reported results easy to interpret. In particular, if a student is female, she is 51% more likely to submit the homework for credit (coefficient significant at 1% level). Extrinsic motivation by grades is, thus, more pronounced in female students.

Another strong result is obtained for the adult students in the dataset. A student classified as adult is 69% less likely to be motivated by the grade. Several interpretations are possible. On one hand, adult students are more likely to be working and to have less time for homework preparation. On the other hand, adult students may have a better idea of the benefit of education for them, and so be motivated intrinsically by the interest in the subject rather than extrinsically by the grade. As a result, getting the additional grade points may not be as important for them.

The data on submitted homework is available for ten points throughout the semester. This can be used to study whether the types of student likely to submit homework from the beginning of the semester are different from those who are motivated by the grade later in the semester. Oettinger (2002) reports that on the final exam students just below a grade level cut-off point exhibit greater effort. The results in Table 3 are designed to examine the characteristics of students that submit coursework for credit late in the semester--restricting the dependent variable to only homework assignments 6 through 10.

Column (1) replicates the estimation reported in Table 2, while Column (2) includes an additional explanatory variable that is equal to one if the student already chose to submit homework for credit early in the semester (that is for homework assignments 1-5). The results reported in Table 3 suggest that types of students that are motivated by grades late in the semester are not different from the ones motivated early. In fact, in column (2) the single biggest coefficient is reported for the early submission variable. If the student already chose to submit the homework at least once early in the semester, they are 57% more likely to do so again late in the semester. This coefficient is significant at the 1% level.

Female student are once again found to be more likely to be submitting homework--by 21%, while adult students are less likely--by 33%. Both coefficients are significant at the 5% level. The one area of difference as compared to the total semester results is the importance of the student's major area of study. Late in the semester, students whose area of study is not related to the course subject are 29% less likely to be motivated by the assignment grade. One explanation is that late in the semester the students choose to spend their limited time on the courses related to their major field of study, as success in such courses is more important for their career.

Conclusion

This study presents a systematic investigation into the response of different student types to grading incentives. The study was motivated by the fact that only 57% of the students enrolled in a required Macroeconomics course chose to expend the effort and voluntarily submit homework for credit. An examination of the dataset on homework submission provided evidence on the extrinsic motivation grades provide to students. It was found that this motivation is not uniform. In particular, the results suggest that female students are more likely to respond to the grading incentive, while adult students are less likely. In addition, later in the semester the students whose major field of study is not related to the course subject are less likely to be motivated by the grade in the course as well.

The evidence presented above demonstrates that the design of the grading system in a course does not affect students enrolled in the course in a uniform fashion. Some groups of students are likely to respond to the grading incentive and increase their effort in reading the textbook and preparing the assignments, while others are not. Therefore, grades should not be relied upon as the sole student motivation mechanism. Alternative motivation mechanisms may be recommended for the student bodies with groups that exhibit lower response to grades.

The results of this study provide an important perspective on the debate over extrinsic and intrinsic motivation. They demonstrate that the extrinsic motivational impact of grades is stronger for certain groups of the student body, particularly for students with a related major, and for female students. Adult students may be more motivated by the intrinsic factor such as the desire to learn the course material, and not the concern for the grade. Since grades do not motivate all students equally, the effort expended in receiving the grades will vary. A different question is whether the final grade actually reflects the student's performance accurately when some students are not motivated by the grade to fully perform on the course assignments. This question, however, cannot be answered using the current dataset and remains a topic for further examination.

The author acknowledges support for this project from the Indiana University Kokomo Summer Faculty Fellowship.

References

Betts, J. & Grogger, J. (2003). The impact of grading standards on student achievement, educational attainment, and entry-level earnings. Economics of Education Review, 22 (4), 343-353.

Covington, M. (2001) Intrinsic Versus Extrinsic Motivation: An Approach/Avoidance Reformulation. Educational Psychology Review, 13 (2), 157-177.

Cullen Jr., F., Cullen, J., Hayhow, V. & Plouffe, J. (1975). The Effects of the Use of Grades as an Incentive. Journal of Educational Research, 68 (7), 277-279.

Figlio, D. & Lucas, M. (2004). Do high grading standards affect student performance? Journal of Public Economics, 88 (9-10), 1815-1835.

Germain, M. & Scandura, T. (2005). Grade Inflation and Student Individual Differences as Systematic Bias in Faculty Evaluations. Journal of Instructional Psychology, 32 (1), 58-67

Goetz, C., Tucker, J. & Weber, W. (1968). Student Incentive Payments And Academic Achievement: An Empirical Test. Social Science Quarterly, 54 (1), 159-168.

Jacob, B. (2005). Accountability, incentives and behavior: the impact of high-stakes testing in the Chicago Public Schools. Journal of Public Economics, 89 (5-6), 761-797.

Oettinger, G. (2002). The Effect Of Nonlinear Incentives On Performance: Evidence From "Econ 101", Review of Economics & Statistics; 84 (3), 509-518.

Wynne, E. & Ayers, B. (1994). Resolved: Student Incentives. Curriculum Review, 33 (5) 3-8.

Dmitriy V. Chulkov, Ph.D., Assistant Professor, School of Business, Indiana University Kokomo.

Correspondence concerning this article should be addresseed to Dr. Dmitriy V. Chulkov, Assistant Professor, School of Business, Indiana University Kokomo, P.O. Box 9003, 2300 S. Washington St., Kokomo, Indiana 46904; Email: [e-mail]

Table 1
Descriptive statistics

                                         Mean     Mean only for student
                                       for all       that choose to
Variable                               students      submit homework

Receive low score and have to submit
homework for credit                      69%              100%

Choose to submit homework for credit     39%              100%

Choose to submit homework early in
the semester                             26%               66%

Choose to submit homework late in        27%               69%
the semester

Female                                   50%               61%

Adult and non-traditional                12%               5%

Major is not course-related              30%               27%

Freshmen                                 25%               31%

Sophomores                               46%               46%

Juniors                                  22%               18%

Seniors                                   7%               5%

Ultimately receive A                     29%               11%

Ultimately receive B                     39%               46%

Ultimately receive C                     23%               34%

Table 2
Effect of student type on response to grading incentives

                                  Dependant variable equals one
                                    if homework is submitted

Equals one if student is female             0.198 **
                                             (2.68)

Equals one if student is adult              -0.270 *
                                             (2.50)

Equals one if student's major                -0.048
is not course-related                        (0.58)

Equals one if student is                      0.064
fresheman or sophomore                       (0.77)

Semester dummy variables                    Included

Observations                                   189

* significant at 5%; ** significant at 1%

Coefficients for marginal effects are reported in the table. Absolute
values of robust z statistics are in parentheses.

Table 3
Effect of student type on response to grading incentives late in the
semester

                                      (1)                   2)

                               Dependant variable   Dependant variable
                                 equals one if        equals one if
                                  homework is          homework is
                                submitted late       submitted late
                                in the semester      in the semester

Equals one if student chose                              0.393 **
to submit homework early in                               (4.68)
the semester

Equals one if student is            0.177 **             0.143 *
female                               (2.71)               (2.17)

Equals one if student is x         -0.268 **             -0.231 *
female                               (2.77)               (2.46)

Equals one if student's             -0.140 *             -0.200 *
major is not course-related

* significant at 5%; ** significant at 1%

Coefficients for marginal effects are reported in the table. Absolute
values of robust z statistics are in parentheses.