The impact of Microeconomics: The Puzzle Game on student evaluations.
The use of supplements in academic classrooms is widespread. This paper is different in that it tests whether or not course and instructor evaluations improve as a result of the use of a new supplement, Microeconomics: The Puzzle Game (Lipscomb and Ostapski 2008), which utilizes crossword puzzles as tools to reinforce basic economic concepts. More importantly, this paper suggests that regression at the mean does not provide the full picture with respect to the impact of the use of supplemental materials on course and instructor evaluations. In fact, regression at the mean suggests that course and instructor evaluations significantly decrease when the crossword puzzle supplement is used. However, using the power of quantile regression, we stumble upon an interesting finding- that course and instructor evaluations increase as a result of the crossword puzzle supplement in the upper quantiles of the course and instructor evaluation distribution.
The data used for this analysis come from the records of the author's ECON 2106 classes (Principles of Microeconomics), spanning seven semesters (Fall 2005 to Fall 2008) and using the same primary textbook. Microeconomics: The Puzzle Game was used for the first time in the Spring 2008 semester. So, of the 602 observations in the dataset, 185 (or 30.7%) used the supplement in their classes.
To estimate the impact of the supplement on course and instructor evaluations, at least three different ways to examine the data come to mind. First, to establish a baseline, we estimate two OLS regression models where the dependent variables are course evaluations (COURSE) and instructor evaluations (INSTRUCT). Independent variables used to explain variation in evaluations are a student's average grade on four class examinations (EXAMAVG) and whether or not Microeconomics: The Puzzle Game was used (PUZGAME). The results of the regression models (with Z- statistics in parentheses) are:
COURSE = 4.09 - .077PUZGAME + .0002EXAMAVG [R.sup.2] = .03 (77.46) (-4.68) (0.30) INSTRUCT = 4.34 - .086PUZGAME - .0007EXAMAVG [R.sup.2] = .10 (131.45) (-7.97) (-1.73)
These baseline results suggest that the crossword puzzle supplement has a negative impact on course and instructor evaluations.
A second possibility is that EXAMAVG is endogenous. A path analysis (available upon request), where EXAMAVG is an intervening variable between PUZGAME (likely the only truly exogenous variable in the system) and the evaluation variables, suggests essentially the same total effect of PUZGAME on course evaluations (-.077) and instructor evaluations (-.083). So far, these two different empirical approaches suggest that the supplement has a negative impact on evaluations.
A third approach to estimate the impact of PUZGAME on evaluations is to see if results vary depending on the particular quantile of the dependent variables under examination. The quantile regression method (Koenker and Bassett 1978), implemented using the "sqreg" command available in Stata, estimates the same model specification at different points (the .25, .5, and .75 quantiles) along the evaluation distribution. We find that the estimated coefficients on PUZGAME at the different quantiles are -.065, -.092, and .027 (for course evaluations) and -.130, -.007, and .048 (for instructor evaluations). The main outcome is that the supplement has a positive and significant impact on course (p=.000) and instructor (p=.016) evaluations in the upper quantile under examination. This suggests that the use of regression at the mean causes one to overlook some details that may be helpful to instructors using supplements to improve student test performance. Particularly, one instructor might forego the use of a supplement like The Puzzle Game because of the regression at the mean results presented earlier. Another instructor with a "maximin" approach (to maximize the grades of the worst performing students in a class) may inadvertently hurt students who innately perform well in courses like microeconomics. Yet another instructor who wants to ensure higher grades from those at the upper end of the evaluation distribution would be interested in the quantile regression results presented here.
In addition, estimation of the same model specifications for different groups of EXAMAVG (results available upon request) suggests that the negative coefficient on PUZGAME attenuates as one moves to the upper quantiles of the evaluation distributions.
Koenker, R., & Bassett, G. (1978). Regression quantiles. Econometrica, 46, 33- 50.
Lipscomb, C. A., & Ostapski, S. A. (2008). Microeconomics: The puzzle game. Valdosta: Teachers Own Publishing.
C. A. Lipscomb ([mail])
Department of Marketing and Economics, Langdale College of Business
Valdosta State University, 1500 N. Patterson Street, Valdosta, GA 31698-0075,
Published online: 5 February 2010
|Printer friendly Cite/link Email Feedback|
|Author:||Lipscomb, Clifford A.|
|Publication:||Atlantic Economic Journal|
|Date:||Jun 1, 2010|
|Previous Article:||Aid and public finance: a missing link?|
|Next Article:||The limitations of prospect theory and the expected utility theory: a new theory.|