Grading software programs accompanying selected Principles texts.
An increasing number of non-statistical software packages are being written as supplementary instructional material for economics Principles texts. This paper reviews the software programs currently available as ancillary material to eight major Principles texts. To avoid simply listing what the programs do, a detailed framework for economic software evaluation was created and applied to each program. This evaluation instrument gives the reviewer a total of 31 questions, in five different categories to guide the software review. A summary table is presented which allows direct comparison of each package across each of the five evaluation categories.
An increasing number of Principles textbooks include non-statistical computer-assisted instructional (CAI) software packages as part of the supplementary instructional material available to the student. The diskette (s) containing the program (s) and any written documentation are usually provided either free or for a small charge to students buying the textbook. The software combines one or both of the following two program types: Tutorials, in which the student is presented with a monolog, incorporating text and graphics, that reviews important concepts covered in the relevant textbook chapter, and Drills in which the student is presented with a series of questions designed to allow a self-test of their understanding of the material covered in the relavant chapter (1). This paper reviews the Tutorial and Drill software programs currently available as ancillary material to ten major Principles texts. The programs reviewed here range from packages containing Tutorials or Drills alone, to those offering some combination of the two. The
question that launched this review was, "Are the Tutorial and Drill programs supplied with Principles texts created following sound pedagogical principles?"
II. EVALUATION INSTRUMENT
An Instructional Software Evaluation Form was developed in an effort to answer that question (2). The form list 31 questions a reviewer should ask when working through a program, its documentation, and the student/teacher manual. The questions are divided into five different sets.(3)
The first set of questions is on General Issues regarding the general design of the program. The second set of questions deals with the Economics Content of the tutorials and drills. The third set of questions is the most critical, covering the Instructional Quality of the program. The fourth set of questions covers the Technical Quality of the program itself, as well as any documentation provided. For those packages that contain drill questions designed to assess the student's understanding, the fifth set of questions covers the likely Effectiveness of the Assessment Measures on learning.
The reviewer answers each question in a category by assigning a whole number ranging from -3 to +3 (including zero). When all the questions in a single category are answered, the numerical rankings are summed, and that sum compared to the reviewer's subjective "grading scale" for each category For example, if a category has seven questions my grading scale is set up in the following way: a Sum less than or equal 0 receives a Grade of F;a Sum between 1 and 6 receives a Grade of D; a Sum between 7 and 13 receives a Grade of C; a Sum between 14 and 20 receives a Grade of B; a Sum = 21 receives a Grade of A. By assigning points to the letter grade received in each category a GPA can be calculated for each program. For example, a Grade of F is assigned 0 points; D, 1 point; C, 2 points; B, 3 points;and A, 4 points.
III. RESULTS Preliminary Comments
Space limitations do not allow a question-by-question comparison among all ten of the programs reviewed. Table I shows a summary of the Grades assigned in each of the five categories. The scores assigned to each program are based on an absolute scale and reflect the views of the author--they are neither objective rankings, nor survey results. The remainder of the paper will be devoted to discussing Table 1 and to making additional comments on the packages.
Of the CAI packages reviewed, only four received positive(4) scores in Instructional Quality (column ): the package offered with McConnell's textbook ("Interactive Economics Tutorial for Graphic Analysis-IETGA"). the package offered with Thompson's textbook ("GraphEcon II"), the package offered with Baumol and Blinder's textbook ("Computer Assisted Program for Economics Review-CAPER"), and the package offered with Dolan's textbook ("EconoGraph"). Columns 6 and 7 of Table I shows the GPA of each program across all five categories. Column 6 shows the GPA received from all five categories, and Column 7 subtracts the points from the fifth category (Effectiveness of Assessment). The separation was made in an effort to better compare the programs, some of which contain tutorials only, some drills, only, and others a combination of the two. With the current version of the Instructional Software Evaluation form, failure to include Assessment questions penalizes Tutorials, but a similar penalty does not exist for Drill programs that do not contain Tutorial sections. Although the "ideal" package would contain both tutorials and drills, the questions are not categorized along those lines. Discussion of the Software
IETGA is designed as a menu-driven tutorial program. The student has access to 14 tutorial "Chapters"; 3 covering introductory material, 6 covering microeconomic material, and 5 covering macroeconomic material. Each Chapter contains from 2 to 8 Sections that review the graphs, concepts, and jargon discussed in the text, continually prompting the student with some good, thought-provoking questions. The tutorials cover the majority of the concepts and models commonly taught at the principles level. The graphics are among the clearest, and most quickly drawn of all the programs reviewed. The screen is divided into three "windows"; a graphics window; a scrollable text window (this is the only program that allows the student to scroll back line-by-line and review earlier portions of the lesson); and a summary window. The summary window reminds the student of the parameter values assumed in the lesson (i.e., Income = $100,Px = $2.00,etc.), and informs the student of the specific pages they may read in the textbook for further study of the graph, model, or concept. The program's GPA could be improved considerably by incorporating detailed learning objectives at the beginning of each Section, a review of those objectives at the end, and post-tutorial quiz questions. WALBERT: GRADING SOFTWARE
GRAPHECON II is a revised and improved software package that combines a menu-driven tutorial program with drill and practice routines. The package contains a four "Volume" set of tutorial, drills and applications: Introduction, Microeconomics, Macroeconomics, and International Trade. Each volume has 7 Tutorials, each with its own 5 question Quiz, and 2 Applications (which are, essentially, expanded lessons based on groups of Tutorials). Each Quiz may be repeated as often as the student desires, and the student's average Quiz score is saved in a file under the student's name. After the student enters an answer to a Quiz question, a detailed explanation is given whether the student answers the question correctly or not. Each volume ends with a Final Exam made up of 42 multiple choice questions, and may also be repeated as often as the student wishes. both the average Quiz and Final Exam scores may be read by the instructor and thus may be used to assign grades. Unfortunately, no diagnostic feedback is given to the student on the basis of her or his Quiz of Final Exam performance. The program combines text and graphics, animation and sound very effectively. One innovative feature new with version II is the "Escape Menu" which gives the student five ways to move through the Tutorial: Return to the Main Menu; Return to the Tutorial; Restart the Tutorial; Fast Forward through the Tutorial; or Quit the Tutorial. The Restart and Fast Forward and options allow the student to review material, or quickly return, at a later time, to exactly the point in the Tutorial where he or she ended. It is not as easy as simply "paging" back to earlier material, but it is a good step toward making tutorials more flexible. One of the programs main drawbacks is that it is not in fact tied to the text--it is a stand-alone package that, while offering comprehensive topic coverage, does not always present concepts, definitions, and graphs in exactly the same way as the textbook author.
CAPER is designed primarily as a menu-driven drill program, with three pedagogically well-constructed tutorials added. The questions in CAPER are well-developed, quite challenging, and cover all the chapters in the textbook. At the end of each drill, the student is given a diagnostic summary of the specific topics that gave them the most trouble, and the specific pages they should review in the text. The student receives an explanation for a wrong answer automatically, but may also obtain an explanation if their answer was correct. One instructional flaw in the tutorials (on graphing, elasticities, and the multiplier) is the failure to state learning objectives. A major omission from the drills--a flaw common to all drill packages--is that the level of difficulty of the drill questions is random. Some of the questions are more challenging than others, but the level of difficulty does not vary automatically with the ability of the student to answer the questions posed.5 ECONOMIC INQUIRY One of the most important roles a computerized drill program can play is that of the drill master patiently, and persistently forcing the student to push to a higher level of understanding by increasing the level of questions difficulty.
ECONOGRAPH is designed as a menu-driven tutorial program. The number of lessons offered is relatively small; 6 lessons focusing on microeconomic models and concepts and 3 lessons focusing on macroeconomic models and concepts. The lessons begin with a detailed listing of specific learning objectives, and key concepts. Each lesson is divided into several units, with each unit covering a different learning objective. The student is given a specific learning goal at the beginning of each unit, and is reminded of that goal at the end of the unit. The student can easily cycle through each unit in the lesson as often as needed. However, to review a unit one may not back up to a particularly troublesome part, but must start each unit from the beginning. Here too the units in each lesson are short, but the length of the unit would not be of concern if the parameter values changed the data used in the tables or graphs (this could be done either automatically or at the student's discretion). The tutorial periodically prompts the student with questions, allowing two attempts to complete the answer correctly. Incorrect answers are given a brief explanation immediately, but students are not referred to specific pages in the text for additional readings. The screen incorporates text and graphics rather well, but the more complicated the graphs the longer it takes to draw--at times this can be very slow. Very little animation is used in the lessons, and sound is not used at all.
MAID is designed as a menu-driven tutorial program. The package offers 8 Tutorials--4 on microeconomic topics, and 4 on macroeconomic topics. The program restricts coverage to those topics which make the best use of animated graphical analysis. Each tutorial begins with a list of learning objectives, but does not follow up on them at the end. The student is then given a chance to review how to read a graph. Although this is an important tutorial to include in the package, the "graph review" option makes up the first part of every tutorial, rather than taking a position as a separate tutorial altogether. As with most tutorials, the student is periodically prompted with questions as the lesson proceeds. In this case, the student is given three attempts to answer a question correctly before a "dialog box" pops up with the correct answer. The program makes good use of variable screen division between text and graphics, and the graphics are clear, most often easy to read, and accurate--if somewhat slow to draw.
MICROSTUDY is best described as a student workbook on diskette. In fact, with the exception of random changes in the order of the multiple choice or true-false drill questions presented, the program offers nothing different from the paperback student workbook also supplied with the text. The program consists of options to read a chapter-specific overview, learning objectives, or key terms and concepts; and to take chapter-specific true-false WALBERT: GRADING SOFTWARE and multiple choice drills. Although the program opens with a multi-colored supply demand graph, no graphs are otherwise used in the program--even with the True-False or Multiple Choice drill questions, a serious omission in an economic course. The drill options in the program keep track of the number of right and wrong answers, as well as the current score. No explanation is given for wrong answers, but the student is allowed 5 attempts before the next question is called up. One annoying bug surfaced repeatedly; at times, when I tried to access the chapter learning objectives from the main menu, an "unrecoverable error" occurred which required re-booting the program. At other times the "unrecoverable error" message was accompanied by the ubiquitous "Press Space Bar to Continue," which then brought up the Main Menu. MICROSTUDY does have one of the best menu structures, and one of the most complete HELP files of any CAI program supplied with a principles text.
TARGET is designed as a menu-driven drill program. The program opens with a menu of "modules" corresponding to the textbook chapters, making for complete coverage of topics commonly taught at this level. The "Instructions" menu is comprehensive, but difficult to read. The program offers 1000 variations on any given question, and allows the student to work through the questions as "examples"--where a score is not kept--or a "problem"-- where the score is kept (although not retained after the student signs off). The ability to "practice" a problem before taking a quiz is especially helpful because the student can receive complete feedback on their answer. Unfortunately, offering 1000 variations of exactly the same problem does not present students with a learning environment that challenges them to improve their understanding beyond an initial level.
INTERECON is designed primarily as a tutorial program, but the student's answers to the true-false, multiple choice, or fill-in prompt questions are scored and recorded for future a access by the course instructor. The program consists of 2 microeconomic, and 3 macroeconomic lessons. The student is given a list of topics he or she is expected to learn; the program then uses those topics as the tutorial's outline. When the student is presented with a question, she or he is allowed two attempts before the discussion continues or another, question appears, but the question must be correctly on the first try or the student loses a point. The program ties together well with the text: If the student answers a question incorrectly, a message appears which explains the error and indicates the specific pages in the text where the student may turn for additional reading. Maneuvering within the program is possible, but awkward. When the student is given a question, typing INFO re-displays the information given prior to the question; HELP gives a hint on how to answer the question; TELLME reveals the answer; and BACK returns to the previous question. The program is written using a coursebuilder routing which may be the source of several annoying problems: The program is very slow, especially between questions; an unforgivable number of graphs are poorly drawn, incomplete, or incorrect; usually only three options are given in the multiple choice questions; branching errors appear at times; if a question is reviewed, the program scores the original response as a second try (hence reducing the points earned by the student); graphs and tables are not always on the same screen as the questions, so the student must use the INFO option to review the graph or table, then go to the next screen and answer the question.
GRAPHICS TUTOR is a menu-driven tutorial/drill program. The student is presented with two disks, one with 6 topics in microeconomics, and the other with 6 topics in macroeconomics, a topic on Using Graphic is common to each. Each tutorial is imbedded with a number of graded prompt questions. At any time within the program, typing the backslash (1) key brings up an auxiliary menu that allows access to a HELP file, a list of KEY TERMS (only after they have been introduced in the tutorial). or the MAIN MENU; other options let the student REPEAT a topic, view his or her TEST GRADES, or change the colors of the screen output. When a color monitor is used, the screen output is by far the most colorful of the packages reviewed here. While the use of animation and the speed and clarity of the graphics are very good, the amount of screen space taken up the graphics "window' reduces the available space for text to three lines. The fill-in questions are not forgiving of typing errors, nor are synonyms accepted--the student must consult the list of key terms to determine which is correct (a strategy not mentioned in the Instructions, and not always helpful at that). Most of the requested information only promotes at the lowest level. At times part of the question is missing from the screen containing the prompt. An incorrect response elicits this non-encouraging message: "No Perhaps you should review the text." In some cases, the answer requested does not appear in either the table or graph. In other cases, even typing in the terms exactly as they appear on the graph will be labeled an incorrect response. There are also more than a few errors in both the text and graphs.
MACRO/MICROGRAPH is designed as a menu-driven tutorial: Only 8 "modules" are available, four each on microeconomic and macroeconomic topics. The module on long-run adjustment under perfect competition is particularly good. The program does not list the learning objectives at the beginning of each module. However, on the MicroGraph disk, a Summary unit at the end of the tutorial reminds the student of the key concepts in each module. Unfortunately each Summary unit lists no more than three learning objectives--far too few given the amount of material usually covered in each module. The program combines text and color graphics with animation. The low quality of the graphics, the slow speed at which the graphs draw, the small number of prompt questions, and the occasionally awkward placement of some text relative to the prompt questions, reduces the program's teaching effectiveness. If the students types in the wrong answer to a prompt question, the only response given is the correct answer. Paging through the program is awkward; the space bar is pressed until the page is filled with text, or the graph is completed, then the return key must be pressed to go to a new page. One nice feature is the ability to move backward or forward to the beginning of the various units with each module. IV. CONCLUSION
As the teacher's saying remind us, learning economic concepts takes place only when the student is actively involved in learning economics: "I do and I understand." It is in this role that microcomputers have the greatest potential to affect learning. But it's not the computer hardware the helps the student learn, it's the software--the lines of code that tell the computer how to respond to input from the user. Writing a good instructional software program requires keeping an eye on the instructional impact of the program. It is this careful preparation, more than the medium itself, that ultimately is responsible for helping the user to understand the lessons.6 The programs that accompany the ten Principles texts reviewed here are generally not created following sound pedagogical principles, thus, they are lacking in Instructional effectiveness. In fact, as shown in Table I, the top four programs would receive a grade of D for Instructional Quality alone based on the Instructional Software Evaluation Form used for this review. My impression is that few of the programs are written with an eye on their instruction impact, many of the programs seem to be hastily added as part of the package for marketing purposes, with too little thought given to the impact they may have on the student's ability to learn economic principles.
(*)Assistant Professor, Department of Economics, Illinois State University,I would like to thank Mat Morey and Ray Cohn for their many helpful suggestions.
(1.)The DRI/Data Diskettes which accompany Economics by Samuleson and Nordhaus, and Economics by McConnell, do not fit in either of these categories. These packages contain data and up to 3 problem sets that the student attempts to answer using LOTUS 1-2-3. The problem sets cover about half of each textbook's chapters. The package accompanying the Samuelson and Nordhaus text offers help to the novice 1-2-3 user, references to specific pages in the text, and graphics. Because these software packages and those reviewed in the remainder of the paper have no common basis for comparison, a detailed ranking and discussion is omitted.
(2.)This form follows the format of the National Science Teachers Association Microcomputer Evaluation Instrument, as presented by Klopfer . The evaluation questions were developed by the author from suggestions made by Klopfer , Futrell and Geisert , and by Heerman . A copy of the Instructional Software Evaluation Form is available from the author.
(3.)See Walbert  for a discussion of fourteen specific software design strategies that would help improve the pedagogic quality of most CAI programs.
(4.)Total points in the Instructional Quality category may range from -18 to +18. As noted in the text, in my evaluation procedure a negative score in this category is given a Grade of O(F). A strictly positive score is needed to received a Grade above F. The four best packages do follow some sound pedagogical principles in their design. However, their GPA's are not high enough to receive a Grade above a D.
(5.)I am of only one program, Market Master, a stand-alone drill program, that automatically adjusts the level of difficulty as a result of how well the student answers the questions.
(6.)This point is made quite well by Clark .
REFERENCES Software Bierman, Scott and Todd Proebsting. "Interactive Economics Tutorial for Graphic
Analysis-IETGA," accompanies Campbell McConnell, Economics (10th edition). Delta Software. "MicroStudy: A Computer Assisted Learning Program," accompanies Martin
Bronfenbrenner, Werner Sichel, and Wayland Gardner, Economics (2nd edition). Gunther, William and Thomas Gunther. "Graphics Tutor," accompanies Bradley R. Schiller, The
Economy Today, 3rd edition). Hoerneman, Calvin, David Howard, Karen Wilson, and D. John Cole. "Computer Assisted Program
for Economics Review-CAPER," accompanies William Baumol and Alan Blinder,
Economics: Principles and Policy (3rd edition). Lile, Stephen, "Interactive Economics-InterEcon," accompanies Lila Truett and Dale Truett,
Economics. Link, Charles, Jeffrey Miller, and John Bergman. "EconoGraph," accompanies Edwin Dolan,
Economics. Olvey, Donne and James Golden. "GraphEcon II, accompanies Allen Thompson, Economics
(2nd edition). Parker, Jeffrey. "Tutorial Accompanying Ruffin and Gregory's Economics Text-TARGET,"
accompanies Roy Ruffin and Paul Gregory, Principles of Economics (2nd edition). Strickler, Darryl, Steven Paul Peterson, and Decision Development Corporation.
"MicroGraph/MacroGraph," accompanies William A. McEachern, Economics. Strickler, Darryl and Trueblood & Associates. "Microeconomics: An Interactive
Demonstration-MAID," accompanies Ryan Amacher and Holley Ulbrich, Principles of
Economics (3rd edition). Readings Clark, Richard. "Reconsidering Research on Learning From Media." Review of Educational
Research, Winter 1983,445-59. Futrell, Mynga and Paul Geisert. The Well-Trained computer: Designing Systematic Instructional
Materials for The Classroom Microcomputer, Englewood Cliffs, N.J.: Educational
Technology Publications, 1984. Heerman, Barry Teaching and Learning With Computers, San Francisco: Jossey-Bass Publishers,
1988. Klopfer, Leopold, "Intelligent Tutoring Systems in Science Education: The Coming Generation
of Computer-Based Instructional Programs." Journal of Computers in Mathematics and
Science Teaching, Summer 1986,16-32. Walbert, Mark, "Avoiding Pedagogically Naive 'Captive' Software," Forthcoming, Journal of
Economic Education, November 1989.
|Printer friendly Cite/link Email Feedback|
|Title Annotation:||economics software evaluation|
|Author:||Walbert, Mark S.|
|Date:||Jan 1, 1989|
|Previous Article:||Do devaluations improve the trade balance? The evidence revisited.|
|Next Article:||Do prices lead money? A reexamination of the neutrality hypothesis.|