Essay versus multiple-choice: student preferences and the underlying rationale with implications for test construction.
Many universities are experiencing budget cuts that result in larger classes. The need to teach these larger classes without suffering an excessive reduction in the amount of time available for research and service-related endeavors exerts pressure on faculty to make increasing use of multiple-choice questions on quizzes and exams. Although multiple-choice questions provide the obvious benefit of being easier to grade, their use is in conflict with the intuitive feelings of many faculty that multiple-choice questions are inferior INFERIOR. One who in relation to another has less power and is below him; one who is bound to obey another. He who makes the law is the superior; he who is bound to obey it, the inferior. 1 Bouv. Inst. n. 8. to essays and other open-ended forms of assessment both as a measure and as a promoter A person who devises a plan for a business venture; one who takes the preliminary steps necessary for the formation of a corporation.
Promoters are the people, who, for themselves or on behalf of others, organize a corporation. of student learning.
Previous studies have demonstrated that the use of assessments that test higher-order thinking can encourage students to pursue study strategies that develop a deeper level of understanding rather than utilizing strategies that achieve little more than test-related recall. Previous work has also demonstrated that the use of these deeper learning strategies tends to lead to greater learning, at least for those learning tasks requiring higher-order thinking. This paper will review those results, relating them to the choice between multiple-choice and essay, and also discuss various other strengths and weaknesses of the essay and multiple-choice format. The paper's major purpose is to extend the knowledge of student assessment preferences by investigating the rationale behind those preferences. It will report the results of a survey questionnaire in which undergraduate business students in a junior level Principles of Management course and a capstone Strategic Management course specified not only their preferences for multiple-choice or essay but also the reasons for those preferences. The patterns that result will demonstrate two main factors driving student opinions, one factor focused on the ease of obtaining a high test score (with preferences for multiple-choice) and the other focused on the fairness and validity of the assessment measure (with preferences for essay). These results, in conjunction with various ideas drawn from the literature, will be used to generate examrelated pedagogical ped·a·gog·ic also ped·a·gog·i·cal
1. Of, relating to, or characteristic of pedagogy.
2. Characterized by pedantic formality: a haughty, pedagogic manner. recommendations.
ESSAY VERSUS MULTIPLE-CHOICE AS A MEAURE OF LEARNING
Because the primary goal of an exam is to accurately assess student learning, one of the key issues to consider is whether or not open-ended and multiple-choice questions differ in terms of reliability and validity. The literature tends to favor multiple-choice. For example, Bridgeman (1992) suggested that although multiple-choice is less reliable on a question-by-question basis due to guessing, the fact that multiple-choice questions take less time to answer (and grade) would allow an exam made up entirely of multiple-choice to contain more questions and therefore be more reliable than an exam containing fewer open-ended questions. Hassmen and Hunt (1994) agreed but also suggested that guessing shouldn't should·n't
Contraction of should not.
shouldn't should not
shouldn't should be discouraged dis·cour·age
tr.v. dis·cour·aged, dis·cour·ag·ing, dis·cour·ag·es
1. To deprive of confidence, hope, or spirit.
2. To hamper by discouraging; deter.
3. because guesses are generally based at least partially on student knowledge of the relevant course content. A weakness of essay questions cited by many authors is that they require subjective grading, with even factors that are unrelated to answer content shown to have an impact on exam scores. For example, Powers, Fowles, Farnum and Ramsey Ramsey, residential borough (1990 pop. 13,228), Bergen co., NE N.J.; settled 1846, inc. 1908. Dairy and truck farms are in the area. (1994) found that handwritten hand·write
tr.v. hand·wrote , hand·writ·ten , hand·writ·ing, hand·writes
To write by hand.
[Back-formation from handwritten.]
Adj. 1. answers tended to be graded more leniently le·ni·ent
Inclined not to be harsh or strict; merciful, generous, or indulgent: lenient parents; lenient rules. than identical answers written using a word processor. The authors suggested that graders may be more forgiving of handwritten answers because poor penmanship can disguise Disguise
Dishonesty (See DECEIT.)
enters nunnery as convert to retrieve money. [Br. Lit.: The Jew of Malta]
disguised as a woman to avoid conscription. [Gk. spelling and grammar errors, because the scratch-outs that appear in handwritten answers demonstrate the students' efforts at improvement and because word processed answers tend to fill less space on the page and therefore may appear to be less complete.
A related measurement issue concerns the question of whether or not essays and other forms of open-ended questions truly assess different dimensions of learning than are assessed by multiple-choice. Many studies have found that student scores on open-ended questions were so closely related to their scores on multiple-choice as to suggest that both types of questions were measuring the same things (Bridgeman, 1992; Lukhele, Thissen & Wainer, 1994; Walstad & Becker, 1994), suggesting that the difficult-to-administer open-ended questions might not be worth the extra effort because multiple-choice alone could be used to assess the same learning. Lukhele et al. noted further that essays became particularly unhelpful as a measure of learning if students were allowed to select the questions to answer from a longer list of questions provided, allowing the students to avoid topics they hadn't had·n't
Contraction of had not.
hadn't had not
hadn't have learned. In contrast, Thissen, Wainer and Wang (Wang Laboratories, Inc., Lowell, MA) A computer services and network integration company. Wang was one of the major early contributors to the computing industry from its founder's invention that made core memory possible, to leadership in desktop calculators and word processors. (1994) and Harris and Kerby (1997) found that essay scores and multiple-choice scores, although significantly correlated cor·re·late
v. cor·re·lat·ed, cor·re·lat·ing, cor·re·lates
1. To put or bring into causal, complementary, parallel, or reciprocal relation.
2. , were not so closely related as to be identical. Even more convincingly, Becker and Johnston (1999) applied two-stage least squares, a more sophisticated statistical procedure than was used in most previous studies of this topic, and found that essay and multiple-choice scores were not significant predictors of each other, suggesting that they were clearly measuring different dimensions of knowledge.
Various authors have provided potential explanations for the high correlations often seen between multiple-choice and essay scores, implying that this should not be interpreted as a suggestion that both types of questions measure the same learning. Driessen and van der Leuten (2000) suggested that student reasoning is so difficult to grade that the grading process for essay answers may unintentionally deteriorate de·te·ri·o·rate
1. To grow worse in function or condition.
2. To weaken or disintegrate. into an exercise in counting the number of facts provided in the answer, thereby turning even an essay question into a recall-oriented question. Because many students recognize that quantity is often rewarded and therefore attempt to provide it in their answers, timed exams may not give students the opportunity to provide both the quantity-oriented list of facts and the more thoughtful organization, integration, and analysis of those facts that would demonstrate higher order thinking (Minbashian, Huon & Bird, 2004). Students who are fearful of failure are particularly prone to this behavior of attempting to provide quantity (Diseth & Martinsen, 2003).
Despite the mixed evidence for the existence of a meaningful difference between what is measured by multiple-choice questions and what is measured by open-ended questions, many authors suggest that open-ended questions are more effective at assessing higher order thinking in contrast to the recall-oriented focus often seen in multiple-choice (Bridgeman, 1992; Scouller, 1998; Scouller & Prosser, 1994; Walstad & Becker, 1994). For example, essay answers can demonstrate student thought processes This is a list of thinking styles, methods of thinking (thinking skills), and types of thought. See also the List of thinking-related topic lists, the List of philosophies and the . and creativity in a way that cannot be achieved using multiple-choice (Walstad & Becker, 1994). Students may hold these same perceptions of multiple-choice questions as being targeted at lower level recall-oriented learning (Scouller, 1998), even for multiple-choice questions that have been specifically designed to test higher level learning (Scouller & Prosser, 1994). Faculty should pay full attention to student perceptions of assessment methods because those perceptions may influence student study strategies, as will be discussed further below.
ESSAY VERSUS MULTIPLE-CHOICE AS A PROMOTER OF LEARNING
Student perceptions of what learning really is and the study strategies used to achieve that learning are conventionally classified into two categories, surface and deep. Surface-oriented strategies focus on course content as a set of disconnected ideas that are to be accepted passively from the instructor or text. The primary learning goal is to memorize mem·o·rize
tr.v. mem·o·rized, mem·o·riz·ing, mem·o·riz·es
1. To commit to memory; learn by heart.
2. Computer Science To store in memory: facts and subsequently reproduce re·pro·duce
1. To produce a counterpart, an image, or a copy of something.
2. To bring something to mind again.
3. To generate offspring by sexual or asexual means. them on an exam (Entwistle & Entwistle, 1992; Minbashian et al., 2004). To a certain extent, surface strategies could be viewed as simply the lack of a strategy. As shown by Scouller and Prosser (1994), surface-oriented students tend not to be self- reflective Refers to light hitting an opaque surface such as a printed page or mirror and bouncing back. See reflective media and reflective LCD. about their studying processes and may not really be able to appreciate the difference between the recall of facts Recall of facts is a value of education which is to be de-emphasized in reform of science according to new national curriculum standards. Recall facts is often associated with rote learning. and a true understanding of the material. The surface orientation can apply to the instructor as well as to the students. A surface-oriented instructor, operating under the assumption that students bring little expertise of their own to the class, would tend to emphasize lecture with the view that his primary task is to impart information.
In contrast, deep learning strategies focus on achieving true understanding rather than simply preparing for the test. This would involve more self-directed learning with less focus on the teacher being in charge, increased collaboration Working together on a project. See collaborative software. between students, integration of ideas, critical evaluation of the logic and evidence that support each new concept, inference (logic) inference - The logical process by which new facts are derived from known facts by the application of inference rules.
See also symbolic inference, type inference. of abstract principles from examples, and careful monitoring of one's own understanding accompanied by efforts to clear up any misunderstandings
as they arise (Chi, Bassok, Lewis, Reimann & Glaser, 1989; Minbashian et al., 2004). Students report that the sense of being able to integrate the various ideas into one's own structure and recognize the patterns that result is a key component of their understanding (Entwistle & Entwistle, 1992). Students using deep strategies tend to have a feeling of being in control of their own learning as well as a sense of excitement. Because these students, in essence, construct their own knowledge through their strategic efforts, deep learning is often referred to as constructivist con·struc·tiv·ism
A movement in modern art originating in Moscow in 1920 and characterized by the use of industrial materials such as glass, sheet metal, and plastic to create nonrepresentational, often geometric objects. . A deep-oriented instructor would deemphasize lecturing and instead encourage questions and discussion, present multiple perspectives rather than portraying ideas as completely true or completely false, provide time and opportunities for student collaboration, relate course content to professional practice, provide frequent feedback and encourage students to monitor their own understanding.
Any given student would generally apply a mix of surface and deep strategies, with most students having a bias toward one or another and some being adept at adjusting their choice of strategy in reaction to the characteristics of a particular class or assignment. Some students, unfortunately, apply little of either type, seeming unwilling or unable to evaluate their own largely strategy-free behavior or to develop the study skills necessary to become more strategic. Many authors (Cassady, 2004; Hagedorn , Sagher & Siadat, 2000; Taylor & Hyde, 2000) point out the importance of teaching students how to study effectively in addition to teaching course content. In general, students tend to gravitate grav·i·tate
intr.v. grav·i·tat·ed, grav·i·tat·ing, grav·i·tates
1. To move in response to the force of gravity.
2. To move downward.
3. toward taking more control over their own learning and adopting more sophisticated study habits as they age and progress through college, graduate school and working careers (Baxter Magolda, 2004; Gijbels, 2005; Richardson; 1995). The conventional assumption that older students returning to school after significant work experience will have poor study practices is not necessarily correct.
Although the results in the literature are somewhat mixed, generally students emphasizing deep strategies and deemphasizing surface strategies have been found to learn more effectively and achieve better performance on assignments involving higher order learning, although their performance may not be better on lower order activities (Gijbels, 2005; Gravoso, Pasa and Mori, 2002; Minbashian et al., 2004; Taylor & Hyde, 2000). As reported by Scouller and Prosser (1994), the use of deep strategies has also been found to lead to greater student satisfaction. Thus it would seem worthwhile for instructors to behave in such a way as to encourage students to adopt deep learning strategies. Trigwell, Prosser and Waterhouse (1999) found that instructors who use surface-oriented teaching methods encourage a similar surface approach from the students and also found a similar although weaker relationship for the deep orientation. A particular danger is that instructors utilizing surface-oriented teaching methods may encourage a surface approach not only in their own classes but also in subsequent classes, making it harder for instructors in those later more advanced classes to prompt the adoption of deeper learning strategies (Raimondo, Esposito & Gershenberg, 1990).
A key question given this paper's contrast of multiple-choice and essay questions, is whether or not the type of exam questions utilized in a course will impact student learning strategies. Scouller (1998) found that students believe different abilities are being assessed by multiple-choice and essay questions, with essays being viewed as testing higher level thinking than multiple-choice. These student perceptions were found to translate into action as students applied more deep strategies and fewer surface strategies when preparing for essays. Although not specifically comparing multiple-choice and essay, Entwistle and Entwistle (1992) similarly found that students' study strategies were impacted by the level of questions they expected to see on the exam, with narrowly focused questions that stressed recall rather than the integration of ideas encouraging the use of surface strategies. Even students who had applied deep strategies throughout the semester se·mes·ter
One of two divisions of 15 to 18 weeks each of an academic year.
[German, from Latin (cursus) s and had achieved a high level of understanding felt pressured to apply surface-oriented memorization mem·o·rize
tr.v. mem·o·rized, mem·o·riz·ing, mem·o·riz·es
1. To commit to memory; learn by heart.
2. Computer Science To store in memory: in the last days leading up to the exam. However, there is also evidence suggesting that students with a general preference toward deep strategies will continue to use deep strategies even if the exam itself does not encourage that (Entwistle & Entwistle, 1992; Scouller & Prosser, 1994). An overly heavy workload The term workload can refer to a number of different yet related entities. An amount of labor
While a precise definition of a workload is elusive, a commonly accepted definition is the hypothetical relationship between a group or individual human operator and task demands. may undermine efforts to encourage deep strategies, possibly due to students believing that recall-focused surface strategies will enable them to quickly gather enough information to pass an exam that they do not have enough to time prepare for more carefully (Taylor & Hyde, 2000). Similarly, an exam on which achieving a high score is particularly critical, for example a single exam on which an entire semester's grade will be based or a highly competitive exam for admission into an exclusive graduate program, may encourage a "beat the test" mentality men·tal·i·ty
The sum of a person's intellectual capabilities or endowment. and therefore foster an increased reliance on surface strategies (Diseth & Martinsen, 2003).
Overall, the evidence suggests that the use of exams focused on higher-order learning is an important component of any teaching effort intended to encourage greater student adoption of deep strategies. Most authors view open-ended questions, including essays, as more capable of achieving a higher-order focus. Gijbels (2005), for example, suggests that even when worded in such a way as to test higher-order thinking, multiple-choice will be approached by students as a task calling for surface-oriented strategies. Nevertheless, there are some counterarguments. Wainer and Thissen (1993) suggest that skilled test writers can create multiple-choice questions that effectively test higher-order thinking and that the obvious efficiency advantages of multiple-choice should promote greater faculty efforts to develop that skill. Suskie (2004) agrees that multiple-choice questions can be written to test some, although not all, of the higher-order thinking skills from Bloom's Taxonomy taxonomy: see classification.
In biology, the classification of organisms into a hierarchy of groupings, from the general to the particular, that reflect evolutionary and usually morphological relationships: kingdom, phylum, class, order, . Similarly, but from the opposite perspective, Raimondo et al. (1990) advise that not all essay questions are necessarily higher-order as many simply call for recall-oriented answers.
There are several additional advantages and disadvantages that should be mentioned briefly to complete the discussion. A key disadvantage of multiple-choice questions, especially in light of the fact that grading efficiency is one of their key benefits, is that multiple-choice questions tend to be more difficult to write, thus negating some of the time savings achieved during the grading process. One danger of this problem is that it tends to be more difficult to write multiple-choice questions designed to test higher order thinking than those that test recall, which can result in exams that are more recall-oriented than the instructors might have actually intended (Suskie, 2004). The writing difficulty can also encourage faculty to protect their questions for use in future semesters by not handing back the graded exams or by handing them back only long enough for the students to see the scores (Roediger, 2005), a practice would not only deprive de·prive
1. To take something from someone or something.
2. To keep from possessing or enjoying something. the students of valuable feedback but might also tend to encourage a belief among the students that the test score is the important thing, not the learning. In fairness, it should be noted that the ease with which multiple-choice can be graded can allow very quick feedback in the cases in which the instructor is willing to hand back and review the exam (Becker and Johnston, 1999).
Another disadvantage of multiple-choice is that test-takers are exposed to numerous incorrect answers, many of which may be constructed so as to appear to be correct. Roediger (2005) found that students tended to remember these incorrect lures as being correct when queried about them later, suggesting that students actually learn the wrong things Wrong Things is a collaborative short-fiction collection by Poppy Z. Brite and Caitlin R. Kiernan, released by Subterranean Press in 2001. This short hardback includes one solo story by each author and one story written in collaboration, as well as an afterword by Kiernan. as part of the testing process. A related disadvantage is that students receive corrective cor·rec·tive
Counteracting or modifying what is malfunctioning, undesirable, or injurious.
An agent that corrects.
n feedback whenever their own answer does not appear as one of the available alternatives, a prompt to reconsider re·con·sid·er
v. re·con·sid·ered, re·con·sid·er·ing, re·con·sid·ers
1. To consider again, especially with intent to alter or modify a previous decision.
2. the question and correct their mistake that would not be present in an open-ended assessment (Bridgeman, 1992). Some students react to the availability of the possible answers by working backwards to answer the question, particularly on quantitative problems. Bridgeman (1992) found that 81% of the students reported working backwards to solve problems, a problem-solving methodology that would not normally be appropriate when solving realistic problems in the field.
The need to minimize the impact of this corrective feedback encourages faculty to create incorrect answers that match common student errors or that are worded in such a way as to appear to be correct. The problem with this tactic, however, is that those apparently reasonable but incorrect answers are what the students may most clearly remember later as truly being correct. Creating incorrect answers that match common student mistakes, although potentially problematic as discussed above, can also be viewed as a benefit of multiple-choice in that this method allows an instructor to quickly diagnose diagnose /di·ag·nose/ (di´ag-nos) to identify or recognize a disease.
1. To distinguish or identify a disease by diagnosis.
2. the various misconceptions Misconceptions is an American sitcom television series for The WB Network for the 2005-2006 season that never aired. It features Jane Leeves, formerly of Frasier, and French Stewart, formerly of 3rd Rock From the Sun. that appear to be occurring most frequently in a given semester.
An additional disadvantage of multiple-choice questions is that they can be gender-biased. As reported by Hassmen and Hunt (1994), numerous studies have shown that men tend to have an advantage on multiple-choice. Including essay questions on an exam can help to reduce this bias. A major weakness of essay questions is that they are relatively time-consuming to answer, meaning that a timed exam can only include a small number of questions (Walstad & Becker, 1994). This can result in an exam that is unable to evaluate student learning over all of the topics covered in that portion of the course, a problem that can penalize pe·nal·ize
tr.v. pe·nal·ized, pe·nal·iz·ing, pe·nal·iz·es
1. To subject to a penalty, especially for infringement of a law or official regulation. See Synonyms at punish.
2. some students and reward others depending upon which topics go untested. This problem of failing to test a significant portion of the course content is exacerbated (but tends to become more favorable fa·vor·a·ble
1. Advantageous; helpful: favorable winds.
2. Encouraging; propitious: a favorable diagnosis.
3. for the students) if students are allowed to select the questions that they prefer to answer from a longer list of questions provided.
A final advantage of essay, probably more accurately characterized char·ac·ter·ize
tr.v. character·ized, character·iz·ing, character·iz·es
1. To describe the qualities or peculiarities of: characterized the warden as ruthless.
2. as a disadvantage of multiple-choice, is that essay or other open-ended questions can more appropriately drive the curriculum and instructional activities. While it would be inappropriately simplistic sim·plism
The tendency to oversimplify an issue or a problem by ignoring complexities or complications.
[French simplisme, from simple, simple, from Old French; see simple to view teaching as an exercise in preparing students to pass exams, most faculty would be somewhat accepting of the idea that one of teaching's main goals is to enable students to correctly complete relevant problem-solving exercises and broadly focused integrative essays. Very few, in contrast, would view teaching toward the test to be valid if that test were made up of only multiple-choice questions (Harris & Kerby, 1997).
The intent of this study is to evaluate student preferences for multiple-choice or essay questions and, in particular, to extend that evaluation to investigate the rationale behind those preferences and the extent to which those preferences may change depending on whether the student feels himself to be well prepared or poorly prepared for an upcoming exam. Numerous studies, some cited above, have investigated the relationships between teaching methods, student study strategies and student performance. Various studies have also evaluated student preferences for multiple-choice versus open-ended questions. However, little work has been done to determine student rationales for having given preferences. The data on student preference as a function of preparedness pre·par·ed·ness
The state of being prepared, especially military readiness for combat.
Noun 1. preparedness - the state of having been made ready or prepared for use or action (especially military action); "putting them will implicitly suggest which type of question students believe requires greater preparation and may provide some insight concerning the type of exam questions that instructors should use to prompt greater study effort.
Data was gathered using a survey instrument that was completed by 81 undergraduate students enrolled in either the junior-level Principles of Management course or the senior-level Strategic Management capstone course. Because both courses are required for all business students in the program, the sample included a mix of the usual business school majors, i.e. accounting, finance, economics, management, marketing and so forth. Seven of the respondents In the context of marketing research, a representative sample drawn from a larger population of people from whom information is collected and used to develop or confirm marketing strategy. were pursuing majors other than business.
The first section of the survey, for which students were instructed to reflect upon their entire college experience and not just the course in which they were completing the survey, required the students to respond to eleven statements and two questions. Of the eleven statements, two concerned general preferences for multiple-choice or essay questions and nine concerned the rationale behind those preferences. The students responded to each statement by selecting "Disagree Strongly," "Disagree," "Neutral," "Agree" or "Agree Strongly." The nine available reasons for preferring one type of exam over another were based on discussion with fellow business faculty and extensive anecdotal evidence anecdotal evidence,
n information obtained from personal accounts, examples, and observations. Usually not considered scientifically valid but may indicate areas for further investigation and research. concerning student opinions about testing methods. These nine statements allowed the expression of opinions such as multiple-choice questions being unfair due to the inability to earn partial credit and multiple-choice being easier because the student doesn't need to know the topic thoroughly but rather just enough to recognize the right answer. More detail on the nine statements will be provided during the discussion of the survey results.
The first section of the survey ended with two questions focused on student preferences for multiple-choice or essay questions as a function of how well prepared the students were for a hypothetical Hypothetical is an adjective, meaning of or pertaining to a hypothesis. See:
- Hypothetical (album)
The second section of the survey contained a series of statements and questions that were intended to apply only to the class in which the students were currently enrolled. Although this section was designed primarily to obtain feedback on certain class-specific teaching strategies and not as part of this study, some of the results are relevant and will be included in the discussion of the survey results. The third and final section of the survey asked for demographic information such as major and grade point average. The survey concluded with an open-ended question A closed-ended question is a form of question, which normally can be answered with a simple "yes/no" dichotomous question, a specific simple piece of information, or a selection from multiple choices (multiple-choice question), if one excludes such non-answer responses as dodging a asking for comments in order to solicit input concerning any preference-related rationales that might have been omitted from the nine statements in the first section. The data were analyzed an·a·lyze
tr.v. an·a·lyzed, an·a·lyz·ing, an·a·lyz·es
1. To examine methodically by separating into parts and studying their interrelations.
2. Chemistry To make a chemical analysis of.
3. using SPSS A statistical package from SPSS, Inc., Chicago (www.spss.com) that runs on PCs, most mainframes and minis and is used extensively in marketing research. It provides over 50 statistical processes, including regression analysis, correlation and analysis of variance. .
RESULTS AND DISCUSSION
For purposes of analysis the responses to the 11 statements from the first section of the survey were coded using 1 for "Disagree Strongly" through 5 for "Agree Strongly" such that values of 1 or 2 signified sig·ni·fied
The concept that a signifier denotes.
[Translation of French signifié, past participle of signifier, to signify.]
Noun 1. disagreement with the statement, a value of three signified neutrality, and values of 4 or 5 signified agreement. The null hypothesis null hypothesis,
n theoretical assumption that a given therapy will have results not statistically different from another treatment.
n tested for each statement was "students on average do not agree with the statement," which was expressed numerically nu·mer·i·cal also nu·mer·ic
1. Of or relating to a number or series of numbers: numerical order.
2. Designating number or a number: a numerical symbol. as "the population average is less than or equal to 3.0." The alternative hypotheses of "students on average agree with the statement" was expressed numerically as "the population average is greater than 3.0." An abbreviated version of each statement can be found below in Table 1 along with the sample mean, t score and level of significance. The statements have been numbered for convenience, with the order changed somewhat from that used on the survey to facilitate the discussion that follows the table. Throughout the discussion recall that rejection of the null hypothesis signifies agreement with the relevant statement.
Note that statements 1 and 2 (which did not appear consecutively in the survey) are mirror images of one another, with the first stating a preference for multiple-choice over essay and the second stating the reverse. The correlation between these two statements was -.843, close to -1.0 as would be expected. The results for the hypothesis tests of these two statements, as shown in Table 1, led to a rejection of the null hypothesis for the statement preferring multiple-choice and an acceptance of the null A character that is all 0 bits. Also written as "NUL," it is the first character in the ASCII and EBCDIC data codes. In hex, it displays and prints as 00; in decimal, it may appear as a single zero in a chart of codes, but displays and prints as a blank space. for the statement preferring essays. Because these statements are opposites of one another it would be unreasonable to obtain data that would lead to a rejection of the null for both. The results for these two statements were evaluated further using paired t tests.
The paired t test of the difference between the multiple-choice preference and the essay preference resulted in a t score of 1.862 and a significance level of .066, suggesting that there was almost but not quite sufficient evidence to conclude that multiple-choice questions were preferred more highly. Breaking the sample into two subsets, one containing students in the junior-level Principles of Management course and the other containing students in the senior-level capstone Strategic Management course, led to results that were a little more enlightening en·light·en
tr.v. en·light·ened, en·light·en·ing, en·light·ens
1. To give spiritual or intellectual insight to: . The paired t test for the Principles of Management students achieved a t score of 2.824 and a significance level of .007, suggesting that multiple-choice questions were preferred by a significant margin. The Strategic Management students, however, produced a t score of -0.437 and a significance level of .655, suggesting no difference in preference. This finding that the preference for multiple-choice declines for more advanced students is in alignment with the ideas discussed earlier that students tend to develop more of a deep focus and a drive for greater understanding as they progress through school (Baxter Magolda, 2004; Gibjels, 2005; Richardson, 1995).
To investigate whether the preferences were impacted by the presence of a quantitative orientation, the subset A group of commands or functions that do not include all the capabilities of the original specification. Software or hardware components designed for the subset will also work with the original. of the sample majoring in accounting, finance or economics was compared to the other students. The supposition that quantitative skills might be correlated with a dislike for writing and hence a dislike of essays was not supported by the data.
As can be seen in Table 1, the null hypothesis was rejected for six of the nine rationale-related statements. The results suggested that students agreed with three of the statements related to the ease of multiple-choice: (3) Multiple-choice easier: I only need to recognize correct answer; (4) Multiple-choice easier: I can eliminate obviously wrong answers; and (5) Multiple-choice easier: I might guess correctly even if I don't know. The results also suggested agreement with one statement about the unfairness of multiple-choice, (7) Multiple-choice less fair: I can't earn partial credit. The results demonstrated support for two statements concerning essay questions, one statement about the difficulty of essays, (9) Essay Harder: I must fully understand topic to produce good answer, and one statement about the fairness of essays, (10) Essay more fair: More accurately show what I know/don't know. Only three statements were not supported: (6) Multiple-choice harder: I need to know nit-picky details; (8) Essay easier: I can earn partial credit even if topic knowledge low; and (11) Essay less fair: Content knowledge results biased by writing skill.
In general the significant statements appear to show two trends, one suggesting that students believe that multiple-choice questions are easier and another suggesting that students feel essay questions to be fairer and more valid. To further investigate this phenomenon a factor analysis was run which resulted in two factors, the first of which explained 40.46% of the variance The discrepancy between what a party to a lawsuit alleges will be proved in pleadings and what the party actually proves at trial.
In Zoning law, an official permit to use property in a manner that departs from the way in which other property in the same locality and the second of which explained 20.06%. Excluding loadings between .40 and -.40, the first factor had positive loadings on all four statements viewing multiple-choice as easier or similarly essays as more difficult, statements 3, 4, 5 and 9, and loaded negatively on all other statements. The second factor loaded on all three fairness-related statements. It loaded positively on (7) Multiple-choice less fair: I can't earn partial credit, and (10) Essay more fair: More accurately show what I know/don't know, and loaded negatively on (11) Essay lass fair: Content knowledge results biased by writing skill. The second factor also loaded positively on (4) Multiple-choice easier: I might guess correctly even if I don't know. These results suggest that the ability to obtain a high score more easily (via multiple-choice) and the ability to receive a score that accurately measures what the student has learned (via essay, except for statement 4) are both valued by the students.
While the first factor could be viewed as somewhat disappointing although certainly not surprising, the second factor is encouraging from an instructor's perspective. It is particularly encouraging to note that the inclusion of writing ability as a component of essay performance was not seen as a drawback DRAWBACK, com. law. An allowance made by the government to merchants on the reexportation of certain imported goods liable to duties, which, in some cases, consists of the whole; in others, of a part of the duties which had been paid upon the importation. of that format. It is unclear, however, whether that result is primarily due to the students having had experiences that suggest that most instructors do not pay particularly close attention to writing when grading or whether the students truly accept the idea that writing is an important skill in business and is therefore a reasonable thing to assess. It is curious that being able to guess on multiple-choice loaded positively on a fairness-related factor. Possibly the students agree with the Hassmen and Hunt (1994) contention that guessing is valid because it is generally based on at least partial knowledge of the relevant course content. These results also agree with Bridgeman's (2006) finding that although 81% of the students reported preferring multiple-choice, only 43% felt that multiple-choice was more valid. The fact that ease appears to be more important to the students than validity also supports O'Neill (2001), who studied multiple sections of the same course, some of which received multiple-choice exams and others essay exams, and found that the essay sections' students were the ones who complained about being treated badly.
The data for the two questions on preference as a function of preparedness were coded using 1 for students who preferred an exam made up entirely of multiple-choice, 2 for those who wanted a mix of multiple-choice and essay, and 3 for those who preferred entirely essay. The mean response was 1.74 under the hypothetical situation in which the student was well prepared for the upcoming exam and only 1.44 for the situation in which the student was poorly prepared. With both means less than 2.0, these results match those seen earlier in which the students in general preferred multiple-choice. However, the preference for multiple-choice diminished di·min·ish
v. di·min·ished, di·min·ish·ing, di·min·ish·es
a. To make smaller or less or to cause to appear so.
b. as the students became more prepared. A paired t test of the difference between the well prepared preference and the poorly prepared preference resulted in a t score of 3.829 and a significance level of less than .001, suggesting that the appeal of essays increases as preparedness does. In light of the results discussed above, this suggests that the fairness and validity attraction of essays may begin to overcome the easiness attraction of multiple-choice when the students have prepared sufficiently well to not fear the increased difficulty of essay questions.
Although the second section of the survey was focused on only the classes in which the respondents were registered (all of which were taught by the author) and thus asked questions geared specifically towards methods used in those classes, some of the results from that section are relevant to the discussion here. These classes all utilized a pedagogical approach in which the students were given a study guide in advance of the coverage of each chapter that pointed out the key concepts to learn for that chapter. Each study guide contained two sections, one with a list of terms and concepts to learn and another with several essay questions. The students were told to know and understand the definitions for each of the terms on the top half of the page and to recognize the key management issues related to each term. They were also told to be ready to fully answer every essay question as all of them were eligible to be selected for the exam. The essays were often broadly focused and required the students to contrast or integrate a variety of ideas, often including ideas from previous chapters. It was made very clear that complete, detailed and organized answers were required for the essays. The exams were a mix of multiple-choice and essay, with the one or two essay questions drawn word-for-word from the study guides and the multiple-choice designed such that students who had learned all of the terms from the study guides should be able to achieve a good score. The students were not able to use the text during the exams but were allowed to use their notes. They were also allowed to write out essay answers in advance of the test if they chose to do so and then simply hand in those answers if the questions they had answered turned out to be on the exam. The instructor's goal, as was explained to the class on the first day of the semester, was to provide students with a significant incentive to study carefully and take good notes, operating under the assumption that most students who did so would learn the material successfully (an assumption that end-of-semester instructor evaluation surveys have supported repeatedly).
This section of the survey was structured similarly to the first section, containing a series of statements followed by two questions concerning exam preference as a function of preparedness. Four of the statements referred to whether the respondent In Equity practice, the party who answers a bill or other proceeding in equity. The party against whom an appeal or motion, an application for a court order, is instituted and who is required to answer in order to protect his or her interests. was "encouraged to study more diligently dil·i·gent
Marked by persevering, painstaking effort. See Synonyms at busy.
[Middle English, from Old French, from Latin d than I would have otherwise" as a result of, respectively, the study guides, the use of open notes exams, the presence of essays on the exams, and the requirement for the essays to be answered very completely in order to obtain a good score. Two other statements concerned whether being required to provide more complete essay answers than usual was made fair by the student's ability to see the questions in advance on the study guides and whether the student felt that the extra study effort put forth had actually resulted in greater learning. The students demonstrated highly significant agreement with these statements, with all of them achieving a level of significance less than .001. Taken as a group, these statements suggest several things. First, students are prompted to study harder both by the presence of essays on an exam and by the fact that those essays will be rigorously graded. Second, students find it reasonable to include rigorously graded essays if the content of the questions can be viewed in advance to direct the students' study efforts. And third, students believe that their increased efforts led to greater learning than in the usual class.
These results support the literature discussed previously that demonstrated that prompting students to adopt deep-oriented study strategies would tend to encourage them to do so and that this increased adoption of deep-oriented strategies would tend to lead to an increased level of learning. Narrowing the focus to this paper's comparison of multiple-choice and essay questions, these results suggest that students are simply more fearful of essays and recognize that they need to put forth more effort to answer them effectively.
IMPLICATIONS FOR TEST CONSTRUCTION
The results of this study demonstrate that students tend to prefer multiple-choice and that their preference is driven largely by the belief that this type of question is easier, with the preference becoming even stronger when the students are poorly prepared for an exam. More encouragingly from a faculty perspective, the results also demonstrate that students appear to have an appreciation for the fairness and validity of essay questions as a measure of the success of their learning efforts and become more accepting of essays when well prepared for an exam. The results also support previous findings in the literature, for example Entwistle and Entwistle (1992), Scouller (1998), and Trigwell, Prosser and Waterhouse (1999), that suggest that instructor efforts to promote increased student utilization of deep learning strategies can be successful.
These results, in conjunction with previous work discussed earlier in the paper, prompt a variety of pedagogical recommendations for faculty who are being pressured by time or budget constraints CONSTRAINTS - A language for solving constraints using value inference.
["CONSTRAINTS: A Language for Expressing Almost-Hierarchical Descriptions", G.J. Sussman et al, Artif Intell 14(1):1-39 (Aug 1980)]. to make greater use of multiple-choice questions. First, faculty should recognize that it is possible to test certain types of higher-order learning using the multiple-choice format and that a multiple-choice exam does not therefore have to be an assessment of only memorization and recall (Anderson Anderson, river, Canada
Anderson, river, c.465 mi (750 km) long, rising in several lakes in N central Northwest Territories, Canada. It meanders north and west before receiving the Carnwath River and flowing north to Liverpool Bay, an arm of the Arctic et al., 2001; Suskie, 2004; Wainer and Thissen, 1993). Thus an increase in the use of multiple-choice questions does not necessarily lead to an inferior exam, although faculty should be aware when writing multiple-choice questions that there can be an unintentional bias toward writing questions focused on lower-level learning simply because those questions are so much easier to create (Roediger, 2005).
The results for statements 3, 4, 5 and 9 demonstrate that students believe multiple-choice questions to be easier. Thus a second recommendation is that faculty who attempt to develop higher-order multiple-choice questions must make it clear to students that these questions will call for deeper learning than is normally required by multiple-choice. Otherwise the use of multiple-choice may encourage the students to apply primarily surface-oriented study methods or to simply put less effort into studying (Gijbels, 2005; Scouller, 1998; Scouller & Prosser, 1994). They would therefore fail to achieve a level of learning sufficient to succeed on the unexpectedly difficult exam. It would be worthwhile to provide each class with example questions from previous semesters that could be utilized to demonstrate the difference between lower-order questions and those focused on higher-order learning in the context of the discipline of the course.
As an extension of the second recommendation, faculty should consider the possibility of broadening the discussion beyond simply test questions to consider higher--versus lower-order learning in general by beginning each semester with a discussion of relevant pedagogical topics such as Bloom's Taxonomy and the utilization of deep learning strategies (Cassady, 2004; Hagedorn, Sagher & Siadat, 2000; Taylor & Hyde, 2000). Most undergraduates aren't exposed to these topics and therefore haven't received the guidance that would enable them to more fully develop their own study skills. Anderson et al. (2004) provide a good discussion of Bloom's Taxonomy.
A third and closely related recommendation is that faculty should emphasize the fact that exams including higher-order multiple-choice questions will more closely mimic an essay exam's ability to distinguish between students who have utilized surface strategies to memorize a list of isolated facts and those who have applied deep strategies to develop a more complete understanding of course content. Traditional lower-order multiple-choice questions often fail to reward the use of deep strategies because surface strategies are sufficient, i.e. the students who develop a deeper understanding don't benefit tangibly via higher test scores because that deeper understanding isn't required to answer the exam questions correctly. As was demonstrated by statement 10 in this study, students have an appreciation for the fairness and validity of essays due to their ability to reward those who have worked to achieve higher-order learning. Students need to be convinced that higher-order multiple-choice questions will similarly reward those who have achieved greater understanding.
The results from the second section of this study's survey suggest a fourth recommendation that it can be very beneficial for faculty to provide clear guidelines guidelines,
n.pl a set of standards, criteria, or specifications to be used or followed in the performance of certain tasks. to students concerning the level of learning expected for each course topic. With years of experience taking traditional multiple-choice exams, in which most exam questions call primarily for memorization and recall, many students may conclude that all topics deserve approximately the same rather cursory cur·so·ry
Performed with haste and scant attention to detail: a cursory glance at the headlines.
[Late Latin curs study. With the addition of higher-order questions, however, it becomes necessary to let students know which topics are fully deserving de·serv·ing
Worthy, as of reward, praise, or aid.
de·serving·ly adv. of deeper study. Providing study guides, grading rubrics and old exams and assignments to demonstrate the level of understanding required can be an important motivator (Driessen & van der Leuten, 2000). By the way, it should be noted that in most courses there are indeed quite a few topics that really don't deserve more than cursory coverage and therefore probably shouldn't be tested via anything other than lower-order questions. Faculty must overcome the natural tendency to feel that every topic within their discipline is interesting and important in order to focus the students' deeper-oriented study efforts on the truly critical ideas.
A fifth recommendation is that faculty must allow students sufficient time both for the exam specifically and for studying and learning in general. The inclusion of higher-order multiple-choice questions will result in exams that require more reasoning time per question. If faculty write exams containing the same number of questions as before and provide the same amount of time as before, the students will be forced to approach the new higher-order questions with the same rather cursory approach that so many students have learned to utilize for multiple-choice from years of experience with lower-order questions. This situation will not only seem tremendously unfair to the students but may also lead many of them to conclude that there is little payoff from utilizing the deep study strategies that the faculty are hoping to foster. Although no faculty want to offer a class that will become known as easy, it is also important to make sure that students are given achievable learning goals and reasonable (though challenging) workloads. As noted by Taylor and Hyde (2000), students tend to gravitate toward surface strategies in order to quickly accumulate Accumulate
Broker/analyst recommendation that could mean slightly different things depending on the broker/analyst. In general, it means to increase the number of shares of a particular security over the near term, but not to liquidate other parts of the portfolio to buy a security a list of test-ready facts when there isn't enough time to learn more deeply. Providing students with insufficient time to prepare for an exam or insufficient time to take an exam may sabotage sabotage [Fr., sabot=wooden shoe; hence, to work clumsily], form of direct action by workers against employers through obstruction of work and/or lowering of plant efficiency. Methods range from peaceful slowing of production to destruction of property. faculty efforts to encourage the adoption of deep learning strategies.
A final particularly critical recommendation is that faculty should make certain to provide timely detailed feedback. One of the characteristics of students practicing deep learning strategies is a desire to monitor their own understanding and correct misconceptions as they occur (Chi, Bassok, Lewis, Reimann & Glaser, 1989; Minbashian et al., 2004). To facilitate this process, faculty will need to avoid the temptation Temptation
Terror (See HORROR.)
as fruit of the tree of knowledge in Eden, has come to epitomize temptation. [O.T.: Genesis 3:1–7; Br. Lit. to protect multiple-choice questions for repeat use in future semesters by preventing students from seeing anything more than just their exam scores. The inclusion of higher-order questions, which by definition involve more complex reasoning than would usually be seen in multiple-choice exams, makes this feedback even more critical. Faculty must hand back the exam and discuss such questions in detail so that students will fully understand the complex reasoning processes required. By carefully designing the incorrect answer options so that each incorrect answer would appear correct to students guilty of a common misconception mis·con·cep·tion
A mistaken thought, idea, or notion; a misunderstanding: had many misconceptions about the new tax program. or reasoning error, the exam can be used to diagnose these common mistakes. The incorrect answers then become a useful device to drive an enlightening discussion about those common misconceptions and reasoning errors, why they are incorrect, and how to correctly apply the ideas and skills of the discipline to find the right answer. In this way the students' incorrect answers can prompt valuable learning rather than becoming little more than uncorrected mistakes that the students may vaguely remember later as being correct (Roediger, 2005). Purdie, Hattie and Douglas (1996) investigated student utilization of a variety of learning strategies and found that the least-used of the various tactics was reviewing feedback. This suggests that students may view the score as the ultimate outcome of an exam, regardless of the type of questions used. It is important that faculty promoting deep learning strategies make the provision of meaningful informative feedback a key part of the learning process.
Anderson, L.W., D.R. Krathwohl, P.W. Airasian, K.W. Cruikshank, R.E. Mayer, P.R. Pintrich et al. (Eds.) (2001). A taxonomy for learning, teaching and assessing. New York New York, state, United States
New York, Middle Atlantic state of the United States. It is bordered by Vermont, Massachusetts, Connecticut, and the Atlantic Ocean (E), New Jersey and Pennsylvania (S), Lakes Erie and Ontario and the Canadian province of : Addison Addison, village (1990 pop. 32,058), Du Page co., NE Ill.; inc. 1884. An industrial suburb of Chicago, it manufactures machinery and plastic items. Wesley Longman.
Baxter Magolda, M.B. (2004). Evolution of a constructivist conceptualization con·cep·tu·al·ize
v. con·cep·tu·al·ized, con·cep·tu·al·iz·ing, con·cep·tu·al·iz·es
To form a concept or concepts of, and especially to interpret in a conceptual way: of epistemological e·pis·te·mol·o·gy
The branch of philosophy that studies the nature of knowledge, its presuppositions and foundations, and its extent and validity.
[Greek epist reflection. Educational Psychologist psy·chol·o·gist
A person trained and educated to perform psychological research, testing, and therapy.
psychologist , 39(1), 31-42. Retrieved June 13, 2007, from the ERIC database.
Becker, W.E. & C. Johnston (1999). The relationship between multiple choice and essay response questions in assessing economics understanding. The Economic Record, 75(231), 348-357. Retrieved February 24, 2006 from the Business Source Premier database.
Bridgeman, B. (1992). A comparison of quantitative questions in open-ended and multiple-choice formats [Electronic version]. Journal of Educational Measurement, 29(3), 253-271.
Cassady, J.C. (2004). The influence of cognitive test Cognitive tests are assessments of the cognitive capabilities of humans and animals. Tests administered to humans include various forms of IQ tests; those administered to animals include the mirror test (a test of self-awareness) and the T maze test (which tests learning ability). anxiety across the learning-testing cycle [Electronic version]. Learning and Instruction, 14, 569-592.
Chi, M.T.H., M. Bassok, M.W.Lewis, P. Reimann & R. Glaser (1989). Self-explanations: How students study and use examples in learning to solve problems [Electronic version]. Cognitive Science cognitive science
Interdisciplinary study that attempts to explain the cognitive processes of humans and some higher animals in terms of the manipulation of symbols using computational rules. , 13, 145-182.
Diseth, A. & O. Martinsen (2003). Approaches to learning, cognitive style Cognitive style is a term used in cognitive psychology to describe the way individuals think, perceive and remember information, or their preferred approach to using such information to solve problems. , and motives as predictors of academic achievement. Educational Psychology, 23(2), 195-207. Retrieved July 20, 2004 from the Academic Search Premier database.
Driessen, E. & C. van der Vleuten (2000). Matching student assessment to problem-based learning: Lessons from experience in a law faculty. Studies in Continuing Education continuing education: see adult education.
or adult education
Any form of learning provided for adults. In the U.S. the University of Wisconsin was the first academic institution to offer such programs (1904). , 22(2), 235-248. Retrieved June 14, 2007, from the Academic Search Premier database.
Entwistle, A. & N. Entwistle (1992). Experiences of understanding in revising for degree examinations. Learning and Instruction, 2, 1-22. Retrieved June 19, 2007, from the Science Direct database.
Gijbels, D. (2005). The relationship between students' approaches to learning and the assessment of learning outcomes [Electronic version]. European Journal of Psychology of Education, 20(4). Retrieved February 24, 2006, from the Academic Search Premiere database.
Gravoso, R.S., A.E. Pasa & T. Mori (2002). Influence of students' prior learning experiences, learning conceptions and approaches on their learning outcomes [Electronic version]. Proceedings of the Higher Education Research and Development Society of Australasia, 282-289.
Hagedorn, L.S., Y. Sagher & M.V. Siadat (2000). Building study skills in a college mathematics classroom [Electronic version]. Journal of General Education, 49(2), 132-155.
Harris, R.B. & W.C. Kerby (1997). Statewide performance assessment as a complement to multiple-choice testing in high school economics. Journal of Economic Education, 28(2), 122-134. Retrieved June 14, 2007 from the Business Source Premier database.
Hassmen, P. & D.P. Hunt (1994). Human self-assessment in multiple-choice testing [Electronic version]. Journal of Educational Measurement, 31(2), 149-160.
Lukhele, R., D. Thissen & H. Wainer (1994). On the relative value of multiple-choice, constructed response, and examinee-selected items on two achievement tests [Electronic version]. Journal of Educational Measurement, 31(3), 234-250.
Minbashian, A., G.F. Huon & K.D. Bird (2004). Approaches to studying and academic performance in short-essay exams. Higher Education, 47, 161-176. Retrieved Jun 14, 2007, from the Academic Search Premier database.
O'Neill, P.B. (2001). Essay versus multiple-choice exams: An experiment in the principles of macroeconomics macroeconomics
Study of the entire economy in terms of the total amount of goods and services produced, total income earned, level of employment of productive resources, and general behaviour of prices. course. The American Economist, 45(1), 62-70. Retrieved February 24, 2006 from the Business Source Premier database.
Powers, D.E., M.E. Fowles, M. Farnum & P. Ramsey (1994). Will they think less of my handwritten essay if others word process theirs? Effects on essay scores of intermingling handwritten and word-processed essays [Electronic version]. Journal of Educational Measurement, 31(3), 220-233.
Purdie, N., J. Hattie & G. Douglas (1996). Student conceptions of learning ahte their use of self-regulated learning The term self-regulated can be used to describe learning that is guided by metacognition, strategic action (planning, monitoring, and evaluating personal progress against a standard), and motivation to learn strategies: A cross-cultural comparison. Journal of Educational Psychology, 88(1), 87-100. Retrieved June 12, 2007, from the PsycArticles Publications database.
Raimondo, H.J., L. Esposito & I. Gershenberg (1990). Introductory class size and student performance in intermediate theory courses [Electronic version]. Journal of Economic Education, 21(4), 369-381. Retrieved June 11, 2007, from the Business Source Premier database.
Richardson, J.T.E. (1995). Mature students in higher education: II: An investigation of approaches to studying and academic performance. Studies in Higher Education, 20(1), 5-17. Retrieved May 29, 2006, from the Academic Search Premier database.
Roediger, H.L. III (2005). The positive and negative consequences of multiple-choice testing [Electronic version]. Journal of Experimental Psychology: Learning, Memory & Cognition cognition
Act or process of knowing. Cognition includes every mental process that may be described as an experience of knowing (including perceiving, recognizing, conceiving, and reasoning), as distinguished from an experience of feeling or of willing. , 31(5), 1155-1159.
Scouller, K. (1998). The influence of assessment method on students' learning approaches: Multiple choice question examination versus assignment essay [Electronic version]. Higher Education, 35, 453-472.
Scouller, K.M. & M. Prosser (1994). Students' experiences in studying for multiple choice question examinations. Studies in Higher Education, 19(3), 267-279. Retrieved May 29, 2006, from the Academic Search Premier database.
Suskie, L. (2004). Assessing Student Learning: A common sense guide. Bolton, MA: Anker Publishing.
Taylor, R. & M. Hyde (2000). Learning context and students' perceptions of context influence student learning approaches and outcomes in animal science 2. Proceedings of the Teaching and Educational Development Institute Conference on Effective Teaching and Learning at University. Retrieved June 13, 2007, from http://www.tedi.uq.edu.au/conferences/teach_conference00/ papers/taylor-hyde.html
Thissen, D., H. Wainer & X-B Wang (1994). Are tests comprising both multiple-choice and free-response items necessarily less unidimensional than multiple-choice tests? An analysis of two tests [Electronic version]. Journal of Educational Measurement, 31(2), 113-123.
Trigwell, K., M. Prosser & F. Waterhouse (1999). Relations between teachers' approaches to teaching and students' approaches to learning [Electronic version]. Higher Education, 37, 57-70.
Walstad, W.B. & W.E. Becker (1994). Achievement differences on multiple-choice and essay tests in economics [Electronic version]. Proceedings of the American Economic Association, 193-196.
Wainer, H. & D. Thissen (1993). Combining multiple-choice and constructed-response test scores: Toward a Marxist theory of test construction [Electronic version]. Applied Measurement in Education, 6(2), 103-118.
David A. Parmenter, Northern Arizona Northern Arizona is dominated by the Colorado Plateau, the southern border of which in Arizona is called the Mogollon Rim. In the West lies the Grand Canyon, which was cut by the flow of the Colorado River while the land slowly rose around it. University-Yuma
Table 1: Hypothesis Tests of Student Agreement with Assessment Method Statements Statements (abbreviated versions) Mean t score Significance (1) I prefer multiple- 3.30 2.375 .01 choice questions to essay questions (2) I prefer essay 2.84 -1.227 .888 questions to multiple- choice questions (3) Multiple-choice 3.38 3.339 <.001 easier: I only need to recognize correct answer (4) Multiple-choice 3.70 6.008 <.001 easier: I can eliminate obviously wrong answers (5) Multiple-choice 3.52 5.208 <.001 easier: I might guess correctly even if I don't know (6) Multiple-choice 2.79 -1.645 .948 harder: I need to know nit-picky details (7) Multiple-choice less 3.47 3.977 <.001 fair: I can't earn partial credit (8) Essay easier: I can 3.14 1.085 .141 earn partial credit even if topic knowledge low (9) Essay harder: I must 3.42 3.693 <.001 fully understand topic to produce good answer (10) Essay more fair: 3.69 6.888 <.001 More accurately show what I know/don't know (11) Essay less fair: 2.70 -2.449 .991 Content knowledge results biased by writing skill
|Printer friendly Cite/link Email Feedback|
|Author:||Parmenter, David A.|
|Publication:||Academy of Educational Leadership Journal|
|Date:||May 1, 2009|
|Previous Article:||Faculty utilization of end-of-chapter textbook material in developing intermediate accounting students' professional competencies.|
|Next Article:||A comparison of the achievement of students taught by full-time versus adjunct faculty in business courses.|