Stochastic misconceptions of pre-service teachers.

Abstract

Analyzing explanations from 108 participants on a probability and statistics assessment, this study examined the use of two stochastic misconceptions by pre-service K-8 teachers. Representativeness heuristics were used more often when the problem was set in a social context rather than a traditional mathematical setting. Although some participants used representativeness to guide them to a correct answer, most participants were led astray by this heuristic. The majority of the participants did not realize that a conjunction is less probable than either of its constituent parts.

Introduction

According to Shaughnessy (1981) and others, probability is one of the mathematical topics for which misconceptions are highly likely. Shaughnessy (1992) also postulated that unless we deal with misconceptions of teachers, we cannot expect them to help their students. Therefore, this study examines specific misconceptions involving the representativeness heuristic and the conjunction fallacy of future K-8 teachers with the thought that a better understanding of these misconceptions will guide curricular reforms and lead future teachers to a solid background in probability and statistics.

Misconceptions are systematic, rather than careless, errors. Misconceptions can coexist with competing correct ideas in the minds of students unless they are directly confronted and remediated (Carpenter & Hiebert, 1992). Psychologists are interested in stochastic misconceptions because they "provide information about the processes underlying subjective judgments of probability" and "suggest limitations on the quality of human judgments" (Gavanski & Roskos-Ewoldsen, 1991, p. 181). The work of Amos Tversky and Daniel Kahneman showed that the same heuristics that help people make daily decisions under uncertainty also lead to "severe and systematic errors" that contradict established probability theory (1974, p. 1124).

One example is the representativeness heuristic for which people use the degree to which event A resembles class B to estimate the probability that event A belongs to class B (Tversky & Kahneman, 1974). Additionally, "people expect that a sequence of events generated by a random process will represent the essential characteristics of that process even when the sequence is short" (Tversky & Kahneman, 1974, p. 1125). For instance, the sequence of coin tosses HTHTTH appears to be more random (and thus more likely) than HHHTTT. Similarly, the sequence HHHTTT is perceived to be more likely than HHHHTH because the former appears to represent the fairness of the coin better (Tversky & Kahneman, 1974). However, in reality, all 64 outcomes are equally likely. Fast (1997) gave a similar question to secondary mathematics student teachers and found that a third of them failed to provide the correct answer. Tversky and Kahneman (1974) recognized people's belief that all samples were representative of the population regardless of sample size; specifically, people believed that approximately half the babies born in a hospital (regardless of the number of babies born in that hospital) would be male. This contradicts the law of large numbers which indicates that a large sample should be much more representative of the population (50% male) than a small sample. Fischbein and Schnarch (1997) used the same concept with 500 students from fifth grade through college. Only 1% of their participants chose the correct answer, while over 55% indicated that the probabilities were equivalent in both hospitals (Fischbein & Schnarch, 1997).

Tversky and Kahneman (1983) also explored the conjunction fallacy in which people do not realize that the probability of the conjunction of A and B is less than both the probability of A and the probability of B because the conjunction of A and B is contained in both A and B. A possible explanation for this is that people are judging representativeness rather than probability when this problem is placed in a context (Tversky & Kahneman, 1983). However, the fallacy still occurs (though at a lesser rate) when the problem lacks a context (Gavanski & Roskos-Ewoldsen, 1991). Evidence for the conjunction fallacy has been reported for students in many age groups ranging from second grade to college (Davidson, 1995; Fischbein & Schnarch, 1997). Tversky and Kahneman (1983) also point out that this problem is unusual because people handle conjunctions better in their abstract form than in their concrete form which is the opposite of most mathematical concepts.

Research Questions

Given the importance of probability in the K-8 curriculum (National Council of Teachers of Mathematics, 2000) and the complex issues related to the representativeness heuristic and conjunction fallacy; it is important to know the extent of the problem for future K 8 teachers. Therefore, to what extent do pre-service teachers have difficulty with the representativeness heuristic and the conjunction fallacy? Do the reasons that participants give for their answers indicate a consistent underlying reason for the misconception? Fischbein and Grossman (1997) emphasize that the structuring schemata underlying intuitions must be identified so that the schemata can be trained which will in turn improve intuitions. O'Connell (1999) indicates that interrelated misconceptions mask remediation effects. Therefore, we want to know if the results from one question on this survey are indicative of results from another question.

Methodology

This study utilized a within-stage mixed-model design (Johnson & Onwuegbuzie, 2004) for an online survey conducted during the summer of 2004. Initially, data were analyzed quantitatively to investigate the overall implications. Based on the quantitative results, the extended-response answers were explored qualitatively to understand context i.e., the nature and structure of the participants' responses. To facilitate data analysis, the answer choices for each question were ranked from most correct to least correct with the highest score signifying the optimal answer. This provided the ordering necessary to conduct statistical analyses requiring ordinal scaling. Although the scaling is not interval in nature, it is approximately interval, so statistical analyses requiring interval scaling can be interpreted as an approximation. To ensure that the samples were similar across the control variables and could be considered as one group, a Kolmogorov-Smirnov two-sample test was performed for each survey question using instructor as the grouping variable, and a Kruskal-Wallis one-way analysis of variance was performed for each survey question using degree program (leading toward certification in grades Pre-Kindergarten (PK)-4, leading to certification in grades 4-8, or not leading to certification). The first research question was answered by analyzing the percentages for each answer choice. Extended-response answers to each survey question were categorized using Glaser and Strauss' method of constant comparison (Lincoln & Guba, 1985). These categories were used to identify structuring schemata underlying intuitions. A Pearson r correlation using all four survey questions was performed to determine relationships among the questions.

Participants

This study occurred at a large, southern, public university in the United States. The participants were enrolled in an introductory statistics class offered through the college of education in partial fulfillment of certification requirements for pre-service teachers who were not specializing in mathematics or science, but seek certification for grade bands PK-4 or 4-8. This class does not explicitly cover probability topics. Of the 108 participants who took this survey, 95.4% were female. Eighty-eight percent were Caucasian, 6.5% were Hispanic/Latino, 0.9% were African American, and 4.6% were of other ethnicitics or did not respond. Seventy-five (69.4%) of these participants were enrolled in a program to earn certification to teach grades PK-4, while 25 (23.2%) were enrolled in a program to earn certification to teach grades 4-8, and eight (7.4%) were not enrolled in a certification program or did not respond. Participants responded during the last two weeks of consecutive five-week summer courses (n 34, n = 74; respectively). As mentioned earlier, these two groups did not differ on misconceptions using the Komogorov-Smimov and the Kruskal-Wallis by instructor or degree program. These results are important because the sample self selected instructor and section. It was important to investigate whether the groups were similar at the onset so subsequent analyses could use the entire data set.

Instrumentation

The questions used for this research (see Appendix, issue website http://rapidintellect.com/AEQweb/fal2005.htm) were administered as part of a larger online survey that also examined conceptual and procedural understandings of central tendency and standard deviation (questions 2, 4, and 6), and attitudes toward statistics (a separate survey). Only the four questions concerning probabilistic misconceptions were of concern for this study. Participants were allowed access to this survey online (one week for the first class and two weeks for the second class). The questions for this part of the study were chosen to align with Tversky and Kahneman's classic problems relating to the representativeness heuristic and conjunction fallacy, but their style aligns more closely with the version of the classic questions used by Gerald Fast (1997) in his WDYTTCA (What Do You Think The Chances Are?) instrument.

Questions one and three focus on the representativeness heuristic studied by Tversky and Kahneman (1974). The first question measures misconceptions of chance and the third centers on misconceptions of sample size in which participants only consider ratios rather than sample size. Questions five and seven focus on the conjunction fallacy, the belief that a conjunction is less probable than its constituent parts. Question seven is set in a social context whereas question five was designed to be devoid of context.

Results

The Komogorov-Smirnov two-sample test analyzing the instructor effect for the two classes that took this survey, with the order of the answer choices ranked from least correct to most correct, found no statistically significant difference between the two classes on any of the questions (p = 0.174, p > 0.999, p = 0.896, p = 0.982; respectively). Similarly, using the Kruskal-Wallis one-way analysis of variance, no statistically significant differences were found among the three degree programs on any of the questions (p = 0.340, p = 0.877, p = 0.987, p = 0.918; respectively). See Table 1 ['or the number of participants' responses to the four multiple-choice questions. See issue website http://rapidintellect.com/AEQweb/fal2005.htm

Representativeness Heuristic

The results of this survey indicate that the representativeness heuristic can both aid and hinder participant performance. For the first question, 81 of the 108 participants (75%) correctly answered that all choices were equally likely while only 15 participants (13.9%) correctly answered that three tails in five flips is more likely than 3000 tails in 5000 flips for question three (Q3) concerning the law of large numbers.

Sixty-seven of 108 participants (62%) treated the birth order and law of large numbers problems similarly and claimed that the choices were equally likely. The following typical elaborations on these two items illustrate this similar treatment: "The probability of a child being a boy is 50%, and the probability of a child being a girl is 50%," and "There is a 50/50 chance of heads or tails." An over-reliance on the concept of "equally likely" was evidenced by participants. Seventy (86.4%) of the 81 participants who correctly answered question one (Q1) also chose the "equally likely" response for one or more of the other questions where it was not correct (67 for Q3, 30 for Q5 and 40 for Q7). Additionally, 19 of the 108 participants chose the "equally likely" response choice for each of the four questions.

Eight of the 71 participants (11.2%) who correctly answered the birth order question and wrote an explanation, did so by using the representativeness heuristic to their advantage with comments such as "Going by my own experience and families that I know, I have seen all of these birth orders firsthand so I believe any birth order is possible." All 13 of the participants who missed the birth order question and wrote responses referred to the 50/50 chance of having a boy. Five of those 13 even pointed out that the 50/50 chance was renewed for each birth, but they did not realize that this made all birth orders equally likely. Of the 83 participants who gave written responses to the law of large numbers question, 23 (27.7%) of them reasoned that since the proportions were the same, the choices were equally likely. Only seven (8.4%) of the participants who gave written responses invoked the law of large numbers with comments such as, "Because there is a 50/50 chance of getting heads or tails, the larger sample size should result in a more even probability of heads and tails and your results should be closer to 2500 out of 5000."

There were four participants who correctly answered the law of large numbers question (Q3) and missed the birth order question (Q1). Only one of these participants gave sound reasoning for Q3, two did not write an explanation, and the remaining participant used the representativeness heuristic to his/her advantage. One participant who marked that 3000 of 5000 flips was more likely to occur seemed to contradict that answer with the explanation: "It seems more likely that if you flipped a coin that many times (5000) that it would be closer to 50/50, where as 3 out of 5 isn't far away from 50/50."

Conjunction Fallacy

Although the probability of a conjunction is always less than or equal to its constituent parts, most participants in the survey did not use this line of reasoning. On question five regarding the coin and dice game, 49 (45.4%) correctly stated that rolling a five after flipping a coin was most likely, but 43 (39.8%) indicated that all choices were equally likely. For the conjunction problem situated in the social context of a doctor and a woman, 31 respondents (28.7%) correctly stated that, of the choices given, the woman is most likely a doctor, 56 (51.9%) said that the choices were equally likely, and 21 (19.4%) said that she is most likely a doctor and a mother.

Although participants may have considered real-life experiences, none of the 68 written responses to the question about the coin/die game indicated real-life representations as a consideration in the problem utilizing a less social context. However, 42 of the 73 (58.3%) written responses to the doctor/woman question referenced the real world (four of these 42 apparently helped the participants while the remaining 38 led the participants astray). Of the 31 participants who correctly answered the doctor/woman question, 20 of those also correctly answered the coin/die question. Of the 22 participants who correctly answered the doctor/mother question and wrote responses, 15 directly stated information about the conjunction of two events being less probable than their constituent parts while six of the remaining seven participants alluded to this fact. An example of these comments is, "The woman is a doctor and a mother is more specific than the woman is a doctor, so we would have to multiply the 2 factors together as opposed to knowing just the one. In other words we'd have to multiply the probability that the woman is a doctor by the probability the woman is a mother which would result in a lower probability than simply the woman is a doctor since neither situation has a 100% chance of occurrence."[1] Only nine of the 21 participants who cited or alluded to the conjunction of two events as being less probable than their constituent parts made similar comments or allusions on the coin/die problem. Of the 11 participants who correctly answered the doctor/mother question and incorrectly answered the coin/die question, two of them indicated a misunderstanding of the instructions to the "game" for the coin/die problem, four did not give an explanation, and the remaining five gave answers indicating that everything was equally likely. Most of the answers to the doctor/mother question relied on representativeness indicating the strong influence of the social context. Two example responses are, "It is probably impossible to say 'equal." Are there as many women doctors as there are mothers? There are many other factors that influence this probability - hospital or street? what part of the world? urban or rural? what kind of doctor? etc." and "if a woman is carrying a baby, she probably is the mother, so being a doctor is the only thing that 1 see as a big chance and both of the answers have that option." Of the 29 participants who correctly answered the coin/die problem but missed the doctor/woman problem, 24 gave written explanations, 18 of which indicated that the representativeness heuristic led the participants in the wrong direction.

Only three of the 108 participants (2.8%) answered all four questions correctly. The Pearson correlations between these four questions were weak. The strongest correlation was 0.159 between Coin/Die and Doctor/Woman, which was not statistically significant (p = 0.103).

Discussion

These results indicate that pre-service teachers have difficulty with the representativeness heuristic, the conjunction fallacy, and deciding when situations are "equally likely." For the first question, thirteen participants (12%) said that GGGBBB and GBGBGB are equally likely, and more likely than GGGGGB. This indicates that they believe that even small samples should reflect the same probabilities as a single birth and should contain 50% boys and 50% girls. The fact that 12 participants (11.1%) chose GBGBGB as the most likely choice, indicates that in addition to believing that half the children should be boys, these participants probably believe that alternating the genders appears more random than grouping all the girls then all the boys.

Most participants realized that all choices were equally likely for the birth order question. Based on the written responses, it appears that the representativeness heuristic was helpful to most participants. The pervasive belief that choices are equally likely (whether or not they really are equally likely) may help explain why many participants could correctly answer the birth order question and not other questions. According to Cox and Mouw (1992), correct use of the representativeness heuristic for this problem reinforces the heuristic and makes it harder to interrupt when other questions are posed.

Participants who answered that both choices were equally likely for Q3 were probably using the representativeness heuristic to their detriment. However, the law of large numbers indicates that it would be much more unusual to get tails 60% of the time when flipping a fair coin in a large sample than in a small sample because large samples are much more representative of the population than are small samples. The explanations, "It seems more likely that if you flipped a coin that many times (5000) that it would be closer to 50/50, where as 3 out of 5 isn't far away from 50/50" for the coin problem and "I know of many families with many children and there seems to be no pattern behind birth order or amount of girls and boys in any of the families" for the birth order problem, from one participant exemplify answering based on heuristics rather than probability knowledge. Although nothing in life is truly devoid of context, the written comments on the conjunction problems follow the expectations of Gavanski and Roskos-Ewoldsen (1991) and indicate that setting one conjunction problem in a social context and the other one not in a social context makes a large difference in the interpretation of the problems. The large number of written responses based on references to the real world indicates that both the conjunction fallacy and the representativeness heuristic played a part in the doctor/woman question.

In Davidson's work with children, "no clear-cut relationship was found between children's use of the representativeness heuristic on the standard problems and their tendency to commit the conjunction fallacy on the conjunction problems" (1995, p. 340). The lack of correlation between questions indicated by the Pearson correlations leads to the same conclusions in this study. Of the 83 participants who gave written responses to the law of large numbers question, 23 (27.7%) of them reasoned that since the proportions were the same, the choices were equally likely. It is likely that this stems from the curricular emphasis on use of proportions without comments of when proportions would not be an appropriate tool. All of the participants who missed the birth order question and wrote responses referred to the 50/50 chance of having a boy. Five of the 13 even pointed out that the 50/50 chance was renewed for each birth, but they did not realize that this made all birth orders equally likely. This is evidence that the participants have learned about equal probabilities, but they assume equality of probabilities even when it is not appropriate.

As with the representativeness heuristic questions, the conjunction fallacy problems (Q5 and Q7) also indicate that participants are having trouble with the concept of "equally likely." Answers such as, "These are equal because women can be mothers and doctors" only take into account the number of options rather than the actual probability of each option.

Reasons and Remedies

Why do these misconceptions persist and what can be done about them? Hawkins and Kapadia (1984) blame the representativeness problem on contemporary education because it forces deterministic cognitive strategies rather than supporting the development of probability concepts in an indeterministic environment. Cox and Mouw (1992) highlight that these heuristics sometimes yield correct predictions making it even more difficult to replace them. Fischbein and Gazit (1984) experienced some success with the representativeness misconception as a benefit of a 12-lesson program with 6th and 7th grade students covering probabilities and chance. Cox and Mouw (1992) experienced some success with college students and the conjunction effect through the use of problem experiences that added or removed heuristic triggering cues to disrupt the faulty logic. Fast (1997) used Cox and Mouw's "'cues removed" scenario to create "anchors" which people could use for decisions in scenarios that were more prone to misconception. The concept of anchors is a form of scaffolding that is supported by constructivist theory (Fast, 1997). These anchors helped the college-student participants a majority of the time (Fast, 1997). Regardless of the particular procedure used, the student must be actively involved in the process (Mevarech, 1983; Shaughnessy, 1977). This study lends support to fortify teacher preparation programs by including probability and directly addressing misconceptions. If these misconceptions are allowed to persist in the portion of the population who possess the greatest potential to influence what and how K-12 students learn probability, then what is the chance that K-12 students will be better prepared to understand the real-world implications fbr stochastic events?

Summary

The results indicate that these participants do not have a sufficient intuitive understanding of chance. Since probability is essential for statistics, perhaps undergraduate students should have a formal introduction to probability prior to their required statistics class. Alternatively, as Shaughnessy (2003) suggested, data could be used to teach probability. Either way, our future K 8 teachers need to attain a stronger probabilistic background prior to entering the classroom.

References

Carpenter, T. P. & Hiebert, J. (1992). Learning and teaching with understanding. In D. A. Grouws (Ed.). Handbook of research on mathematics teaching and learning (pp. 65-97). Reston, VA: National Council of Teachers of Mathematics.

Cox, C., & Mouw, J. T. (1992). Disruption of the representativeness heuristic: Can we be perturbed into using correct probabilistic reasoning? Educational Studies in Mathematics, 23, 163-178.

Davidson, D. (1995). The representativeness heuristic and the conjunction fallacy effect in children's decision making. Merrill-Palmer Quarterly, 41, 328-346.

Fast, G. R. (1997). Using analogies to overcome student teachers' probability misconceptions. Journal of Mathematical Behavior, 16, 325-344.

Fischbein, E., & Gazit, A. (1984). Does the teaching of probability improve probabilistic intuitions? Educational Studies in Mathematics, 15, 1-24.

Fischbein, E., & Grossman, A. (1997). Schemata and intuitions in combinatorial reasoning. Educational Studies in Mathematics, 34, 27-47.

Fischbein, E., & Schnarch, D. (1997). The evolution with age of probabilistic, intuitively based misconceptions. Journal for Research in Mathematics Education, 28, 96-105.

Gavanski, I., & Roskos-Ewoldsen, D. R. (1991). Representativeness and conjoint probability. Journal of Personality and Social Psychology, 61, 181-194.

Hawkins, A. S., & Kapadia, R. (1984). Children's conceptions of probability--a psychological and pedagogical review. Educational Studies in Mathematics, 15,349-377.

Johnson, R. B., & Onwuegbuzie, A. J. (2004). Mixed methods research: A research paradigm whose time has come. Educational Researcher, 33(7), 14-26.

Lincoln, Y. S., & Guba, E. G. (1985). Naturalistic inquiry. Newbury Park, CA: Sage. Mevarech, Z. R. (1983). A deep structure model of students' statistical misconceptions. Educational Studies in Mathematics, 14, 415-429.

National Council of Teachers of Mathematics. (2000). Principles and standards for school mathematics. Reston, VA: Author.

O'Connell, A. A. (1999). Understanding the nature of errors in probability problem-solving. Educational Research and Evaluations, 5, 1-21.

Shaughnessy, J. M. (1977). Misconceptions of probability: An experiment with a small-group, activity-based, model building approach to introductory probability at the college level. Educational Studies in Mathematics, 8,295-316.

Shaughnessy, J. M. (1981). Misconceptions of probability: From systematic errors to systematic experiments and decisions. In A. P. Schulte & J.R. Smart (Eds.), NCTM 1981 Yearbook (pp. 90-100). Reston, VA: NCTM.

Shaughnessy, J. M. (1992). Research in probability and statistics: Reflections and directions. In D. A. Grouws (Ed.), Handbook of Research on Mathematics Teaching and Learning (pp. 465-494). New York: Macmillan.

Shaughnessy, J. M. (2003). Research on students' understanding of probability. In J. Kilpatrick, W. G. Martin, & D. Schifter (Eds.), A research companion to principles and standards for school mathematics (pp. 216-226). Reston, VA: NCTM.

Tversky, A., & Kahneman, D. (1974). Judgment under uncertainty: Heuristics and biases. Science, 185, 1124-1131.

Tversky, A., & Kahneman, D. (1983). Extensional versus intuitive reasoning: The conjunction fallacy in probability judgment. Psychological Review, 4, 293-315.

Footnote 

[1] It is clear that the student cited (and many others) assumed the independence of being a doctor and being a mother. We did not pose any questions directly related to independence, so we do not have evidence to support whether or not the students made this assumption deliberately.

Tamara Anthony Carter, Texas A&M University

Robert M. Capraro, Texas A&M University

Tamara Anthony Carter is a mathematics education graduate student with interests in teacher preparation. Robert M. Capraro is an assistant professor of mathematics education. His interests" include mathematical assessment and preparation of mathematics teachers.