An analysis of calculator use and strategy selection by prison inmates taking the official GED practice test.
In the Curriculum and Evaluation Standards for School Mathematics (NCTM, 1989), the National Council of Teachers of Mathematics recommended that calculators be made available at all grade levels. Since then, most schools have incorporated calculator use into mathematics curricula, as have many standardized tests. The American Council on Education began allowing calculator use on its Test of General Educational Development (GED) in 2002, when a new generation of the test was introduced. The mathematics subject test is given in two parts; Part I is accompanied by a calculator, and Part II is not.
In theory, the calculator is a computational aid that will enable the GED test to better measure students' problem-solving abilities rather than their arithmetic skills. While this position makes intuitive sense, and is supported in statements by the NCTM, it may not be the case for all students. Prison inmates largely do not participate in mainstream education, and therefore cannot be expected to interact with calculators in the same way as the general population. This study will address two important questions: First, do incarcerated students use the calculator appropriately when taking the Official GED Pactice Test? And second, if not, why not?
The American Council on Education provides a calculator on the GED to reflect trends in mainstream and higher education. (ACE, 2009) Calculators are expected to increase the content validity of mathematical reasoning tests by preventing errors in calculation from artificially deflating scores. While there is ample research supporting this as demonstrating that graphing calculators improve students' attitudes toward and retention of mathematics (Isiksal & Askar, 2005), there is little if any surrounding the issue of inmates' use of, and attitudes toward, calculators in a testing situation.
This study aims first to determine whether these inmates chose appropriate problems on which to use the caluculator. A detailed typology is described in the literature review, but as it mostly refers to graphing calculators, additional criteria are measured. Students were asked which questions they solved with the aid of the calculator, and these questions were examined for number and type of operations, number of decimal places, item difficulty, and content.
The GED test is in some ways unique among standardized tests. It is designed to measure the educational development of adults, and as such, contains many questions that reflect real-life experiences. The questions often involve quantities as opposed to abstract mathematical concepts; for example, test-takers may be asked to calculate a tip or change from a purchase, or an amount of square yardage. There are also a few abstract questions, such as finding the x-intercept of y = 3x - 6 or reflecting a circle in the coordinate plane. It may be that these questions are intended to measure students' ability to succeed in college instead of the workforce (ACE, 2009).
Very often, the real-world questions can be solved using estimation, mental math, or informal methods devised by the student, rather than traditional school-taught algorithms. For example, to find 15% of $240, the formal algorithm would require students to either multiply 240 by 0.15, or to solve claim (Ellington, 2003; Hembree & Dessart, 1986), as well the proportion x/240 = 15/100. Informally, however, the student Thesis Director: Srdjan Divac can find 10% of 240 by dropping the zero, then finding half of that and adding the two quantities: "Ten percent of 240 is 24, and half of that is five percent, which is 12. Add them and you get 36, or fifteen percent."
Nunes, Schliemann, and Carraher (1993) have termed this style of problem-solving "street mathematics", and noted that many practitioners use spoken instead of written procedures. In a testing situation, where talking is prohibited, students use what is commonly called "mental math." An examination of the scrap paper collected at the end of an OPT session reveals that on a 25-question test, most students only use the scrap paper for about 3-5 questions.
The calculator seems to be inconsistent with street mathematics. Perhaps it is, or is perceived as, a tool of written mathematics. One student remarked that with a calculator, there is no way to see the process of computation, so one must accept its answers on faith. Another claimed that his inability to use the memory function prevented him from keeping track of intermediate results. The design of the study allows some time for each student to describe their problem-solving process. Attention is paid to mental versus written methods, and how the calculator interacts with each.
In accordance with state law, all inmates are given reading and mathematics competency tests upon being committed to the facility. The test used is the Test of Adult Basic Education (TABE), which scores students according to grade level and month in both reading and math. A score of 6.2 would mean that the student performed at the equivalent of someone in the second month of sixth grade.
While students sometimes test below their actual level due to alcohol and drug withdrawal, anger at being incarcerated, or apathy, the education department has found the reading score to be a reasonably accurate predictor of academic ability. Scoring 8.0 or above makes an inmate eligible for GED classes, while scoring 7.9 or below means the inmate must take basic literacy (ABE) first. Note that the education department does not use the TABE math scores for placement; other departments, such as vocational education, may use these scores. It is assumed that the TABE math test does not correlate enough with the GED to predict performance. Thus it happens that a typical GED class consists of students with a wide range of math abilities.
The Language Arts Reading, Social Studies, and Science subject area tests of the GED can often be passed without prior study by someone with an eighth-grade or higher reading level. Questions in these sections require very little outside knowledge and instead tend to require students to analyze a paragraph, excerpt, or graphic. The Language Arts
Writing test requires an essay, which does demand a certain amount of class time, but the majority of GED classes at the facility are devoted to math. Teaching styles and materials vary, but the education department generally maintains a pass rate of about 50% on the OPT and about 50% on the full battery, or about 25% for each cycle. If a student takes the full battery and does not pass one or more subjects, they may retake those subjects within three years to earn a credential. Each retest costs $15.
The GED costs $75 per student. Since the Sheriff's office pays for the test, the education department must justify the cost by showing that each student registered has a reasonable chance of passing. To this end, the education department administers an Official Practice Test (OPT) a few weeks before the next scheduled GED testing session. As the ACE (2002) claims the OPT predicts actual GED scores within 50 points, and as the combined passing score on the GED is 2250, the education department only registers those students who earn an overall score of 2200, as well as at least 410 on each subject, on the OPT.
The OPT is half the length of the GED, and half the amount of time is allotted. Recall that the GED mathematics subtest is composed of 50 questions, 25 for each of Part I and Part II, and students are given 45 minutes to complete each part, for a total of 90 minutes. The OPT mathematics subtest is divided into 13 questions for Part I and 12 for Part II. Part I is to be completed in 23 minutes, and Part II in 22 minutes, for a total of 25 questions in 45 minutes. Note also that the education department does not hand out separate booklets as in the GED; students have all test booklets in a folder on the table and are instructed to only open the relevant part. Students are told that when time is called for Part I, they must relinquish the calculator and move on to Part II.
Review of Literature
Most of the existing literature about calculator use on tests analyzes the calculator's effect on test scores or test validity. It would seem that the most common topic in the ongoing dialogue about calculator use in math education is whether calculators artificially inflate scores or prevent tests from measuring certain criteria. (Hembree & Dessart, 1986) There is also some controversy surrounding the interaction between calculators and mental arithmetic. (Ruthven, 1998; Bing, Stewart, & Davison, 2009)
Bing et al. (2009) investigated the effects of calculator use on an employment test of mathematical ability. They found that there is a small decrease in test validity on decimal problems, particularly when performing division operations, while using a calculator. They state:
In sum, calculator use appears to harm the integrity of the measurement of computational ability by allowing less capable test takers to both attempt and answer correctly integer and decimal items of complex operations that they would otherwise not reach without the use of a calculator. However, this effect did not occur for fractions, which, even if changed to decimals by test takers with the use of a calculator, would still have to be converted back to fractions by the lowest common denominator to obtain an answer. (p. 338)
It should be noted that the f x - 260 calculator provided during the GED and pretest has an interface for fractions, and fraction arithmetic is handled automatically. Results are displayed in fraction form rather than in decimal form when computations are entered as such. Fractions are also automatically reduced to lowest terms.
Bing et al. (2009) refer to a possible "reduction or an elimination of mental arithmetic that could result from allowing test takers to use calculators and a subsequent change in the construct measured because of calculator use" (p. 334). They also claim that "A better test for changes in validity from calculator use would be provided by a more comprehensive measurement of mathematical reasoning" (p. 326).
Other studies involving adults and tests included a comparison of the arithmetic skills of nurses who had or had not used calculators in school (Hutton, 1998). The study was conducted in England, where nursing schools had (as of 1998) traditionally prohibited the use of calculators on tests. However, many of Hutton's students claimed that since they had been taught to do arithmetic with calculators, prohibiting their use on tests resulted in lower scores. Hutton found an increase in accuracy of computation, particularly among students who had most recently left school, but no significant change in conceptual skills. In her words, "although calculator use is associated with increased accuracy in arithmetic, its effect on conceptualisation of a mathematics problem is minimal" (p. 168). Alternatively, Shockley, McGurn, Gunning, Graveley, and Tillotson (1989) found a significant decrease in conceptual ability when using a calculator.
A meta-analysis of calculator studies (Hembree & Dessart, 1986) found that calculators caused no harm, and in fact correlated with significant improvements, to K-12 students. In all but Grade 4, calculator use seemed to improve attitudes and basic arithmetic skills. According to the authors,
The use of calculators in testing produces much higher achievement scores than paperand-pencil efforts, both in basic operations and in problem solving. This statement applies across all grades and ability levels. In particular, it applies for low- and high-ability students in problem solving. The overall better performance in problem solving appears to be a result of improved computation and process selection. (p. 96)
While this is at odds with the findings of Shockley et al. (1989), it is a much more mainstream view.
It is believed by the education department staff at the facility that a high proportion of inmates suffer from learning disabilities (LD) and/or Attention Deficit Hyperactivity Disorder (ADHD). This information is hard to verify, as students must self-report IEPs and diagnoses due to privacy concerns, however, it is thought by some that the proportion is as high as 50%. (M. McIsaac, personal communication, January 12, 2010) In her dissertation, Parks (2009) studied the relationship between calculators and math anxiety among students with LD/ADHD. She found that more students reported an increase in anxiety with calculator use than without. Although the increase was statistically insignificant, in fact no more than could be expected by chance, it does seem worth noting: in three disability status groups (no disability, LD, ADHD) of 15 students each, the number of participants who reported an increase in anxiety was 7, 8, and 10 respectively. In other words, although LD/ADHD may or may not contribute to the effect a calculator has on math anxiety, approximately 50% of students in all disability categories reported an increase in anxiety when using the calculator. The participants in Parks' study were middle school students; however, there is no reason to think that math anxiety would decrease with time away from school, especially if school was never successfully completed.
Ellington (2003) conducted a meta-analysis of the effects of calculator use on both test performance and attitude. Students' attitudes improved most consistently when calculators were used both in class and on tests; the effect was often insignificant when calculators were only used in class. Ellington claims that some studies looked at calculator use only on tests, and further research is needed in this area.
Bridgeman, Harvey, and Braswell (1995) studied the effects of calculator use on a test of mathematical reasoning. The test used was the pilot SAT-I Mathematics section new for 1995. They found calculators to have a significant positive effect for students who were already accustomed to using calculators during classroom testing sessions, but only a marginal improvement for students who were not. This may have interesting implications since most GED students at the facility do not take classroom tests. They state:
Candidates who do not know when and how to use a calculator may be at a relative disadvantage on a test that permits calculator use, and if such candidates are distributed disproportionately across gender or ethnic groups, differences involving these groups might also be exaggerated. (p. 324)
Hearn and Lloyd (1987), as cited in Bridgeman et al. (1995), performed an analysis of GED math items and found that calculators only provide an advantage on computation-intensive items.
Kemp, Kissane, and Bradley (1996b) offer a typology for math test items related to graphing calculators. While the Casio f x - 260S OLAR is not a graphing calculator, certain item types apply to scientific calculators as well.. The first three types belong to the category of "Graphics calculators are expected to be used," the next to "Graphics calculators are expected to be used by some students but not by others," and the final five to "Graphics calculators are not expected to be used." Note that while Type 8 is "items where the student is expected to extract the mathematics from a situation," i.e. word problems, this is placed in the third category because the graphing capabilities of such a calculator would be of little use to a student encountering this kind of item on a test. A scientific calculator, however, might still be useful. Items on the OPT are mostly of Type 2 ("Alternatives to graphics calculator use are very inefficient"), Type 4 ("Use and non-use of graphics calculators are both suitable"), and Type 9 ("Graphics calculator use is inefficient."). Using Kemp et al. (1996b)'s definitions, items on the OPT may be classified by appropriateness of calculator use.
While there is some disagreement about what constitutes appropriate calculator use (Ellington, 2003), the typology referenced above is the most explicit in terms of individual problem content. Although most problems may be solved without the aid of a calculator, there are some for which it clearly provides an advanage. There are also some problems whose solution may be expedited by a calculator, but which may also be solved by pencil-and-paper or clever mental algorithms.
Walsh and Anderson (2009) conducted a study that compared participants' use of mental arithmetic versus a calculator to solve multiplication problems. They found that people's problem-solving strategies change in response to the utility these strategies offer: participants generally used the strategy that would result in the fastest correct answer. Their research involved tracking users' mouse movements on a computer-administered test, so the calculator was of necessity a representation on a screen rather than handheld. It stands to reason that a handheld calculator would be faster than a mouse-driven calculator on a computer display, so if anything, the effectiveness of the calculator should increase with a handheld model.
Ruthven (1998) studied the relationship between calculators and mental arithmetic among British schoolchildren. Some of the children had been taught using a "calculator-aware" curriculum that encouraged their use of mental arithmetic, while others had been taught with a more traditional pencil-and-paper curriculum. This curriculum treated the calculator as another classroom tool, and focused more on conceptual investigations than on acquisition of skills. While both groups were exposed to calculators, the first group were encouraged to make their own decisions regarding appropriate use. Ruthven found that high-scoring students who used calculators on test items did so when mental methods did not quickly yield satisfying results, whereas the two lowest-scoring students, who used the calculator on every problem, waited much longer before choosing that strategy. Interestingly, the students who favored calculator use over mental or written strategies came from the second group, without the calculator-aware curriculum.
Ruthven also noted the number of steps students took to solve each problem and found that while mental methods required students to break problems (such as a two-digit multiplication problem) into smaller steps, the students always performed a single operation when using the calculator or written algorithms. This is consistent with the findings of Hope and Sherrill (1987), who studied the mental arithmetic strategies of very skilled and very unskilled high-school students. Those strongest in mental arithmetic would often use strategies closer to Nunes et al.'s street mathematics, while the least skilled students used a mental analogue of paper-and-pencil algorithms.
In their 1993 book Street Mathematics and School Mathematics, Nunes, Carraher, and Schliemann define "street mathematics" as the mathematics that is practiced outside of school. They make the distinction between street mathematics and "informal arithmetic," pointing out that the latter is not well defined. Part of their characterization of street mathematics is that it is practiced in context, for example while tending a store. They therefore also differentiate between street mathematics and oral arithmetic, which is not bound to any context.
Nunes et al. conducted a study with Brazilian schoolchildren that compared the children's different problem-solving styles. They found that the children most often used oral rather than written arithmetic, and that this strategy was very successful independent of context. The study consisted of three problem types: Contextual simulations involving arithmetic (e.g. pretending to be a customer at a store the child tended), word problems, and written computation exercises. The exercises were presented to the child on paper but the child was allowed to use any method to solve them. While oral arithmetic was much more frequently used to solve contextual simulations and word problems, the success rate was equally high when used with written computation problems. Students also showed a marked increase in the use of oral arithmetic when solving written division problems, suggesting that perhaps the division algorithm learned in school was not as useful to them as their own oral methods (Nunes et al., 1993).
Adult prison inmates have done much of their mathematics outside of school. At the facility, the most common last grade an inmate has completed is 10th, and the average inmate is 28 years old. (Gaudet, 2009) While the GED does not represent an actual work-related situation for test-takers, it does involve many word problems and perhaps some written computation. The findings of Nunes et al. are significant because they show that many people can be quite successful at everyday mathematics without using what is traditionally taught in school.
About the Test
The education department has seven forms of the OPT: PA, PB, PC, PD, PE, PF, and PG. To ensure that the department's pretest is as accurate a predictor of actual GED scores as possible, the department currently uses the three most recent forms (PE, PF, and PG).
The chart in Figure 1 compares the percentage of correct answers on forms PE, PF, and PG against the percentage of correct answers on all forms combined. These data were collected at the institution between April 2006 and January 2010.
[FIGURE 1 OMITTED]
The three forms used in this study (N = 437) have a Pearson correlation of 0.96 with the means of the total historical population (N = 970). Thus, the three forms used provide a highly generalizable sample for any OPT. The overall average score for Part I is 0.395, versus an average of 0.444 on Part II. For the three forms used in this study, the averages are 0.417 for Part I and 0.5 for Part II.
The GED and OPT math questions are categorized according to the four strands of the NCTM: Number Operations and Number Sense; Data, Statistics, and Probability; Measurement and Geometry; and Algebra, Functions, and Patterns (NCTM, 1989). Most questions are multiple choice with five options; 10 of the questions on the OPT are open-response, using a 5-digit grid for students to enter numerical answers. One item (question 22) uses a coordinate plane grid where the student fills in one bubble representing a location with both an x- and a y-coordinate. Many of the questions do not ask for a numerical answer, but rather an equation or expression that could be used to solve the problem or model the problem situation.
It is possible that most students are around question 10 when time is called for Part I and calculators collected, and that this disruption accounts for the sudden dip in scores around questions 9, 10, and 11. It could also be that most students simply run out of time at this point and have to guess at questions after that. However, the scores show a slight improvement on questions 12 and 13, implying that this is not the case.
Questions 18 and 22 are also frequently answered incorrectly. Question 22 is the only one that uses the alternate-format coordinate grid on the answer sheet; many students do not understand how to fill in the grid correctly, and this likely accounts for many errors. Question 18 also uses an alternate response form: the standard grid, which allows students to fill in up to five digits, with the option to use a decimal point or a fraction. Questions 3, 4 and 10 also use the standard grid.
Although the typology by Kemp et al. (1996b) refers to graphing calculators, many of the types apply to scientific calculators as well. In the example they give for Type 2 ("alternatives to [...] calculator use are very inefficient"), they explain that students may be "expected to realise for themselves that solution methods that [do] not involve calculator use are quite time-consuming."(p. 5) Such items certainly exist on the GED, for instance when students must sum two columns of three-digit numbers and subtract to compare the results. Thus computation-intensive items, those which Hearn and Lloyd (1987) claimed were made easier by the availability of a calculator, were assigned Type 2 in this study.
Likewise, there are some problems which may be solved equally well by a student using a calculator or one skilled at mental arithmetic. These problems often involved percentages such as 10%, 20%, or 50%, simple fractions, or whole amounts. Items like these were assigned Type 4 ("use and non-use of [...] calculator are both suitable"). Questions which did not require calculations, or which involved only single-digit or round numbers, were assigned Type 9 ("[...] calculator use is inefficient").
For comparison, items which required the student to read data from a graph were considered to be of Type 10 ("Task requires that a representation of a graphics calculator screen will be interpreted"). While the graphs on the OPT are not pictures of a graphing calculator screen, they are two-dimensional coordinate representations of data, and these items often do not require calculations. Items which required students to choose an expression or equation rather than compute an answer were marked as Type 6 ("Symbolic answers are required").
Table 1 shows the distribution of item type for each of the forms used.
Table 1. Distribution of Item Type--Part I Item# 1 2 3 4 5 6 7 8 9 10 11 12 13 Type(PE) 9 2 2 4 2 2 2 2 9 4 2 4 4 Type(PF) 2 4 2 4 4 2 4 4 9 4 2 4 2 Type(PG) 4 2 2 9 4 9 4 10 4 9 4 2 9
Note that very few of the questions on Part I for any form are calculatorneutral. Most items have a type below 6, indicating that calculator use is either advantageous or at least reasonable. Compare this with Part II (table 2), where most questions have type 6 or above. Although there are a few questions which could be answered with or without a calculator (Type 4), only one question on one form has Type 2, i.e. would be better solved with a calculator. Clearly the OPT is designed to take advantage of the availability of a calculator when given.
Table 2. Distribution of Item Type--Part II Item # 14 15 16 17 18 19 20 21 22 23 24 25 Type(PE) 2 6 4 9 9 4 9 10 10 10 9 9 Type(PF) 10 9 9 9 4 9 4 10 10 6 6 9 Type(PG) 10 10 6 4 9 9 9 4 10 9 4 10
The tests are scored on an interval scale much like the SAT--scores on each subtest range from 200 for no right answers to 800 for a perfect score. The minimum passing score for each subtest is 410, however the average score across five subtests must be at least 450. It is worth noting here that the test scores are not linear; that is, each question is not necessarily worth the same number of points. Forms PF and PG require only 11 correct answers out of 25 for a passing score, whereas form PE requires 16 correct answers. Also, the points awarded for each question are fewer as the number of correct responses increases. The American Council on Education weights each item according to item difficulty and adjusts scores accordingly. Theoretically, the same student who answers 11 questions correctly on form PF should be able to answer 16 correctly on PE. Therefore, consideration must be given when rating the skills of students taking different forms of the test.
The study consisted of interviewing 13 students after an OPT. Students were offered participation after testing was complete so that awareness of the interview process did not generate additional test anxiety. Also, in this way students did not worry that the interview and debriefing would affect their eligibility to take the GED. I conducted the interviews within one week of testing so as to ensure the greatest amount of recall. Both the GED and the OPT take three days of testing, and with the department's scanner, scores cannot be processed until all subtests are complete. Although the mathematics subtest is on the second day, students were not interviewed until after tests had been scored on the third day. This restriction enabled me to inform students of their test scores during interviews.
The interview instrument was a single-page form that I filled out based on students' responses. The form provided space for recording the strategy the student used (Mental, Written, and/or Calculator). If a student first attempted to answer the question mentally, then switched to the calculator, both "M" and "C" were circled. Participants were then asked to describe their strategy for answering each test item, and whether they had used the calculator.
Participants were randomly selected from a group of 73 students taking a single department pretest. Seven men and six women were interviewed in all. Students' scores ranged from 320 to 450. Five of the students had taken form PE, three had taken form PF, and five had taken form PG. Not all students who were offered participation chose to take part in the study. It may be that exceptionally low-performing students (or those with exceptionally low self-concept) did not participate due to the fear of increased frustration. Also, those with exceptionally high scores may not have felt it necessary to improve the effectiveness of their problem-solving strategies by interviewing and debriefing. The result is that the participants represent a cross-section of the middle range of test-takers, with outliers omitted.
The qualitative component of the study also revealed important information. Many problems that I thought "needed" the calculator were solved in surprising and creative ways by the students, and many students gave unique insights into their feelings about the availability of the calculator on the test. They also frequently offered their preferences for problem-solving in general.
Figure 2 shows a comparison of this sample's percentage of correct responses with the percentage of correct responses of the historical population. In both cases only forms PE, PF, and PG are scored. These data have a Pearson correlation of 0.82, and a Student's t-test revealed a p-value of 0.04. While the correlation between the experimental group and the historical population is not as strong as that between the historical data on the three forms used here and the seven forms available, it is certainly strong enough to imply that this sample is representative of the group as a whole.
[FIGURE 2 OMITTED]
Table 3 contains an overview of each student's problem-solving strategy distribution. A cascading hierarchy was used to tag each response as using (C)alculator, (W)ritten, or (M)ental strategies. If a student claimed to have used the calculator, the item was marked with a (C), even if other strategies were used. Next in the hierarchy is (W), followed by (M). So, if a student used a combination of mental math and scrap paper on a particular item, the item was marked (W).
Table 3. Problem-solving Strategy Distribution Form: PE PF PG ID# 1 2 3 12 13 5 9 10 11 4 6 7 8 C 3 3 3 3 3 0 1 0 3 1 3 2 0 W 3 4 2 2 9 11 3 6 8 7 3 1 4 M 9 5 1 19 10 6 20 13 14 17 7 4 13
As mentioned before, only three students did not use the calculator at all. Among those who did, however, it was used for a maximum of three questions out of the 13 possible. Most students who used the calculator used it for three questions, while one used it for two and two used it for one.
In table 4, we see the number of times students used the calculator on questions for which it was expected to be used. These totalled 16 instances of use out of 65 possible, or about
27%. 12 of these instances were on form PE, and indeed there are more Type 2 questions on that form than on any other.
Table 4. Calculators Are Expected to be Used PE Item # 2 3 6 7 8 11 PE uses # Uses 1 1 2 3 3 2 12 PF Item # 1 3 6 11 13 PF uses # Uses 0 0 0 0 0 0 PG Item # 2 3 12 PG uses Total # Uses 0 3 1 4 16
Table 5 shows how often students used the calculator when calculator use and non-use were both reasonable strategies. While there were collectively 71 opportunities to use the calculator for these types of questions, students used it 9 times, or about 13% of the time.
Table 5. Calculators Are Expected to be Used By Some Students and Not By Others PE Item # 4 5 10 12 13 PE uses # Uses 1 1 0 1 1 4 PF Item # 2 4 5 7 8 10 12 PF uses # Uses 0 1 0 0 0 0 0 1 PG Item # 1 5 7 9 11 PG uses Total # Uses 0 2 2 0 0 4 9
It is clear that the calculator does not provide much advantage on the test. In the next table (6) we see the number of correct (C)alculator, (W)ritten, and (M)ental responses for each student. Student 12 answered three questions correctly when using the calculator; all others got one or zero.
Table 6. Distribution of Accuracy of Strategies Form: PE PF PG ID# 1 2 3 12 13 5 9 10 11 4 6 7 8 C correct 1 1 1 3 1 0 1 0 1 0 0 0 0 W correct 3 2 1 2 6 6 0 3 5 4 1 0 2 M correct 7 2 1 9 4 4 8 7 8 8 3 3 6
Table 7 compares the accuracies of each student's strategies as a ratio of the number of correct responses to total responses using that strategy. Seven students performed better when using written strategies than mental. The number of items on which students used written strategies was much lower, and the average accuracy for these items was 52%, compared to 56% for mental. The average calculator accuracy, however, was only 37%.
Table 7. Comparison of Strategy Accuracy Form: PE PF PG ID# 1 2 3 12 13 5 9 10 11 4 6 C .33 .33 .33 1 .33 - 1 - .33 0 0 W 1 .5 .5 1 .67 .55 0 .5 .63 .57 .33 M .78 .4 1 .47 .4 .67 .4 .54 .57 .47 .43 Form: ID# 7 8 C 0 - W 0 .5 M .75 .46
Although the data are not perfect, it seems that most students perform significantly better when using mental math or scrap paper than when using the calculator. Only student
12 and student 9 had better than 33% accuracy when using the calculator, and student 9 only used the calculator for one item (which she answered correctly).
Among those who disliked using the calculator, reasons were varied. Some responses included:
"I've never been good with calculators."
"The calculator just slows me down."
"I'd rather just do it out. When I use the calculator, I just have to go back and check it anyway."
"I just do it on my fingers."
"I want to test myself, you know, so I didn't use it." "When I'm in that test, I go back to doing it the old way." Students 5, 6, 10, and 13 all stated explicitly that they prefer to solve problems on paper over using a calculator.
Student 11 stated that he wanted to know more about the calculator. He said that because he did not understand the memory function, he found it harder to keep track of intermediate results. Problems that required keeping track of three or four numbers became too complicated to compute without using paper. This student recorded intermediate results on the scrap paper for at least 11 questions, although three of them were finally solved using the calculator.
One student, during his interview, explained how he evaluated a linear equation at a particular value by using the description of the problem rather than the formula (in the form y = b + mx) provided. Another performed a series of five additions mentally using the fact that all numbers were monetary amounts in denominations of either five or ten cents. Finally, a third student solved a Pythagorean theorem question by using the edge of his test booklet as a ruler. He marked off (with his finger) the length of the hypotenuse and used this length to make an accurate estimate of the length of a leg.
The goal of this study was to determine whether inmates at this facility use the calculator appropriately in testing situations. The NCTM claims that appropriate calculator use is essential to modern mathematics education, yet does not define it. Ellington (2003) states that there is some disagreement about what is appropriate use of the calculator. The typology created by Kemp et al. (1996b) outlines certain types of problems on which calculator use would be appropriate. This study looks at inmate calculator use on these types of questions, but also takes into account the theories of Walsh and Anderson (2009), that "appropriate" problems for calculator use would be those that the student could solve better (faster or more accurately) with the calculator than with mental math.
The most striking result of this study is the limited extent to which these students actually use the calculator. Assuming all students finished Part I in the time allotted, 13 students each had 13 questions on which the calculator could be used. Of those 169 possible uses, students only used the calculator
26 times, or on an average of about 15% of the items. The ACE (2009) mentions the ability to use a scientific calculator as part of the measurement of the Number Operations and Number Sense content area. According to the ACE, this content area accounts for 20 to 30% of the mathematics subject test; perhaps testers are not expected to use the calculator significantly more often than these students did.
When the students in this study did use the calculator, they used it appropriately. 16 of the 26 instances of calculator use occurred for Type 2 questions ("Alternatives to [...] calculator use are very inefficient"). Nine of the instances occurred for Type 4 questions ("Use and non-use of [...] calculators are both suitable"). Only one student used the calculator on a Type 9 question ("[...] calculator use is inefficient"). The students preferred not to use the calculator when the questions did not demand it; they were only about half as likely to use it for Type 4 questions as for Type 2. Based on the findings of Nunes et al. (1993), it is likely that these long-time practitioners of "street mathematics" are better served by their own (often mental) algorithms. Walsh and Anderson (2009) indicate that such students would be unlikely to use the calculator, except as a last resort, as it would not offer more utility than their own algorithms.
Equally interesting are the accuracies of the students' calculator use. None except student 12 answered more than one question correctly when using the calculator. While the calculator clearly does not help these students in any significant way, it is difficult to say whether it does harm. Bing et al. (2009, p. 327) found that when using a calculator on tests, "[s]ome items become easier, perhaps because of a reduction in computational errors, whereas other items become more difficult, presumably because of inappropriate use of the calculator." The students in this study prevented a minimal number of computation errors by using the calculator, however, they did not use the calculator inappropriately. Calculatorneutral items should therefore not have become more difficult. The issue remains whether calculator use negatively impacts incarcerated students' performance.
The scientific calculator offers the advantage that it evaluates expressions according to the order of operations rules, as opposed to evaluating them as entered (which a four-function calculator does). In theory, this makes more complicated expressions easier to manage. As student 11 pointed out, however, this is not always the case. Keeping track of more than a few numbers becomes cumbersome unless the user is fluent with the memory operations of the calculator. Also, if the user is not familiar with the order of operations, additional errors may be introduced. Students may, for example, evaluate expressions of the form a+b as a + b with the calculator because they expect it to behave like a four-function machine.
Most students show a great preference for solving problems mentally; this is consistent with earlier analysis of OPT scrap paper and the findings of Nunes et al. (1993). Only student 5 preferred the scrap paper to mental math. These data are also not entirely complete; some students may have had trouble remembering what they wrote down and what they did not. Students were able to recall calculator use, however.
A common theme in the interviews was that of keeping track. Some students stated that they preferred not to use the calculator because it was hard to keep track of intermediate results; others used paper to augment their mental arithmetic. Student 11, for example, used the paper to record intermediate results more than to do calculations using paper-and-pencil algorithms. Hope and Sherrill (1987) found similar practice among the most skilled mental calculators in their study. The authors also mention that the students who were least accurate in mental math often attempted to use the paper-and-pencil algorithms, such as borrowing and carrying, in their minds. Student 8 did this in her interview, though she fell near the middle of the mental accuracy range.
The other common theme was time. For perspective, 25 questions in 45 minutes leaves an average of 1:48 (one minute and 48 seconds) to answer each. Obviously, not all questions will require that much time, but for problems with involved calculations, time becomes a critical factor. Walsh and Anderson (2009) suggest that if a calculator does not save time while maintaining or improving accuracy, students will be less likely to use it. This helps to explain why the inmates in this study use the calculator so infrequently.
It seems that while the prison inmates in this study certainly fit the profile of "street mathematics" practitioners, and often devise innovative algorithms of their own, their performance is not as reliable as that of Nunes' subjects. These inmates cannot rely solely on mental math in order to pass the GED. Their written skills are nearly on par, but they do not use them as often. The calculator, however, offers little discernible help and may even decrease their problem-solving ability. These students do not interact with the calculator in the same way as those who participate in mainstream education; unsurprisingly, the calculator is not currently a helpful accommodation.
The ACE (2009) states that "The skills tested include the ability to: [...] Calculate mentally, with pencil and paper, and with a scientific calculator using whole numbers, fractions, decimals, and integers" (p. 28). Perhaps it can be inferred here that the ability to use a scientific calculator is considered a necessary skill in higher education. The authors go on to say that an adult with foundational skills for the workplace "makes reasonable estimates of arithmetic results without a calculator" (p. 66). It would seem that this accounts for the fact that the test currently has two parts, one that permits the use of a calculator and one that does not.
Mechem (2009) and Hinds (2009) found that GED testers were unlikely to place at college level on mathematics placement tests. Both also found that students in remedial or developmental math classes were far less likely to complete two years of college than were their peers. Mechem also found that the GED credential is no longer sufficient for a sustainable career; at least two years of higher education are needed.
According to Ruthven (1998) and Ralston (1999), mental math abilities are just as important as written proficiency, if not more so. Yet it seems that adult students will be expected to demonstrate proficiency with scientific calculators. Kemp, Kissane, and Bradley (1996a) state that students do not see the value of calculators unless calculators are fully integrated into the curriculum. While Ruthven supports this claim with his comparison of a "calculator-aware" versus traditional curriculum, such a curriculum would be difficult to implement in a short-term facility. Classes at the facility usually have a rolling admissions structure, and the GED testing cycle is only three months long. Additionally, integrating the calculator into every lesson carries the risk of making students dependent on it, which would put them at a disadvantage in a two-part test like the GED.
Still, it appears that the only way to help incarcerated learners take full advantage of the calculator is to integrate it into every lesson. If teachers are aware of inmates' predispositions toward calculator use, inmates will hopefully learn to make good choices about when to use the calculator, as did the students in Ruthven's study. Prison inmates seem primed to take advantage of a "calculator-aware" curriculum, as they, like the best students in Ruthven's study, only use the calculator when mental math fails them. Additionally, if they can learn to trust the calculator's consolidation of intermediate steps, they should be able to overcome their difficulties keeping track of their work. In the end, this could improve their speed.
The "calculator-aware" curriculum described by Ruthven treats the calculator like any classroom resource rather than the focus of a lesson by itself. Classroom activities are investigations into concepts rather than exercises designed to hone arithmetic skills, and the calculator is used to support these investigations. Activities appropriate to the GED might include compound interest, approximating irrational square roots, or scientific notation. Students may begin to see the utility of the calculator when it hastens their pursuit of patterns.
Given the predictions of Bing et al. (2009), Bridgeman et al. (1995), Hearn and Lloyd (1987), and Shockley et al. (1989), and the results of this study, it is clear that not all students use the calculator in the expected way. These studies, as well as those by Nunes et al. (1993) and Parks (2009), provide insight as to why prison inmates may not take full advantage of the calculator. Further research should measure the feasibility and effectiveness of a curriculum such as the one studied by Ruthven, which is designed to leverage the calculator's power in finding patterns and investigating concepts. Such a curriculum would necessarily be different due to the limitations of the correctional setting; however, the principles of exploration and independence would be a fine fit for the inmate learner population.
ACE. (2002). Official GED practice tests administrator's manual [Computer software manual].
ACE. (2009, March). Technical manual: 2002 series GED tests [Computer software manual].
Bing, M.N., Stewart, S.M., & Davison, H.K. (2009). An investigation of calculator use on employment tests of mathematical ability. Educational and Psychological Measurement, 69(2), 322-350.
Bridgeman, B., Harvey, A., & Braswell, J. (1995). Effects of calculator use on scores on a test of mathematical reasoning. Journal of Educational Measurement, 32(4), 323-340.
Ellington, A.J. (2003). A meta-analysis of the effects of calculators on students' achievement and attitude levels in precollege mathematics classes. Journal for Research in Mathematics Education, 34(5), 433-463.
Gaudet, R. (2009). 2007 release data. (Statistical breakdown of male release data for year 2007)
Hearn, D.L., & Lloyd, B.H. (1987). Use of calculators on standardized math tests: Effects on performance and the potential for bias.
Hembree, R., & Dessart, D.J. (1986). Effects of hand-held calculators in precollege mathematics education: A meta-analysis. Journal for Research in Mathematics Education, 17(2), 83-99.
Hinds, S. (2009). More than rules: College transition math teaching for GED graduates at the city university of new york.
Hope, J.A., & Sherrill, J.M. (1987, March). Characteristics of unskilled and skilled mental calculators. Journal for Research in Mathematics Education, 18(2), 98-111.
Hutton, B.M. (1998, July). Should nurses carry calculators? In Adults learning maths-4: Proceedings of alm-4, the fourth international conference of adults learning maths - a research forum held at university of limerick, ireland (pp. 164-172). London: Goldsmiths College, University of London, in association with ALM.
Isiksal, M., & Askar, P. (2005, November). The effect of spreadsheet and dynamic geometry software on the achievement and self-efficacy of 7th-grade students. Educational Research, 47(3), 333-350.
Kemp, M., Kissane, B., & Bradley, J. (1996a). Graphics calculators and assessment.
Kemp, M., Kissane, B., & Bradley, J. (1996b). Graphics calculator use in examinations: accident or deisgn? The Australian Senior Mathematics Journal, 10(1), 36-50.
Mechem, T. (Ed.). (2009). Preliminary findings from the school-to-college database.
NCTM. (1989). Curriculum and evaluation standards for school mathematics. National Council of Teachers of Mathematics.
Nunes, T., Schliemann, A. D., & Carraher, D.W. (1993). Street mathematics and school mathematics. Cambridge University Press.
Parks, M. (2009). Possible effects of calculators on the problem solving abilities and mathematical anxiety of students with learning disabilities or attention deficit hyperactivity disorder. Unpublished doctoral dissertation, Walden University.
Ralston, A. (1999). Let's abolish pencil-and-paper arithmetic. Journal of Computers in Mathematics and Science Teaching, 18(2), 173-194.
Ruthven, K. (1998). The use of mental, written and calculator strategies of numerical computation by upper primary pupils within a 'calculator-aware' number curriculum. British Educational Research Journal, 24(1), 21-42.
Shockley, J., McGurn, W., Gunning, C., Graveley, E., & Tillotson, D. (1989). Effects of calculator use on arithmetic and conceptual skills of nursing students. Journal of Nursing Education, 28(9), 402-405.
Walsh, M.M., & Anderson, J.R. (2009, May). The strategic nature of changing your mind. Cognitive Psychology, 58(3), 416-440.
NATHANIEL STAHL taught GED math in a correctional setting for more than five years. He has a Master of Liberal Arts in Mathematics for Teaching from Harvard University Extension School, and is currently studying Mathematics Education at Teachers' College, Columbia University. Harvard University
|Printer friendly Cite/link Email Feedback|
|Publication:||Journal of Correctional Education|
|Date:||Nov 1, 2011|
|Previous Article:||Using self-determination theory in correctional education program development.|
|Next Article:||CEA recognizes award recipients.|