The AP descriptive chemistry question: student errors.
For over a decade, the authors have been involved in a design theory experiment providing software for high school students preparing for the descriptive question on the Advanced Placement (AP) chemistry examination. Since 1997, the software has been available as a Web site offering repeatable practice. This study describes a 4-year project during which incorrect responses using the most recent interface were collected and saved in a dataset. A descriptive analysis suggests that the most basic errors include: failing to write appropriate chemical formulas, recognize reactive species in net ionic reactions, and recognize weak electrolytes. The largest single group of reported errors, recognizing weak electrolytes, implies strategies for improved student performance on the AP examination. In addition, the results of this study support the development of greatly enhanced feedback for future learner who use the site.
For virtually every year since its inception, the advanced placement (AP) chemistry examination has included an item generally called the "descriptive chemistry question." An example of such a question would be a statement such as:
Methane is burned in an excess of oxygen
An accepted answer from the student is:
C[H.sub.4] + [O.sub.2] --> C[O.sub.2] + [H.sub.2]O
The answer accepted by the graders of the AP exam is a net ionic i·on·ic
Of, containing, or involving an ion or ions.
pertaining to an ion or ions.
iontophoresis. representation that is not necessarily balanced. Each student is given eight such statements and expected to select and respond to five. Traditionally, each item is scored using a three-point scale where one point is earned for providing the formula for the correct reactant reactant /re·ac·tant/ (re-ak´tant) a substance entering into a chemical reaction.
n. (s) and two points are earned for the correct product(s).
Since the early 1990s, our research has focused on software development aimed at increasing knowledge of descriptive chemistry and enhancing student performance on the AP chemistry examination. Our current software is available on a Web site for both teachers and students; for students it provides an opportunity for repetitive practice.
This study analyzed 4 years of student errors collected from the Web site using the most current interface and saved in a dataset. Some errors are typographical ty·pog·ra·phy
n. pl. ty·pog·ra·phies
a. The art and technique of printing with movable type.
b. The composition of printed material from movable type.
2. ; these appear rarely--at most a few times out of the thousands of incorrect responses measured. Patterns of student errors may reflect deeply held misconceptions Misconceptions is an American sitcom television series for The WB Network for the 2005-2006 season that never aired. It features Jane Leeves, formerly of Frasier, and French Stewart, formerly of 3rd Rock From the Sun. . In addition to our analysis, teachers are provided with access to our database of errors so that they can use it to focus their instruction.
Our research with the AP descriptive chemistry Web site is grounded in literature concerning the use of Web-based repetitive practice for teaching and learning. Although use of the Web for teaching/learning is becoming very common, empirical support for specific strategies is lacking. Much is known about the effectiveness of repetitive practice with feedback, however.
Using the Web as a teaching tool appears to be as effective as traditional classroom teaching. In experimental studies, Web-based instruction consistently produces no significant difference in test scores when these scores are compared to traditional test scores (Wegner, Holloway, & Garton, 1999). Web instruction does seem to impact student impressions of the course in positive ways, however (Brown, 1995; Wegner et al., 1999).
By nature, computers are ideal for providing students with practice scenarios and problems. Computers can provide problems, evaluate responses to those problems, and provide immediate feedback. Providing appropriate feedback is known to increase the rate of learning (Lhyle & Kulhavy, 1987). In fact, effective practice must occur with feedback (Bransford, Brown, & Cocking, 1999). Feedback that explains how answers are determined and provides further instruction is most effective (Pressley & McCormick, 1995; Renkl, 1998). Engaging in practice and receiving feedback makes for better learning by promoting self-regulation (Butler & Winne, 1995). Pintrich describes self-regulated learning The term self-regulated can be used to describe learning that is guided by metacognition, strategic action (planning, monitoring, and evaluating personal progress against a standard), and motivation to learn as "involving the active, goal-directed, self-control of behavior, motivation and cognition cognition
Act or process of knowing. Cognition includes every mental process that may be described as an experience of knowing (including perceiving, recognizing, conceiving, and reasoning), as distinguished from an experience of feeling or of willing. for academic tasks by individual students" (Pintrich, 1995).
While practice typically implies homework or some type of classroom work, testing also can serve as practice. Practice tests and quizzes not only prepare students for the type and variety of questions on a graded exam, they also can serve as a teaching and learning tool for course content (Foos & Fisher, 1988).
The Web facilitates the delivery of practice and supports distance learning. We conjecture CONJECTURE. Conjectures are ideas or notions founded on probabilities without any demonstration of their truth. Mascardus has defined conjecture: "rationable vestigium latentis veritatis, unde nascitur opinio sapientis;" or a slight degree of credence arising from evidence too weak or too that the success of Web practice as a learning tool hinges on providing effective feedback (Brooks, Schraw, & Crippen, 2005). The nature of the feedback ensures that students not only identify what they do not know, but have the opportunity to reflect and acquire missing knowledge.
Using the Web to grade homework is a very practical application of the technology. Online homework systems have been shown to have a positive effect on test scores for both physics and chemistry students (Dufresne, Mestre, Hart, & Rath rath (rä, räth), circular hill fort protected by earthworks, used by the ancient Irish in the pre-Christian era as a retreat in time of danger. , 2002; Penn, Nedeff, & Gozdzik, 2000). This is especially so when the exam questions require the use of memorized concepts or the use of simple problem-solving algorithms (Paull, Jacob, & Herrick, 1999).
Parallel to Web-based homework systems are Web-based practice systems where students can take practice quizzes on their own volition vo·li·tion
1. The act or an instance of making a conscious choice or decision.
2. A conscious choice or decision.
3. The power or faculty of choosing; the will. . While these practice exams do not affect a student's grade, early research suggests that students who utilize such practice score higher on actual exams than do students who use only traditional study methods (Hall, Pilant, & Strader, 1999). Charlesworth (2000) reports a direct correlation Noun 1. direct correlation - a correlation in which large values of one variable are associated with large values of the other and small with small; the correlation coefficient is between 0 and +1
positive correlation between online practice quiz scores and traditional exam scores for general chemistry students.
Our research on supporting the AP descriptive chemistry question has focused on software development using an instructional design Instructional design is the practice of arranging media (communication technology) and content to help learners and teachers transfer knowledge most effectively. The process consists broadly of determining the current state of learner understanding, defining the end goal of perspective. We use a design theory methodology that emphasizes an ongoing repetitive cycle of design, testing, data collection, and analysis (Cobb, Confrey, diSessa, Lehrer, & Schauble, 2003). Consistent with design theory, our analysis and development cycles rely heavily on retrospective analysis.
A decade ago, we developed a desktop software tool for teachers that allowed them to generate sets of eight items that would be quite reminiscent of the AP question. That is, items were grouped and selections were made using random choices from within those groups such that the distribution of items reflected a typical AP question. Teachers could print examples of eight-item questions and the related answer keys.
Since 1997, our original software has been available as a Web site offering repetitive practice. Annually, students and teachers use this system extensively (Crippen, Brooks, & Abuloum, 2000). Prior to 2000, the site offered the same functionality as the desktop version of the software; upon request, it randomly built eight-item quizzes that were similar in format to the AP exam. Using results from early studies, we produced a major revision of the Web site in 2000 (Brooks & Crippen, 2001; Crippen & Brooks, 2001, 2002, 2004). The current interface permits students either to engage in AP exam practice (answering eight items distributed from among the groups) or to focus on learning the content by practicing with individual items (responding to one item at a time until correct).
In addition to building and grading quizzes and providing feedback to students, the most recent version of our practice Web site has been capturing student responses for each item attempt. Theoretically, the responses were chemical formulas for both products and reactants for several categories of chemical reactions This is the 18th episode of television drama Men in Trees. It originally aired on June 25, 2007 on the TV2 network in New Zealand as a continuation of season 1. Recap
Marin and Cash have a stew cook off, she admits his is better than hers. .
Descriptive Statistics descriptive statistics
see statistics. : During the period January 9, 2001 through November 12, 2004, 5,304 users created records. Of these, 3,446 (65%) accepted the invitation to permit study of their performance (University of Nebraska - Lincoln IRB IRB
See: Industrial Revenue Bond #99-11-068FB). The self-identification of these users is displayed in Table 1.
To develop forensics See computer forensics. related to potential hacking, each interaction was noted whether or not the user accepted our terms. The average number of discrete interactions with the Web site of those accepting participation in the study was 52, while that of those declining participation was 45. We have no information about those declining participation in the study other than that they logged in.
The data for AP high school students and teachers was examined in more detail. Some self-descriptions of the AP students appear in Table 2.
A most striking outcome of these reports is that so few of the students are 19 or over: almost 1,700 students (90%) report themselves as being members of a 'protected' group for such studies. While more students used the site from home than from school, the school-based usage is reported as a part of classroom work. Of the top 1,000 users in the student group, the average number of uses was 132. Of the top 100 users, that number was 525. The top 10 AP student-users made an average of 1,110 uses of the site, a very high level of practice.
Supported by previous results (Brooks & Crippen, 2001), our latest design sought to encourage practice on individual items instead of taking a typical eight-item AP questions. Of the 1,895 students, only 402 (21%) ever requested a full set of eight items. While one user received 322 eight-item questions; the next largest number was 95 and only 6 users took more than 50. When ordered by the number of items received, the 100th user listed took 6 full AP questions. Thus, there was dramatic use of the item-by-item practice strategy and seemingly less emphasis on holistic practice for the AP examination.
Student use of computer help systems is not supported empirically (Alevin alevin
advanced fry. , Stahl, Schworm, Fischer & Wallace, 2003). We interpret these findings as an issue of design. The results provided here contradict the trend and show that students do use our help system. In our system, 998 AP student users (53%) did use tutoring at least once. Some 567 AP students (30%) devoted at least 5% of their user interactions to accessing the tutors.
In support of using our materials in a traditional classroom setting, 473 self-reported AP teachers used the site with an average of 26 uses each. Of these, 307 teachers (65%) reported using the site at school.
Incorrect Responses: In interacting with the site, the 5,304 users produced over 135,000 incorrect responses in responding to 227 items. When creating AP-like test questions consisting of eight items, a sampling procedure was used such that randomized ran·dom·ize
tr.v. ran·dom·ized, ran·dom·iz·ing, ran·dom·iz·es
To make random in arrangement, especially in order to control the variables in an experiment. items were selected from within groups, making each selection typical of eight-item AP questions. For practice, however, the items were sequenced in groups and each user was brought through the group sequence one item at a time. Because users made such extensive use of practice items, items that appeared early in the group's queue received a disproportionate number of responses than those coming later. It is inappropriate, therefore, to try to analyze for the 'most missed' item or the 'most difficult item from the combustion group.'
An incorrect formula for any combination of reactant or product for a given reaction would identify a submission as an incorrect response. These incorrect response submissions were extracted and disaggregated Broken up into parts. by reaction type. The error submissions were then analyzed descriptively by counting occurrences and normalizing using a percentage. Using an emergent set of categories, the authors then applied their combined experience as chemistry teachers to diagnose potential sources for the most common errors. Selected results of user inputs are shown in Table 3.
It would come as no surprise to experienced teachers that some error patterns occur frequently.
Students write incorrect formulas. This occurs frequently because they have not learned charges of ionic species. For the item, "50.0 mL of 0.100 M sodium carbonate sodium carbonate, chemical compound, Na2CO3, soluble in water and very slightly soluble in alcohol. Pure sodium carbonate is a white, odorless powder that absorbs moisture from the air, has an alkaline taste, and forms a strongly alkaline water is added to 50.0 mL of 0.100 M hydrochloric acid hydrochloric acid: see hydrogen chloride.
or muriatic acid
Solution in water of hydrogen chloride (HCl), a gaseous inorganic compound. ," student responses of NaC[O.sub.3] and C[O.sub.3] together accounted for 11% of the errors. For the reaction, "solid lithium oxide lithium oxide
A strongly alkaline white powder, Li2O, used in ceramics and glass. Also called lithia. is added to distilled water Noun 1. distilled water - water that has been purified by distillation
H2O, water - binary compound that occurs at room temperature as a clear colorless odorless tasteless liquid; freezes into ice below 0 degrees centigrade and boils above 100 degrees centigrade; ," student failure to learn charges, as reflected by writing LiO as a reactant, was very common (nearly 20% of errors).
Students fail to invoke dissociation dissociation, in chemistry, separation of a substance into atoms or ions. Thermal dissociation occurs at high temperatures. For example, hydrogen molecules (H2 when they should and do invoke dissociation when they should not. For the item, "50.0 mL of 0.100 M sodium carbonate is added to 50.0 mL of 0.100 M hydrochloric acid," responses such as N[a.sub.2]C[O.sub.3] and NaCl accounted for 12% of the errors. For the reaction, "the gases ammonia and hydrogen chloride hydrogen chloride, chemical compound, HCl, a colorless, poisonous gas with an unpleasant, acrid odor. It is very soluble in water and readily soluble in alcohol and ether. It fumes in moist air. It is not flammable, and the liquid is a poor conductor of electricity. are mixed," the most frequent errors involved writing product ions such as N[H.sub.4.sup.+] instead of N[H.sub.4]Cl (27%).
Students do not recognize weak electrolytes, especially weak acids. This is a common error. For the item "aqueous aqueous /aque·ous/ (a´kwe-us)
1. watery; prepared with water.
2. see under humor.
adj. ammonia is added to hydrofluoric acid hydrofluoric acid /hy·dro·flu·o·ric ac·id/ (-floor´ik) a gaseous haloid acid, HF, extremely poisonous and corrosive.
n a compound consisting of hydrogen and flourine. ," the most frequent error (21%) involved writing [H.sup.+] as a reactant, implying failure to recognize that HF is a weak electrolyte electrolyte (ĭlĕk`trəlīt'), electrical conductor in which current is carried by ions rather than by free electrons (as in a metal). . For the item, "aqueous potassium dichromate potassium dichromate
A bright yellowish-red crystalline compound, K2Cr2O7, used as an oxidizing agent, and in pyrotechnics, explosives, and safety matches.
Noun 1. is added to acidified acidified /acid·i·fied/ (ah-sid´i-fid) having been made acid. aqueous sodium sulfite sodium sulfite
A white crystalline or powdered compound, Na2SO3, used in preserving foods, silvering mirrors, developing photographs, and making dyes. ," the high frequency of S[O.sub.3.sup.2-] as an erroneous response (12%) suggests failure to identify a weak acid (HS[O.sub.3.sup.-]).
Including spectator ions spectator ion
Chem an ion which is present in a mixture but plays no part in a reaction (those not involve in the net ionic reaction) also is common. For the item, "chlorine gas is bubbled into cold, dilute sodium hydroxide sodium hydroxide, chemical compound, NaOH, a white crystalline substance that readily absorbs carbon dioxide and moisture from the air. It is very soluble in water, alcohol, and glycerin. It is a caustic and a strong base (see acids and bases). ," the species N[a.sup.+] was used as a reactant in 9% of reported errors.
Misconceptions are obvious. Common errors include: Writing CuO instead of Cu as a product for the item, "solid copper(II) sulfide is heated strongly in oxygen gas" (28%); selecting Ni(N[H.sub.3][).sub.4.sup.2+] instead of Ni(N[H.sub.3][).sub.6.sup.2+] for the item, "excess 15 M ammonia is added to aqueous nickel(II) sulfate sulfate, chemical compound containing the sulfate (SO4) radical. Sulfates are salts or esters of sulfuric acid, H2SO4, formed by replacing one or both of the hydrogens with a metal (e.g., sodium) or a radical (e.g., ammonium or ethyl). " (24%); and writing SiF instead of Si[F.sub.4] as a product in "hydrogen fluoride hydrogen fluoride, chemical compound, HF, a colorless, fuming liquid or colorless gas that boils at 19.54°C;. It is miscible with water and is soluble in benzene, toluene, and concentrated sulfuric acid. gas is passed over moist silicon dioxide silicon dioxide: see silica.
(SiO2) A hard, glassy mineral found in such materials as rock, quartz, sand and opal. In MOS chip fabrication, it is used to create the insulation layer between the metal gates of the top layer and the silicon elements below. " (15%).
Certain typographical errors were predictable and commonplace. H20 (h-two-zero) was typed instead of H2O (h-two-oh) as a product in 28% of the errors for "aqueous hydrogen peroxide hydrogen peroxide, chemical compound, H2O2, a colorless, syrupy liquid that is a strong oxidizing agent and, in water solution, a weak acid. It is miscible with cold water and is soluble in alcohol and ether. is decomposed de·com·pose
v. de·com·posed, de·com·pos·ing, de·com·pos·es
1. To separate into components or basic elements.
2. To cause to rot.
1. using a catalyst." Errors of this type would not be problematic in the essay portion of the AP test as evaluated by readers.
Perhaps it is not surprising that the most frequent errors produced at the Web site supporting the AP descriptive chemistry question revolve around Verb 1. revolve around - center upon; "Her entire attention centered on her children"; "Our day revolved around our work"
center, center on, concentrate on, focus on, revolve about the most fundamental issues involved in writing net ionic equations in chemistry. In spite of the fact that a large portion of the errors would not be surprising to general chemistry teachers, this data has once again provided the opportunity to redesign the site such that very specific feedback can be developed on an item-by-item basis. We have been encouraged by the fact the users do seek tutoring. This suggests that redesign of the tutoring section might enhance learning. Our next design revision will include an error analysis algorithm that will detect known errors and provide chemistry-specific feedback. Therefore, when a user writes [H.sup.+] and [F.sup.-] instead of HF for hydrofluoric acid, that student will receive specific feedback about HF being a weak acid.
Interested parties (teachers) may contact the authors to receive a password that will permit them access to an item-by-item error analysis directly from the database. The problem list is at: http://dwb3.unl.edu/Courses/chem/Desc/test.html. To receive a login Signing in and gaining access to a network server, Web server or other computer system. The process (the noun) is a "login" or "logon," while the act of doing it (the verb) is to "log in" or to "log on. id, please contact: David W. Brooks (email@example.com).
Table 1 Status Self-Identification of 3,446 Users Not Reported 533 AP HS Student 1,895 Non-AP HS Student 84 College Student 140 Other Student 44 AP HS Teacher 473 Non-AP HS Teacher 107 College Teacher 57 Other Teacher 46 Chemist (not student or teacher) 19 Other 48 Total 3,446 Table 2 Self-Description of 1,895 Self-Identified AP Students Gender Not Reported 73 Male 969 Female 853 Age Not reported 167 [greater than or equal to] 19 30 < 19 1,698 Location of access Not reported 235 Home 915 School 745 If accessing at school, time of access Not reported 38 During class 555 Table 3 Selected Errors Emergent Item Error % Problem Incorrect Category Incorrect Solid lithium LI 20 formula oxide is added to distilled water Failure to 50.0 mL of 0.100 N[a.sub.2]C[O.sub.3];NaCl 12 dissociate M sodium carbonate is added to 50.0 mL of 0.100 M hydrochloric acid Inappropriate The gases N[H.sub.4.sup.+] 27 dissociation ammonia and hydrogen chloride are mixed Weak Aqueous ammonia [H.sup.+] 21 acid/base is added to hydrofluoric acid Including Chlorine gas is N[a.sup.+] 9 spectators bubbled into cold, dilute sodium hydroxide Misconceptions Solid copper(II) CuO 28 sulfide is heated strongly in oxygen gas Typographical Aqueous hydrogen H20 (zero) 28 peroxide is decomposed using a catalyst
Aleven, V., Stahl, E., Schworm, S., Fischer, F., & Wallace, R. (2003). Help seeking and help design in interactive learning environments. Review of Educational Research, 73, 277-320.
Bransford, J. D., Brown, A. L., & Cocking, R. R. (Eds.). (1999). How people learn: Brain, mind, experience, and school. Washington, D.C.: National Academy Press.
Brooks, D. W., & Crippen, K. J. (2001). Learning difficult content using the Web: Strategies make a difference. Journal of Science Education and Technology, 10(4), 283-285.
Brooks, D. W., Schraw, G. P., & Crippen, K. J. (2005). Performance-related feedback: The hallmark of good instruction. Journal of Chemical Education, 82(4), 641-644.
Brown, A. (1995). Evaluation of teaching and learning processes in a computer-supported mechanical engineering course. Computers & Education, 25(1), 59-65.
Butler, D. L., & Winne, P. H. (1995). Feedback and self-regulated learning: A theoretical synthesis. Review of Educational Research, 65(3), 245-281.
Charlesworth, P. (2000, August). WebCT@MTU (1) (Maximum Transmission Unit, Maximum Transfer Unit) The largest frame size that can be transmitted over the network. For example, an Ethernet MTU is 1,500 bytes. Messages longer than the MTU must be divided into smaller frames. : Rejuvenating general chemistry. Paper presented at the Biennial Conference on Chemical Education, Ann Arbor Ann Arbor, city (1990 pop. 109,592), seat of Washtenaw co., S Mich., on the Huron River; inc. 1851. It is a research and educational center, with a large number of government and industrial research and development firms, many in high-technology fields such as , MI.
Cobb, P., Confrey, J., diSessa, A., Lehrer, R., & Schauble, L. (2003). Design experiments in educational research. Educational Researcher, 32(1), 9-13.
Crippen, K. J., & Brooks, D. W. (2001). Teaching advanced placement descriptive chemistry: Suggestions from a testing Web site. The Chemical Educator, 6(5), 266-271.
Crippen, K. J., & Brooks, D. W. (2002). An analysis of student learning at a testing Web site emphasizing descriptive chemistry. Journal of Computers in Mathematics and Science Teaching, 21(2), 183-201.
Crippen, K. J., & Brooks, D. W. (2004). A descriptive analysis of a chemistry teacher's Web site. ERIC Digest, ED481938.
Crippen, K. J., Brooks, D. W., & Abuloum, A. (2000). A Web-site supporting the AP descriptive chemistry question. Journal of Chemical Education, 77, 1087-1088.
Dufresne, R., Mestre, J., Hart, D. M., & Rath, K. A. (2002). The effect of Web-based homework on test performance in large enrollment introductory physics courses. Journal of Computers in Mathematics and Science Teaching, 21(3), 229-251.
Foos, P. W., & Fisher, R. P. (1988). Using tests as learning opportunities. Journal of Educational Psychology, 80, 179-183.
Hall, R. J., Pilant, M. S., & Strader, R. A. (1999, March). The impact of Web-based instruction on performance in an applied statistics course. Paper presented at the International conference on Mathematics/Science Education and Technology, San Antonio San Antonio (săn ăntō`nēō, əntōn`), city (1990 pop. 935,933), seat of Bexar co., S central Tex., at the source of the San Antonio River; inc. 1837. , TX.
Lhyle, K. C., & Kulhavy, R. W. (1987). Feedback processing and error correction. Journal of Educational Psychology, 79, 320-322.
Paull, T. A., Jacob, M. J., & Herrick, R. J. (1999, November). Automated homework in electrical engineering technology Electrical engineering technology (EET) is an engineering technology field related to electrical engineering. External Links
Penn, J. H., Nedeff, V. M., & Gozdzik, G. (2000). Organic chemistry and the Internet: A Web-based approach to homework and testing using the WE_LEARN system. Journal of Chemical Education, 77(2), 227-231.
Pintrich, P. R. (1995). Understanding self-regulated learning. In P. R. Pintrich (Ed.), Understanding self-regulated learning (pp. 3-12). San Francisco San Francisco (săn frănsĭs`kō), city (1990 pop. 723,959), coextensive with San Francisco co., W Calif., on the tip of a peninsula between the Pacific Ocean and San Francisco Bay, which are connected by the strait known as the Golden : Jossey-Bass.
Pressley, M., & McCormick, C. B. (1995). Advanced educational psychology for educators, researchers, and policymakers. New York New York, state, United States
New York, Middle Atlantic state of the United States. It is bordered by Vermont, Massachusetts, Connecticut, and the Atlantic Ocean (E), New Jersey and Pennsylvania (S), Lakes Erie and Ontario and the Canadian province of : Harper Collins.
Renkl, A. (1998). Cognitive science cognitive science
Interdisciplinary study that attempts to explain the cognitive processes of humans and some higher animals in terms of the manipulation of symbols using computational rules. : Learning from worked-out examples: A study on individual differences. Retrieved July 10, 2000, from http://www.umich.edu/%7Ecogsci/abstract/5-98renkl.html
Wegner, S. B., Holloway, K. C., & Garton, E. M. (1999). The effects of Internet-based instruction on student learning. Journal of Asynchronous Learning Asynchronous learning is a teaching method using the asynchronous delivery of training materials or content using computer network technology. It is an approach to providing technology-based training that incorporates learner-centric models of instruction. Networks, 3(2).
University of Nevada University of Nevada could refer to either of the universities in the Nevada System of Higher Education:
DAVID W. BROOKS
University of Nebraska-Lincoln