An analysis of student learning at a testing web site emphasizing descriptive chemistry.Since 1997, the World Wide Web (WWW WWW or W3: see World Wide Web.
(World Wide Web) The common host name for a Web server. The "www-dot" prefix on Web addresses is widely used to provide a recognizable way of identifying a Web site. or Web) has been used to enhance the performance of students on the Advanced Placement (AP) exam by providing a web site dedicated to the descriptive portion of the exam. This web site provides repeatable testing with feedback in the form of quizzes produced and graded in less than one second. This study evaluated the effectiveness of such an online testing and feedback system on outcome performance as measured by user records and validated val·i·date
tr.v. val·i·dat·ed, val·i·dat·ing, val·i·dates
1. To declare or make legally valid.
2. To mark with an indication of official sanction.
3. by user surveys.
The results indicate the potential for enhancing learning from instructional web sites focused on teaching with repetitive testing and feedback. Successful web teaching with repetitive testing must emphasize random, repeatable tests with a large item database and extensive feedback given with each item returned for grading. Future web teaching/testing sites should focus enhancements on three areas: (a) the user interface, (b) the tutorial An instructional book or program that takes the user through a prescribed sequence of steps in order to learn a product. Contrast with documentation, which, although instructional, tends to group features and functions by category. See tutorials in this publication. and feedback components, and (c) the nature of the quizzes.
For over 40 years, The Years, The
the seven decades of Eleanor Pargiter’s life. [Br. Lit.: Benét, 1109]
See : Time College Board has offered Advanced Placement (AP) Chemistry Examinations for high school students (College, 2000). This service allows academically well-qualified high school students to remain in high school while still engaging in learning activities that are appropriately challenging and rigorous. In 1999, the College Board reported the grading of over 48,000 chemistry exams (Sheridan Sheridan, city (1990 pop. 13,900), seat of Sheridan co., N Wyo., on Goose Creek E of the Bighorn Mts., in a mineral, livestock, and irrigated farm region; inc. 1884. It is a regional trade and market hub. , 1999).
The AP chemistry exam is broken down into five content areas: (a) structure of matter, (b) states of matter states of matter, forms of matter differing in several properties because of differences in the motions and forces of the molecules (or atoms, ions, or elementary particles) of which they are composed. , (c) reactions, (d) descriptive chemistry, and (e) laboratory. Anecdotal evidence anecdotal evidence,
n information obtained from personal accounts, examples, and observations. Usually not considered scientifically valid but may indicate areas for further investigation and research. suggests that the descriptive chemistry section of the AP chemistry exam is traditionally difficult for high school students. The exam's difficulty can be attributed to the nature of the material and the current structure of the AP curriculum. Descriptive chemistry is difficult to teach because it requires either a large amount of memorization mem·o·rize
tr.v. mem·o·rized, mem·o·riz·ing, mem·o·riz·es
1. To commit to memory; learn by heart.
2. Computer Science To store in memory: or experience; it tends to be disjointed within the traditional curriculum.
Since 1997, the World Wide Web (WWW or Web) has been used to enhance the performance of students on the AP exam by providing a web site dedicated to the descriptive portion of the exam (Crippen Crippen is a surname, and may refer to:
The database contained over 200 descriptive questions and represented all of the descriptive chemistry questions given on AP exams since 1968. The quiz A quiz is a form of game or mind sport in which the players (as individuals or in teams) attempt to answer questions correctly. Quizzes are also brief assessments used in education and similar fields to measure growth in knowledge, abilities, and/or skills. items were not multiple choice. Students were expected to write a net ionic i·on·ic
Of, containing, or involving an ion or ions.
pertaining to an ion or ions.
iontophoresis. chemical equation to represent a written reaction statement. The student's answer then resembled an essay response. In practice, student responses are hand graded with the essay part of the AP exam.
Testing and the Web
The study described here is grounded in literature concerning the use of the Web for teaching/learning, and the use of repetitive testing measures. Although use of the Web for teaching/learning is becoming very common, little is known about its effectiveness. Much is known about the effectiveness of repetitive testing with feedback, however.
Using the Web to deliver instruction is becoming common practice. Colleges and universities are looking seriously at their roles in providing instruction over the Web (Jones, 1995). Entire degree programs are now available over the Internet Internet
Publicly accessible computer network connecting many smaller networks from around the world. It grew out of a U.S. Defense Department program called ARPANET (Advanced Research Projects Agency Network), established in 1969 with connections between computers at the .
Using the Web as a teaching tool appears to be as effective as is the traditional classroom. The use of web instruction has shown no significant difference in test scores when compared to traditional test scores (Wegner Wegner is a surname, and may refer to:
In the simplest form, computers are able to process repetitive operations continuously. This makes them ideal for providing students with practice scenarios and problems. Computers can provide problems, evaluate responses to those problems, and provide immediate feedback. Providing appropriate feedback is known to increase the rate of learning (Lhyle & Kulhavy, 1987). As students practice with concepts, whether in the form of homework problems, in-class assignments, or with quizzes and tests, they learn concepts. Practice helps make the use of concepts automatic (Posner Prominent people with the surname Posner or Pozner include:
Feedback from practice need not be confined con·fine
v. con·fined, con·fin·ing, con·fines
1. To keep within bounds; restrict: Please confine your remarks to the issues at hand. See Synonyms at limit. to the correctness of an answer. Kulhavy and Stock (1989) described effective feedback as potentially providing the learner with two types of information: verification and elaboration. Verification distinguishes correct answers; elaboration is information providing the learner with clues to distinguish the correct answer. Since feedback points out missing or misunderstood mis·un·der·stood
Past tense and past participle of misunderstand.
1. Incorrectly understood or interpreted.
2. knowledge, it provides an ideal teaching moment. Thus, any feedback that further explains what students do not know adds to the value of the feedback.
Effective practice must occur with feedback. Feedback that explains how answers are determined and provides further instruction is most effective (Pressley & McCormick, 1995; Renkl, 1998). Many experts believe that technology's role in learning is providing feedback while allowing time for reflection and revision (Bransford, Brown, & Cocking cock 1
a. An adult male chicken; a rooster.
b. An adult male of various other birds.
2. A weathervane shaped like a rooster; a weathercock.
3. A leader or chief. , 1999)
Practice with feedback promotes self-regulated learning The term self-regulated can be used to describe learning that is guided by metacognition, strategic action (planning, monitoring, and evaluating personal progress against a standard), and motivation to learn . Pintrich described self-regulated learning as "involving the active, goal-directed, self-control self-control
Control of one's emotions, desires, or actions by one's own will. of behavior, motivation, and cognition cognition
Act or process of knowing. Cognition includes every mental process that may be described as an experience of knowing (including perceiving, recognizing, conceiving, and reasoning), as distinguished from an experience of feeling or of willing. for academic tasks by individual students" (Pintrich, 1995). Brooks, Nolan, & Gallagher advocated the promotion of self-regulation through the inclusion of specific strategies in web-based instructional design Instructional design is the practice of arranging media (communication technology) and content to help learners and teachers transfer knowledge most effectively. The process consists broadly of determining the current state of learner understanding, defining the end goal of (2001).
While practice typically implies homework or some sort of classroom work, testing also can serve as practice. Practice tests and quizzes not only prepare students for the type and variety of questions on a final graded exam, they also can serve as a teaching/learning tool for course content (Foos & Fisher, 1988).
Studies show that the Web facilitates the delivery of tests and supports distance learning (Shiffler, 1996). Integral to the delivery of online courses is a testing component. Many support the notion that the role of schools in the future is linked not to the information that they provide, but the role they take in assessment (Jones, 1995).
While not a panacea Some antidote or remedy that completely solves a problem. Most so-called panaceas in this industry, if they survive at all, wind up sitting alongside and working with the products they were supposed to replace. , web testing shows great promise in many areas. The success of web testing may hinge on Verb 1. hinge on - be contingent on; "The outcomes rides on the results of the election"; "Your grade will depends on your homework"
depend on, depend upon, devolve on, hinge upon, turn on, ride providing effective feedback. The nature of the feedback ensures that the student not only identifies what they do not know, but has the opportunity to reflect and acquire missing knowledge.
Full scale web teaching requires that all materials and testing be delivered over the Web. Current practice reveals that web testing is being implemented on a more narrow scale and in selective situations.
Using the Web to grade homework is a very practical application of the technology. Online homework systems have shown to have a positive effect on student test scores (Penn, Nedeff, & Gozdzik, 2000). This is especially so when the exam questions require the use of memorized concepts or the use of simple problem solving problem solving
Process involved in finding a solution to a problem. Many animals routinely solve problems of locomotion, food finding, and shelter through trial and error. algorithms The following is a list of the algorithms described in Wikipedia. See also the list of data structures, list of algorithm general topics and list of terms relating to algorithms and data structures. (Paull, Jacob Jacob (jā`kəb), in the Bible, ancestor of the Hebrews, the younger of Isaac and Rebecca's twin sons; the older was Esau. In exchange for a bowl of lentil soup, Jacob obtained Esau's birthright and, with his mother's help, received the blessing , & Herrick, 1999).
Using a web-based homework system has shown to increase exam scores by 5-10% and increase student self-confidence (Penn et al., 2000) in organic chemistry. Penn et al. showed a logistic lo·gis·tic also lo·gis·ti·cal
1. Of or relating to symbolic logic.
2. Of or relating to logistics.
[Medieval Latin logisticus, of calculation increase in practice exam score as the number of attempts increased.
Parallel to web-based homework systems are web-based practice systems where students can take practice quizzes of their own volition vo·li·tion
1. The act or an instance of making a conscious choice or decision.
2. A conscious choice or decision.
3. The power or faculty of choosing; the will. . While these practice exams do not affect a student's grade, early research suggested that students who use such practice score higher on actual exams than do students who use only traditional study methods (Hall, Pilant, & Strader, 1999). Charlesworth (2001) reports a direct correlation Noun 1. direct correlation - a correlation in which large values of one variable are associated with large values of the other and small with small; the correlation coefficient is between 0 and +1
positive correlation between online practice quiz scores and traditional exam scores for general chemistry students.
The purpose of this study was to evaluate the effectiveness of such an online testing and feedback system on outcome performance as measured by user records and validated by user surveys. This study was undertaken to elucidate e·lu·ci·date
v. e·lu·ci·dat·ed, e·lu·ci·dat·ing, e·lu·ci·dates
To make clear or plain, especially by explanation; clarify.
To give an explanation that serves to clarify. the manner in which effective teaching and learning can be conducted using the Web. The following three research questions guided the investigation:
1. Do users learn descriptive chemistry from using the AP descriptive chemistry web site?
2. Do self-reported AP students perceive the site to be effective in learning descriptive chemistry?
3. Are the students' perceptions of effectiveness in learning descriptive chemistry and their use patterns at the descriptive chemistry web site consistent with the literature on the effectiveness of repetitive testing?
The design of the AP descriptive chemistry web site emphasized tracking of an individual's use and surveying user perceptions. This design yielded information about the effectiveness of the site as a learning tool. The site's design was consistent with the suggestions and considerations of many experts in providing online instruction for high school science students (Hoffman & Others, 1997; Lyons & Others, 1997; Ritchie & Hoffman, 1997). In addition, this site was designed with the understanding that incentives positively affect student learning and performance (Gettinger, 1989).
The research sample represents individuals interested in descriptive chemistry, especially as it applies to the AP chemistry examination. High school students were intended subjects. The sample was recruited by word of mouth, conference presentations, listserve postings (CHEM-ED-L, AP-CHEM), and newsgroup newsgroup
Internet forum for discussion of specific subjects. Newsgroups are organized into subjects (e.g., automobiles); each typically has several subgroups (e.g., classic cars, Formula One racing cars). postings (misc.education.science, k12.education.science). The final sample included 404 self-reported AP students.
Study participants were identified by the e-mail address See Internet address.
e-mail address - electronic mail address they provided. Once informed consent was given, each new user had the option of completing a preliminary survey. By completing this survey, each user established a login Signing in and gaining access to a network server, Web server or other computer system. The process (the noun) is a "login" or "logon," while the act of doing it (the verb) is to "log in" or to "log on. . Each time a user logged in, all of his/her transactions were recorded together with the e-mail login. The access time, access address (computer IP number), specific identify of the items sent, responses made, and tutoring requested were all recorded.
Each user was asked to complete a brief demographic survey the first time they used the site. By completing this survey, each user established a logon See login.
1. (jargon) logon - login.
2. (networking) logon - In ACF/VTAM, an unformatted session-initiation request for a session between two logical units. . Each subsequent visit to the site required the user to login and bypassed the survey.
The database used the logon to create a unique record for each user. This record was accessible to the user; it documented their use of the site. From the record, each user could review previous quizzes as well as their use of tutoring.
Once logged in, students requested a quiz. This quiz was a randomly generated set of eight items. The items were drawn from 14 categories defined arbitrarily by one of the authors.
The content of most quizzes was similar to that used by The College Board. This was accomplished by assigning as·sign
tr.v. as·signed, as·sign·ing, as·signs
1. To set apart for a particular purpose; designate: assigned a day for the inspection.
2. a chemistry topic category to each quiz item. When a quiz was constructed, each of the eight items came from a different category. Weight was given to the categories based upon the total number of items available from that category. The item database grows each year that an AP examination is given; so, the quiz evolves with the AP test-questioning scheme. While probability of a duplicate DUPLICATE. The double of anything.
2. It is usually applied to agreements, letters, receipts, and the like, when two originals are made of either of them. Each copy has the same effect. quiz was extremely low, the likelihood of seeing items repeated, though small at first, grew with each test.
The user had the option of having tutoring sent with the items. This tutoring was in the form of web pages written for each category of items. Having tutoring with the items allowed the user to learn more about the category of reaction before submitting an item response for grading. The database tracked which items and tutoring were sent to each user, and this information was included in their record. The database also tracked which tutoring was used with each item.
The user also had the option of choosing to have an entire quiz constructed from one of nine predefined item categories. Students could request a content topic, such as electrolysis electrolysis (ĭlĕktrŏl`əsĭs), passage of an electric current through a conducting solution or molten salt that is decomposed in the process. , and then have their entire eight-item quiz chosen from that topic area. By choosing typical AP Quiz, however, users received a randomized ran·dom·ize
tr.v. ran·dom·ized, ran·dom·iz·ing, ran·dom·iz·es
To make random in arrangement, especially in order to control the variables in an experiment. quiz similar to what they would receive at an AP examination. This feature allowed users to hone their skills on specifics with their choice of similar environment, or prepare for the randomness of the actual AP examination.
Following the 2000 AP Chemistry Examination, the raw data was removed from the database. The e-mail addresses were inspected, and obvious false addresses were removed.
A survey designed specifically for the AP students was sent on Tuesday, May 16, 2000, the day of the AP chemistry exam. The survey instrument was an e-mail response survey composed primarily of Likert scale Likert scale A subjective scoring system that allows a person being surveyed to quantify likes and preferences on a 5-point scale, with 1 being the least important, relevant, interesting, most ho-hum, or other, and 5 being most excellent, yeehah important, etc responses. This method was chosen over other options because of its potential range and ease of use for the respondents In the context of marketing research, a representative sample drawn from a larger population of people from whom information is collected and used to develop or confirm marketing strategy. (Bowers Bowers is a surname, and may refer to
Specifications of materials, physical measurements, processes, performance of products, and characteristics of services rendered. Design standards may be established by individual manufacturers, trade associations, and national or currently reported in the literature (Bowers, 1998; Gaddis, 1998). Respondents simply selected reply in their e-mail program Software in the user's computer that can access the mail servers in a local or remote network. Also known as an "e-mail client," "mail client," "mail program," and "mail reader," it provides the ability to send and receive e-mail messages and file attachments. , added their responses to the original message, and then selected send to return the survey to the researchers.
Statistics used in demographic descriptions are as they appear; date, time, gender, and so forth. A tutoring request implied that a user requested some form of extra information not supplied in the quiz itself and was counted by subcategory sub·cat·e·go·ry
n. pl. sub·cat·e·go·ries
A subdivision that has common differentiating characteristics within a larger category. . The nature and use of quizzes however, require the use and explanation of some special statistics. The use of such statistics allows for a thorough documentation of learning at the descriptive chemistry web site. A quiz sent by the database was not necessarily completed or returned by the user. Thus, the emphasis of the learning analysis focuses upon those quizzes returned for grading. Quiz numbers where students did not attempt any items were removed before analysis.
Quizzes returned for grading were given a unique number and broken down item by item. Each of eight items contained six potential data fields for answers. If any of the six data fields for a particular item contained text, the item was counted as an attempt at that item. An individual item score was computed for each item of every quiz. Scores for completed items were combined to compute To perform mathematical operations or general computer processing. For an explanation of "The 3 C's," or how the computer processes data, see computer. the Average Score per Completed Item statistic statistic,
n a value or number that describes a series of quantitative observations or measures; a value calculated from a sample.
a numerical value calculated from a number of observations in order to summarize them. . This statistic is a numerical numerical
expressed in numbers, i.e. Arabic numerals of 0 to 9 inclusive.
a numerical code is used to indicate the words, or other alphabetical signals, intended. average including only those items attempted by the student.
Quiz items were further classified as new or as repeat. A new item is one from the database that the user had never seen before. A repeat item is one that appeared on a previous quiz the user has submitted for grading. Thus, the average score per item completed statistic can be further sub-classified by item status. In addition, the value and significance of repeat items were quantified.
The AP student population was split 50% male, 45% female, and 5% not reported. The largest percentage of AP students reported their age as less than 18 years (84%). The students were located throughout the United States United States, officially United States of America, republic (2005 est. pop. 295,734,000), 3,539,227 sq mi (9,166,598 sq km), North America. The United States is the world's third largest country in population and the fourth largest country in area. with the largest concentration in the northeastern US (Figure 1).
More than one third (41%) of the AP students reported using the descriptive chemistry web site during chemistry class. Ninety-seven percent of the self-reported AP students who responded to a post AP chemistry examination survey reported having taken the AP chemistry examination on May 16, 2000.
AP Student Site Use Pattern
The self-reported AP students averaged 21.2 logons to the web site and averaged 7.7 tutoring requests per individual over the 1999-2000 academic term. Acid/base chemistry was the most requested tutoring component by a sizable siz·a·ble also size·a·ble
Of considerable size; fairly large.
siza·ble·ness n. margin.
Over the 1999-2000 academic term, the AP student group requested an average of 4.7 quizzes. The quizzes were mostly of the typical category, indicating the quizzes were similar in item composition to the actual ALP (language) ALP - A list processing extension of Mercury Autocode.
["ALP, An Autocode List-Processing Language", D.C. Cooper et al, Computer J 5:28-31, 1962]. chemistry examination.
AP Student Learning Analysis
Learning at the site is documented using the following statistics: average raw quiz score, average score per completed item, average score per new item, average score per repeat item, date, day number in the study, and quiz number. The average raw quiz score per quiz number and the average score per completed item (sub-classified as overall, new, and repeat) all increase m a linear fashion with high correlation coefficients Correlation Coefficient
A measure that determines the degree to which two variable's movements are associated.
The correlation coefficient is calculated as: . The average raw quiz score increased as the study progressed on a day-by-day basis. Significance testing was not applied to the general AP student population as the numbers of students attempting any quiz ranges from 300 students attempting one quiz to a single student making an attempt after 56 quizzes.
AP students did not complete all of the items on every quiz. Many quizzes were returned partially completed. However, no pattern emerged for the number of items completed. Raw quiz score increases linearly as the quiz number increases, r = +0.873.
This pattern suggests that AP students earned progressively higher quiz scores as they took more quizzes. However, the inconsistent nature of the number of quiz items completed suggests that the raw quiz score is not an appropriate statistic for documenting learning.
In addition, due to the size and random nature of the database, it was possible for students to have attempted and completed repeat quiz items only. This would cause the raw score to rise, but not necessarily imply the acquisition of new knowledge. Instead, it might imply an acquired ability to answer specific test items.
Following the first quiz, the number of repeat items attempted on each successive quiz grows linearly as the number of new items decreases linearly. Beyond quiz number 28, all of the quiz items were essentially repeat questions. Completing only repeat items could account for the raw score increase. Interestingly, the AP students did not attempt all of the repeat items for every quiz. The percentage of repeat items attempted per quiz hovers around 80% until quiz 35 where it climbs to essentially 100% as the number of quiz attempts falls.
A plot of the average score per completed item for all 56 quizzes attempted by members of the AP student group reveals a linear relationship, r = +0.852. This indicates a steady improvement over time. The average score per repeat item is also linear with quiz number, r = +0.775.
Yet, the change in average score per completed item is lower for the repeat items. While a portion of the linear increase in the overall average score per completed item can be attributed to the repeat items, some additional factor also was improving the scores.
As further evidence of improved performance and learning through using the site, the average raw score was plotted as a function of the day in the study. The average raw score increases in a positive fashion as the study progressed through the 1999-2000 academic term.
The data show an increasing trend. This trend coupled with a linear increase in the average score per completed item gives strong evidence for learning by the AP students as they request, complete, and are provided with feedback on graded quizzes.
The AP Special Group
The AP Special Group represents a 55-member subset A group of commands or functions that do not include all the capabilities of the original specification. Software or hardware components designed for the subset will also work with the original. of the AP student population that took a minimum of seven unique quizzes and for which significance testing was applied. This subset represents a group of students that made extensive use of the site for taking quizzes. This subset was chosen to further elucidate and identify characteristics of a population intent on using the web site as a means for achieving success on the AP chemistry examination.
The demographics The attributes of people in a particular geographic area. Used for marketing purposes, population, ethnic origins, religion, spoken language, income and age range are examples of demographic data. and descriptive use statistics for the AP Special Group were consistent with those of the general AP student population. However, exceptions included:
* One hundred percent of the AP special group who responded to a post AP chemistry examination survey reported having taken the AP chemistry examination on May 16, 2000.
* The AP special group averaged 62.5 logons to the web site, more than four times that of the remaining AP student population.
* The AP special group averaged 20.1 tutoring requests, three and one-half times that of the remaining AP student population.
AP Special Learning Analysis
For the AP special students, raw quiz score increases linearly with quiz number at a rate close to that of the entire AP student group. This suggests that the AP special group accounts for a large portion of the gains documented by the entire self-reported AP student population.
The average score per completed item as a function of quiz number for the AP special student group reveals linear relationships for repeat items and overall scores consistent with those reported for the general AP student population. An analysis of the first seven quizzes for the AP special students was intended to quantify Quantify - A performance analysis tool from Pure Software. the effect of new items on the linear increase of average score per completed item in the small quiz number range where the student score is most influenced by new items.
Within the first seven quizzes, students attempted a proportion of new and repeat items, allowing for separation and quantification quan·ti·fy
tr.v. quan·ti·fied, quan·ti·fy·ing, quan·ti·fies
1. To determine or express the quantity of.
2. of the impact of these variables on the overall average score per item completed. Because the AP special group represented a sizable, stable sample, significance tests were applied to all correlations for the first seven quizzes.
The correlation between the average score per completed item overall and quiz number is significant, r = +0.871, n = 55, p<0.005, one tail. The correlation between the average score per new item completed and quiz number is significant, r = +0.631, n = 55, p<0.005, one tail. The correlation between the average score per repeat item completed and quiz number is not statistically significant, r = +0.218. The rates of change in average score per new item is close to 1.5 times that of the average score per repeat item, 0.0422 and 0.0282 points per item respectively. However, the average rate of change overall is greater, 0.0594 points per item. Table 1 summarizes the correlation analysis for the first quizzes.
An item analysis of the first seven quizzes indicates that the percentage of new items attempted starts at 100% on the first quiz and decreases approximately 5% with each successive quiz. The repeat items attempted start at 0% on the first quiz and increase proportionally pro·por·tion·al
1. Forming a relationship with other parts or quantities; being in proportion.
2. Properly related in size, degree, or other measurable characteristics; corresponding: . The large percentage of new items attempted in the first seven quizzes means that the overall average score per completed item is largely dependent upon the average score per new item completed. With increasing quiz number, the dependence of the overall average score per completed item shifts progressively from the new items attempted to the repeat items attempted.
The combined linear nature of both the average score per new item completed and the average score per repeat item completed increases the linear nature of the overall average score per item completed. The overall rate of change for the average score per item is a weighted total of the percentage of item types attempted and their average score per completed item value.
The dependence of the overall average on the new item average suggests that the linear increase of overall average score per item completed in the first quizzes can be attributed to a continually con·tin·u·al
1. Recurring regularly or frequently: the continual need to pay the mortgage.
2. improving performance on new items, those never seen before. This increasing positive change is most likely attributable to an increase in knowledge.
Student Perceptions of Site Effectiveness
Thirty percent of the AP student population with valid e-mail addresses responded to the post exam survey (105 students). When asked "How helpful was the descriptive AP chemistry quiz site in preparing you for the AP chemistry examination?" the students that responded generally felt that the site was helpful. However, certain circumstances CIRCUMSTANCES, evidence. The particulars which accompany a fact.
2. The facts proved are either possible or impossible, ordinary and probable, or extraordinary and improbable, recent or ancient; they may have happened near us, or afar off; they are public or such as computer failure, access issues, and reported learning style conflicts caused some students to feel the site was less than helpful. AP special students felt that the site was very helpful. Their comments are universally positive and affirm the design intentions of the site.
Sample student free responses:
* "The site gave me a better idea of what would be expected of me and how the test would look."
* "Though the format for the reaction section changed, the site was still invaluable for practice. Some of the same problems appeared on the exam."
* "Since I wasn't able to cover some areas in my class, I was learning about stuff from the web site."
* "It was helpful with the reactions part of the AP if you actually sat and did some of the questions. It taught you skills."
When asked, "Did using this site improve your performance on the AP chemistry examination?" the students that responded affirmed af·firm
v. af·firmed, af·firm·ing, af·firms
1. To declare positively or firmly; maintain to be true.
2. To support or uphold the validity of; confirm.
v.intr. this site as helping to improve their performance on the AP chemistry examination. Students cited problems entering equations with a computer keyboard, time requirements, and the lack of knowledge about their actual exam scores as reasons for the site not improving their performance on the AP chemistry examination. One hundred percent of the AP special population that responded to the survey felt using the site improved their performance on the AP chemistry examination.
Sample student free responses:
* "This site improved my score I think....Because it had so many questions to practice and helped you learn what kind of reaction you were doing. It was very useful."
* "I learned what I needed to study."
* "It definitely improved my ability to perform the reactions. Without it, I probably wouldn't have known what to look for."
Self-reported AP students used the AP descriptive web site with the explicit purpose of improving either their performance on quizzes or their knowledge of the concepts needed to answer the descriptive questions on the AP chemistry examination. This is supported both by their use of the tutoring components and the quizzing/feedback components at the site. The AP students were not casual observers of the site; their use implies intent to learn descriptive chemistry.
The fact that the AP students used the site during the school day and reported use during chemistry class provides further evidence to support their learning intent. Yet, the majority of students recorded their use in just a short time period before the AP chemistry examination was given. While the students have used the site at effective times in the day and under supportive circumstances, the fact that they access the site just before the AP chemistry examination limits their potential achievement.
Learning by the AP students at the descriptive chemistry web site is documented using a number of statistics. These statistics include an increase in the average raw quiz score per day during the study. While the AP students do show a linear improvement in quiz score with quiz number, they never acquire proficiency pro·fi·cien·cy
n. pl. pro·fi·cien·cies
The state or quality of being proficient; competence.
Noun 1. proficiency - the quality of having great facility and competence . The linear increase in the average raw quiz score with increasing quiz number suggests that the greater the number of quizzes attempted by students, the better the overall score. However, the students start out earning very low quiz scores, and few students achieve a perfect score.
The AP students show a linear increase in the average score per completed item through the first nine quizzes in all item sub-classifications. However, though the students improve in all sub-categories, their total improvement is about one-half of one point per item from a baseline The horizontal line to which the bottoms of lowercase characters (without descenders) are aligned. See typeface.
baseline - released version average of one point per item. At this rate, the students should have achieved proficiency after 29 quizzes.
Some students selected only certain items to attempt: particular chemistry content, new items, or repeat items. Their rationale rationale (rash´nal´),
n the fundamental reasons used as the basis for a decision or action. for this is unclear. No clear pattern exists for any group's completion rates for types of items. Whether or not an item was completed appears to be a random occurrence.
The total size of the AP student group permitted selection of subsets. The criterion for inclusion in one subset was extensive use of the feedback component at the site. This group, named the AP special group, consisted of 55 self-reported AP students who returned the largest number of graded quizzes.
Self-reported AP students who responded to the post AP examination survey generally felt that the site was helpful. However, circumstances such as computer failure, access difficulties, and reported learning style conflicts caused some students to feel the site was less than helpful. This survey feedback supports and is consistent with the learning analysis based on the group's use of the site.
The majority of the AP students affirmed the AP descriptive chemistry web site as helpful in improving their performance on the AP chemistry examination. Students cited problems entering equations with a computer keyboard, time requirements, and the lack of knowledge about their actual score on the May 16, 2000, AP chemistry examination as reasons for the site not improving their performance on the AP chemistry examination. This feedback supports and is consistent with the learning analysis based on the group's use of the site.
If the statements made by the AP students responding to the post AP examination survey concerning both entry errors and computer failures are valid for the entire user population, then such errors and failure could affect the quiz scores. These errors could account for variability in compiling com·pile
tr.v. com·piled, com·pil·ing, com·piles
1. To gather into a single book.
2. To put together or compose from materials gathered from several sources: certain statistics, for example.
The NP special group is a subset of the self-reported AP student group who made extensive use of the quizzing/feedback components of the AP descriptive chemistry web site. The AP special group had the highest use of both the tutoring components and the largest number of quizzes attempted. The AP special group was dedicated to using the NP descriptive web site to learn descriptive chemistry and to improve their score on the AP chemistry examination.
A date and time analysis for the NP special group suggests that this group had similar use patterns to those of the general NP student group. These included logon times, use during the week, and dates during the academic term. The principal distinctions in use for the NP special group were the total number of logons and the consistency of returning graded quizzes.
The AP special group also shows a linear increase in the raw quiz score with quiz number. The minimum, maximum, and average change in score are consistent with those for the general NP population. Again, these scores start low and never reach the maximum score. Although the AP special group makes the most extensive use of all the resources at the AP descriptive web site, few, like the general AP population, ever achieve a perfect score.
Analysis of average score per item with quiz number again is very similar to the general AP population. Statistical significance is shown for the increase in average score per new item completed and in average score per item completed overall for the first seven quizzes. The NP special students start with a higher average score-per-item on the first seven quizzes, but improve an average of 0.0594 points per item. The NP special group scores higher on early quizzes, improves their subsequent scores incrementally, but the rate of increase is very low. At their initial rate, the AP special group should have achieved proficiency after 36 quizzes.
These results suggest that AP students will benefit from taking as many quizzes as possible. The fact that the AP students in this study increase their scores at such a slow rate suggests that some other mitigating mit·i·gate
v. mit·i·gat·ed, mit·i·gat·ing, mit·i·gates
To moderate (a quality or condition) in force or intensity; alleviate. See Synonyms at relieve.
To become milder. factor be required to enhance the effectiveness of the site. However, analysis of a select group of three AP special students reveals an increase in quiz score over several weeks or months. This implies that an early use of the site may influence both the maximum score earned and the rate of score improvement.
Self-reported AP students selected as the AP special group who responded to the post AP examination survey felt that the site was helpful. Their comments are universally positive, and affirm the design intentions of the site. All of the survey respondents (100%) felt using the site improved their performance on the AP chemistry examination. This supports and is consistent with the learning analysis based on the group's use of the site.
The results of this study indicate the potential for enhancing learning from instructional web sites focused on teaching with repetitive testing and feedback. Successful web teaching with repetitive testing must emphasize random, repeatable tests with a large item database and extensive feedback given with each item returned for grading. In this way, the web teaching site focuses users on the content to be learned and supports user self-regulation.
This study merits several recommendations for further study. Future web teaching/testing sites should focus enhancements on three areas: (a) the user interface, (b) the tutorial and feedback components, and (c) the nature of the quizzes.
The user interface should give users many options in how their quizzes are constructed. In addition to choosing the types of items they receive, users should have the option of using pull down menus to aid in formula writing, or to have simple hints like the type of question included with the question. Instead of creating a new quiz when the New Quiz button is selected, the user could be presented with a series of switches that allow them greater flexibility in quiz design.
Early research indicates that computer practice and handwritten hand·write
tr.v. hand·wrote , hand·writ·ten , hand·writ·ing, hand·writes
To write by hand.
[Back-formation from handwritten.]
Adj. 1. testing are not congruent con·gru·ent
1. Corresponding; congruous.
a. Coinciding exactly when superimposed: congruent triangles.
b. (Russell, 1999; Russell & Haney, 1997). Many of the survey respondents in the current study indicated problems with formula entry at the web site. For practice sites where the ultimate measure remains a handwritten exam, site users should have the option to vary the amount of formula typing they are forced to perform in their quizzes by selecting pull down menus as an option. Flexibility in the user interface that allows users to tailor A tailor is a person whose occupation is to sew menswear style jackets and the skirts or trousers that go with them.
Although the term dates to the thirteenth century, tailor their quizzes eliminates bias in the scoring due to entry problems and better identifies techniques of successful users.
Enhancing tutorial components presents the best opportunity for continued research while increasing the rate of learning for the users. Depth and breadth should be added to the content tutoring associated with each question. In addition, the feedback provided with each graded response should be item specific. Item-specific feedback includes remarks consistent with each unique item in the database. In this way, users have the option to take tutoring about reaction types; yet, each graded response returns remarks consistent with the individual items on their quiz.
With the greater flexibility in quiz design provided to the user, they not only have the option to take content tutoring, but they can turn the item-specific tutoring off and on. Item-specific tutoring addresses the low rate of learning in this study and allows the researcher to identify successful learning strategies.
Recommendations from the results of this study have the potential to assist web teachers, researchers, and students. Web teachers should provide repetitive testing components with feedback at their sites to maximize student learning. Further research at the descriptive chemistry web site should include enhancements to the user interface, the tutoring components, and the nature of the quizzes. Including these enhancements could provide valuable data concerning successful use patterns and further assist users in learning the content emphasized at the site.
[FIGURE 2 OMITTED]
Table 1 Correlation Analysis Summary for Average Score Per Item Completed by Quiz Number for the First Quizzes Group Overall New Items Rate BL Corr Rate BL Corr AP Student (9 quizzes) 0.0756 0.8095 +0.935 0.0551 0.8491 +0.889 AP Special (7 quizzes) 0.0594 0.8928 +0.871 * 0.0422 0.9243 +0.631 * Group Repeat Items Rate BL Corr AP Student (9 quizzes) 0.041 1.3339 +0.551 AP Special (7 quizzes) 0.0282 1.3818 +0.218 Rate = points per item per quiz BL = Baseline (points) Corr = Pearson correlation * p<0.005, one tail
Bowers, D.K. (1998). FAQ's on online research. Marketing Research, 10(4), 45-49.
Bransford, J.D., Brown, A.L., & Cocking, R.R. (Eds.), How people learn: Brain, mind, experience, and school. Washington, DC: National Academy Press.
Brooks, D.W., Nolan, D., & Gallagher, S. (2001). Web teaching (2nd ed.). Norwell, MA: Plenum In a building, the space between the real ceiling and the dropped ceiling, which is often used as an air duct for heating and air conditioning. It is also filled with electrical, telephone and network wires. See plenum cable. .
Brown, A. (1995). Evaluation of teaching and learning processes in a computer-supported mechanical engineering course. Computers & Education, 25(1), 59-65.
Charlesworth, P. (2000). WebCT@MTU (1) (Maximum Transmission Unit, Maximum Transfer Unit) The largest frame size that can be transmitted over the network. For example, an Ethernet MTU is 1,500 bytes. Messages longer than the MTU must be divided into smaller frames. : Rejuvenating general chemistry. Paper presented at the Biennial biennial, plant requiring two years to complete its life cycle, as distinguished from an annual or a perennial. In the first year a biennial usually produces a rosette of leaves (e.g., the cabbage) and a fleshy root, which acts as a food reserve over the winter. Conference on Chemical Education, Ann Arbor Ann Arbor, city (1990 pop. 109,592), seat of Washtenaw co., S Mich., on the Huron River; inc. 1851. It is a research and educational center, with a large number of government and industrial research and development firms, many in high-technology fields such as , MI.
College Board (1999). AP Chemistry: Distribution of grades for all 1999 APChemistry candidates. Retrieved from the World Wide Web May 23, 2002 from: http://www.collegeboard.com/ap/students/chemistry/ grade99.html
College Board (2000). AP technical corner: From humble beginnings Humble Beginnings was an American pop punk band from New Jersey. While never gaining large-scale success, many of the band's members went on to mainstream success with other outfits. . Retrieved from the World Wide Web May 21, 2002 from: http://www.collegeboard.org/ap/techman/chap1/
Crippen, K.J., Brooks, D.W., & Abuloum, A. (2000). A web site supporting the AP descriptive chemistry question. Journal of Chemical Education, 77(8), 1087-1088.
Foos, P.W., & Fisher, R.P. (1988). Using tests as learning opportunities. Journal of Educational Psychology, 80, 179-183.
Gaddis, S.E. (1998). How to design online surveys. Training and Development, 52(6), 67-72.
Gettinger, M. (1989). Effects of maximizing time spent and minimizing time needed for learning on pupil achievement. Educational Research Journal, 26, 73-91.
Hall, R.J., Pilant, M.S., & Strader, R.A. (1999). The impact of web-based instruction on performance in an applied statistics course. Paper presented at the International Conference on Mathematics/Science Education and Technology, San Antonio San Antonio (săn ăntō`nēō, əntōn`), city (1990 pop. 935,933), seat of Bexar co., S central Tex., at the source of the San Antonio River; inc. 1837. , TX.
Hoffman, J.L., Kupperman, J. & Wallace Wal·lace , Alfred Russel 1823-1913.
British naturalist who developed a concept of evolution that paralleled the work of Charles Darwin. , R. (1997). Online learning materials for the science classroom: Design methodology and implementation. Paper presented at the Annual Meeting of the American Educational Research Association The American Educational Research Association, or AERA, was founded in 1916 as a professional organization representing educational researchers in the United States and around the world. , Chicago.
Jones, D.P. (1995). Higher education higher education
Study beyond the level of secondary education. Institutions of higher education include not only colleges and universities but also professional schools in such fields as law, theology, medicine, business, music, and art. and high technology: A case for joint action. Denver, CO: Western Governor's Association.
Kulhavy, R.W., & Stock, W.A. (1989). Feedback in written instruction: The place of response certitude cer·ti·tude
1. The state of being certain; complete assurance; confidence.
2. Sureness of occurrence or result; inevitability.
3. . Educational Psychology Review, 1(4), 279-308.
Lhyle, K.C., & Kulhavy, R.W. (1987). Feedback processing and error correction. Journal of Educational Psychology, 79, 320-322.
Lyons, D.J., Hoffman, J.L. Krajcik, J.S., & Soloway, E. (1997, March). An investigation of the use of the World Wide Web for online inquiry in a science classroom. Paper presented at the Annual Meeting of the National Association for Research in Science Teaching, Oak Brook, IL.
Paull, T.A., Jacob, M.J., & Herrick, R.J. (1999). Automated au·to·mate
v. au·to·mat·ed, au·to·mat·ing, au·to·mates
1. To convert to automatic operation: automate a factory.
2. homework in electrical engineering technology Electrical engineering technology (EET) is an engineering technology field related to electrical engineering. External Links
In ancient Greece, an aristocratic banquet at which men met to discuss philosophical and political issues and recite poetry. It began as a warrior feast. Rooms were designed specifically for the proceedings. , Charlotte, NC. Retrieved from the World Wide Web May 23, 2002 from: http://www.asee.org/conferences/search/99conf34.PDF (Portable Document Format) The de facto standard for document publishing from Adobe. On the Web, there are countless brochures, data sheets, white papers and technical manuals in the PDF format.
Penn, J.H., Nedeff, V.M., & Gozdzik, G. (2000). Organic chemistry and the Internet: A web-based approach to homework and testing using the WE_LEARN system. Journal of Chemical Education, 77(2), 227-231.
Pintrich, P.R. (1995). Understanding self-regulated learning. In P.R. Pintrich (Ed.), Understanding Self-Regulated Learning (pp. 3-12). San Francisco San Francisco (săn frănsĭs`kō), city (1990 pop. 723,959), coextensive with San Francisco co., W Calif., on the tip of a peninsula between the Pacific Ocean and San Francisco Bay, which are connected by the strait known as the Golden : Jossey-Bass.
Posner, M., & Snyder, C. (1975). Attention and cognitive control. In R. Solso (Ed.), Information processing information processing: see data processing.
Acquisition, recording, organization, retrieval, display, and dissemination of information. Today the term usually refers to computer-based operations. and cognition. Hillsdale, NJ: Erlbaum.
Pressley, M., & McCormick, C.B. (1995). Advanced educational psychology for educators, researchers, and policymakers. New York New York, state, United States
New York, Middle Atlantic state of the United States. It is bordered by Vermont, Massachusetts, Connecticut, and the Atlantic Ocean (E), New Jersey and Pennsylvania (S), Lakes Erie and Ontario and the Canadian province of : Harper Collins.
Renkl, A. (1998). Cognitive science cognitive science
Interdisciplinary study that attempts to explain the cognitive processes of humans and some higher animals in terms of the manipulation of symbols using computational rules. : Learning from worked-out examples: A study on individual differences. Retrieved from the World Wide Web May 21, 2002 from: http://www.umich.edu/%7Ecogsci/abstract/598renkl.html
Ritchie, D.C., & Hoffman, B. (1997). Using instructional design principles to amplify learning on the World Wide Web. (ERIC Document Reproduction Service No. ED415835)
Russell, M. (1999). Testing writing on computers: A follow-up follow-up,
n the process of monitoring the progress of a patient after a period of active treatment.
follow-up plan study comparing performance on computer and on paper. Educational Policy Analysis Archives, 7(20).
Russell, M., & Haney, W. (1997). Testing writing on computers: An experiment comparing student performance on tests conducted via computer and via paper-and-pencil. Educational Policy Analysis Archives, 5(3). Retrieved from the World Wide Web May 21, 2002 from: http://olam.ed.asu.edu/epaa/v5n3.html
Shiffler, N. (1996). The state of the Web in the web of the states. Paper presented at the Annual meeting of the National Council on Measurement in Education, New York.
Shiffrin, R.M., & Schneider, W. (1977). Controlled and automatic human information processing: II. Perceptual per·cep·tu·al
Of, based on, or involving perception. learning, automatic attending and a general theory. Psychological Review, 84, 127-190.
Wegner, S.B., Holloway, K.C., & Garton, E.M. (1999). The effects of Internet-based Instruction on student learning. Journal of Asynchronous Learning Asynchronous learning is a teaching method using the asynchronous delivery of training materials or content using computer network technology. It is an approach to providing technology-based training that incorporates learner-centric models of instruction. Networks, 3(2). Retrieved from the World Wide Web May 21, 2002 from: http://www.aln.org/alnweb/journal/Vol3_issue2/Wegner.htm