The impact of web-based assessment and practice on students' mathematics learning attitudes.
Web-based assessment provides students with more chances for practice, self-testing, self-regulation, and self-evaluation, while teachers receive more feedback from students, save time in reading and grading, and have closer interactions with students (Fleischman, 2001; Lockwood, 2001). In using a web-based assessment tool, teachers can integrate the different content areas for their students and enhance students' attitude and motivation toward the learning subject. At the same time, students can use online technology in learning and helping themselves succeed (Morgan & O'Reilly, 2001).
PURPOSE OF THE STUDY AND RESEARCH QUESTIONS
Upon recent suggestions of the use of web-based assessment in enhancing mathematics teaching and learning, this study profoundly discovered the effects of web-based assessment and practice on students' mathematics learning attitudes. Explicating unique capabilities of the innovative online practice on making crucial contributions to the traditional practice would lead to suggestions for future use and appropriate development of online mathematics assessment and tutorial system. The comparisons of online and traditional paper-and-pencil practices were conducted to determine the effective and relevant components of online delivery medium. The research mainly investigated: (a) the effect of web-based assessment and practice on middle graders mathematics learning attitudes; and (b) the mathematics learning attitudes among students in the web-based group in terms of genders and different ethnicities.
Emergence of Web-Based Technology in Teaching and Learning
The recommendations of the congressional Web-Based Education Commission (WBEC, 2001) make a compelling case for integrating and making use of the web-based applications in classrooms. Morgan and O'Reilly (2001) have reported that online technologies have significantly increased the opportunities for innovative assessments. These innovative assessments push the boundary of the traditional paper-based learning experience with current online technology. In the recent time, web-based instruction and web-based drill-and-practice have commonly served as tutorials in the educational setting (Nguyen & Allen, 2006). Merill and Hammons (1996) indicated that these tutorials can fill the gaps and reinforce concept learning by presenting alternative methods, checking for understanding, providing explanations when appropriate, and indicating mastery level of computational skills. Tutorial uses can effectively provide students with controls over their learning (Sanchis, 2001). Free access to web-based lesson planning and online tutorials, such as the Math Forum, Compass, SOS, and many other state resources offer students tremendous information to improve their learning. At the same time, students can engage in meaningful learning rather than memorization to develop their own interest toward mathematics learning.
WEB-BASED DRILL AND PRACTICE ENHANCING MATHEMATICS LEARNING ATTITUDE AND ACHIEVEMENT
According to Hart and Walker (1993), students' attitudes toward a learning subject vary based on characteristics of classroom and instruction, such as types of assessment, topics, and material delivery tools. Steele and Arth (1998) also indicated that the flexibility about accepting students' ways of solving problems can increase students' participation, reduce anxiety, and increase positive attitudes in learning. Web-based applications can create different learning and assessment contexts, and produce flexible approaches to instruction and evaluation (Allen, 2001; Liang & Creasy, 2004). Assessment that is provided on the web-based medium allows students to have more control over their practice and to receive reinforcement that can help them to build intrinsic motivation and improve their confidence (Middleton & Spanias, 1999; Nguyen & Kulm, 2005).
Several studies have shown that students who used computer-based learning practice find mathematics more enjoyable. They like the flexible features provided by the computer practice, spend long hours at a computer to complete a task, and enjoy testing out new ideas on a computer (Galbraith & Haines, 1998; Anderson, 1995; Chi, Lewis, Reiman, & Glaser, 1989; Reif, 1987). Galbraith and Haines (1998), and Pemberton (1996) agreed that computer and web-based applications increased students' level of confidence, motivation, engagement, and interaction.
Web-based assessment has the potential to meet the authentic assessment standards defined by Kulm (1994). Those authentic assessment tasks can serve the following objectives: (a) improvement of instruction and learning; (b) feedback for the students, providing information to aid them in seeing inappropriate strategies, thinking, or habits; and (c) improvement of attitudes toward mathematics (Kulm, 1994, p. 4). Previous research on the integration of web-based technology into mathematics teaching and learning has shown promising and positive effects. The web-based technology is emerging as one of the best media of user interface and instructional delivery and assessment. The web-based assessment, according to Allen (2001) constitutes an integral part of the curriculum and learning process.
Limitations of Web-Based Technology in Assessment
This study applied a quasi-experimental design and combinations of quantitative and qualitative data collection and analyses. The instruments of the study contained four homework practice sets adapted from the Connected Mathematics series (Lappan, Fey, Fitzgerald, Friel, & Phillips, 1998) with randomized items and instant feedback, pre and post written surveys and interview questionnaires. Participants were 74 seventh graders, 44 males and 30 females, from a middle school in the Southern Texas. Students were from four class periods taught by the same mathematics teacher. The racial composition was 22% Hispanic, 64% White, and 14% African American. They were randomly assigned to either the paper-and-pencil group (N=33, 44.6 %) or the web-based group (N=41, 55.4 %). Students in the web-based group (the WP group) worked with online practice tasks in their school computer lab during the math class periods. Students in the paper-and-pencil group (the TP group) worked with the conventional paper-and-pencil practice tasks in the traditional classroom with their mathematics teacher.
Chi-square tests were conducted for the comparisons among groups in terms of their gender, grade, and ethnicity. The results showed that there was no significant difference among the proportion of male and female students in both groups, [[[lambda].sup.2] (1, N = 74) = 0.597, p = .440]. Also, there existed no significant difference among the proportion of students with different grades in both groups, [[[lambda].sup.2] (1, N = 74) = 0.016, p = .898]. No significant difference was found among the proportion of students with different ethnicities in both groups, [[[lambda].sup.2] (2, N = 74) = 0.626, p = .73]. Further, chi-square tests also suggested that the two groups did not differ significantly for the variables of "whether have computers at home" [[[lambda].sup.2] (1, N = 74) = 1.317, p = .251], and "number of hours spends on computers each week" [[[lambda].sup.2] (3, N = 74) = 6.853, p = .077]. Insignificant statistical differences among the proportion of gender, grade, ethnicity, and computer using indicate that even though being randomly selected, the characteristics of the two groups were similar. With the presented evidence of groups' equivalence, it is thus appropriate for researchers to conduct comparisons between groups in terms of independent and dependent variables explored.
Homework and practice tasks. Four online homework and practice sets were designed for this study. These sets included practice tasks on fractions and decimals with randomized items, automatic grading, and immediate adapted feedback. These items were selected from the Connected Mathematics series (Lappan et al., 1998), and were examined by six middle school teachers and the Principle Investigator of the National Science Foundation--Middle School Mathematics Project (MSMP). These items were transcribed into a computer database, coded in the form of mathematical formulas with random parameters and conditional statements, and implemented into the web-based server by a mathematics educator who was also a member of the MSMP. The random parameters were chosen by selecting appropriate number range, word group, and/or size and shape type, depending upon each individual item. Before parsing into the mathematical formula, the random parameters had also passed the conditional testing, basically the procedural testing with the combination of numbers, to assure the uniform level of difficulty for each generated item. The feedback was generated upon the preset conditional statements. For adapted feedback to be generated, several predicted mistakes or errors for individual item were encoded in various mathematical categories along with suggestions, those were also in the form of mathematical formulas that took the parsing parameters, for correction. The practice tasks included both multiple-choice and short-answer items regarding fraction operations, including addition, subtraction, multiplication, division and simplification, conversion of fractions and decimals, and word problems. Mathematical skills and concepts were combined and applied in the context of word problems. A variety of mathematical procedures and concepts were included to accommodate different features and mastery skills of each student. The homework and practice tasks were arranged into two parts. The first part was the review of fraction operations and fraction and decimal conversion techniques with brief reviews shown by tables, maps, charts, and worked out examples. The second part contained 15 multiple-choice and short-answer items. Each item was accompanied with detailed feedback and solution that could be seen if the students chose to check the answer after he/she clicked on grading the item. Figures 1, 2a, and 2b show a sample homework question and feedback.
Fill in the missing numbers in the fraction addition below:
[FIGURE 1 OMITTED]
[FIGURE 2A OMITTED]
[FIGURE 2B OMITTED]
Web-based system. Each student could access the online homework with his or her assigned password. While doing homework, the student could get extra help by scrolling back to the review section (the first part of the homework as described in the homework and practice tasks section), and either doing some practices or viewing the worked out examples. Though the student did receive the feedback for the practice, the practice problems while taking the homework would not be graded. Additionally, the online homework was customized so that the student did not have the permission to view the source code. Once the homework was submitted, the student received a summarized table that showed the total score, his or her answer for each question verses the correct answer, and a check mark that indicated correct or incorrect.
Survey questionnaires. Pre-and-postsurveys were designed to collect information regarding students' computer familiarity, attitude toward mathematics, attitude toward computer use, and attitude toward the online mathematic learning and practice.
The presurvey for both WP and TP groups consisted of 20 statements and was divided into two subcategories: (a) Attitude toward mathematics with 10 statements adapted from the Aiken Mathematics Attitude Scales (Aiken & Dreger, 1961) and the Revised Fennema-Sherman Attitude Scales (Fennema & Sherman, 1976) aimed at measuring students' self-perception in mathematics learning, and their motivation and interests in mathematics problem solving; and (b) Attitude toward computer usage with 10 statements adapted from the Instrument for Assessing Educator Progress in Technology Integration from the University of North Texas, aimed at checking students' level of interests and familiarity in computer use (Knezek, Rhonda, Miyashita, & Ropp, 2002). A 5-point Likert scale ranging from strongly agree-5, agree-4, netural-3, disagree-2; strongly disagree-1 was used to record students' responses.
The postsurvey for the TP group included 10 statements, excerpted from the first subcategory in the presurvey questionnaire, which checked students' postattitude toward mathematics. The postsurvey for the WP group contained 25 statement questions: (a) 10 statements, that were identical with the statements for the TP group, aimed at measuring students' postattitude toward mathematics; and (b) 15 statements to capture students' postattitude toward online mathematic learning and practice, which focused on eliciting participants' evaluation, perceptions and opinions regarding the effects of web-based mathematics assessment, scoring, feedback and practice features on their learning.
Interviews. There were 10-minute face-to-face interviews with 12 selected students from the WP group on the last day of the study. These 12 students were randomly selected from different gender and ethnic groups to assure the collection of diverse opinions. The purpose of the interview was to further explore the effects of online assessment on students' attitude and motivation. The interview questions were an oral repeat of the postsurvey statements to probe areas that might be practically difficult for students to articulate in the written form.
Students participated in this study 30 minutes each day and three times a week. The study lasted for three weeks. Prior to the study, all students were asked to complete the presurvey. During the study, both WP and TP groups practiced four different sets of homework as described in the homework and practice tasks section. The WP group practiced the homework in the computer lab. They had an option to check if they got the correct or incorrect answer for each question. They also received immediate feedback for each incorrect answer and the total score when they finished each homework set. Because of the random feature, students were allowed to practice as many times as they wished. These students were informed that the highest practice score would be recorded and reported to the teacher. Every time students repeated each homework set, the homework items were slightly changed by the random parameters, such as numbers, words, or choices, but the mathematics content and concept remained the same.
The TP group practiced the same homework sets as the WP group, but on printed worksheets. There were two alternative versions of each printed homework task available. Students in the TP group were encouraged (but not required) to take these alternative ones for extra practice. If students had questions, they could get help from their teacher. The students' homework papers were collected, manually graded, and returned to students by their teacher. At the end of the study, the WP and TP groups both took the mathematics postsurvey.
Prior to the factor analysis of the entries in the presurvey questionnaire, the Kaiser-Meyer-Olkin Measure of Sampling Adequacy Test was used to examine whether the distribution of values was adequate for conducting factor analysis. In this case, the Kaiser designated 0.798, indicating it was middling and acceptable for factor analysis. Also, the Bartlett's Test of Sphericity presented a significant value (p< .05), meaning that these data did not produce an identity matrix and were thus approximately multivariate normal and acceptable for factor analysis.
Similarly, the Kaiser-Meyer-Olkin Measure of Sampling Adequacy Test was performed to examine whether the distribution of values in the postsurvey questions was adequate for conducting factor analysis. It turned out that the Kaiser was 0.702; the Bartlett's Test of Sphericity also generated a significant value (p< .05), indicating that these data were nearly multivariate normal and acceptable for factor analysis.
Accordingly, exploratory factor analyses were performed on the 10 items of the Attitude toward Mathematics Scale, 10 items of the Attitude toward Computer Usage, and 15 items of the Attitude toward Web-Based Mathematic Learning and Practice, based on the principal axis technique with a varimax rotation (Norusis, 1986). Further, the following criteria: (a) computation of the percentage of variance extracted, and (b) performance of a Scree test (Comrey & Lee, 1992) were used to determine the optimum factor solution.
Descriptive statistics were performed on all variables, including means and standard deviation. Prior to the study, a multivariate analysis of variance (MANOVA) was used to determine differences between the mean scores of attitudes toward mathematics learning and computer usage by group. After the treatment, MANOVA was conducted to determine differences between the mean scores of attitudes toward mathematics learning by group. For the WP group, a MANOVA was computed to determine the differences between the mean scores of students' attitudes toward online practice by gender and ethnicity (African American, Hispanic and White). The alpha level was set at 0.05. Data was analyzed by using SPSS 11.0 for Windows.
In addition to the quantitative data analysis, the content analyses approaches, as described by Emerson, Fretz, and Shaw (1995), were applied to the researchers' journal entries and interview notes. During and upon completion of data collections, the two-phase process of content analyses, open coding and focused coding, was used to analyze the data to profoundly explicate the effects of online practice on students' mathematics learning attitudes.
Factor Analysis of Presurvey Questionnaire for all Participants
The presurvey questionnaire for both WP and TP groups originally consisted of 20 statements with 10 items on Attitude toward Mathematics, and another 10 items on Attitude toward Computer Usage. With the examination of results in factor analysis and reliability test, two inappropriate items were removed. In the end, the presurvey items consisted of 18 questions with 9 items on Attitude toward Mathematics, and another 9 items on Attitude toward Computer Usage.
The examination of the Scree plot and the eigenvalues depicted three factors underlay the presurvey questionnaire. The first factor was associated with 30.78 % of the total common variance, the second with 19.44 % of the total variance before rotation, the third with 9.28 % of the total variance before rotation; these three factors together being associated with 59.50 % of the total common variance. Subsequent factors did not explain more than 5.5% of the total variance alone. Investigation of the rotated factor loadings further revealed that items were weighted high on one of three factors: (a) the value of computer usage in learning, (b) positive affection of doing mathematics, and (c) the value of studying mathematics. Table 1 and Table 2 showed the component (factor) transformation matrix and rotated component matrix of the presurvey items.
Additionally, to validate the reliability of the presurvey, coefficient alpha for each factor was computed and gained the value of [alpha] to be 0.87, 0.85, and 0.74 respectively.
Factor Analysis of Postsurvey Questionnaire for the WP Group
The postsurvey questionnaire for the WP group originally contained 25 statements with 10 items on Attitude toward Mathematics, and 15 items in Attitude toward Web-Based Mathematic Learning and Assessment. With the examination of results in factor analysis and reliability test, six inappropriate items was removed. In the end, the postsurvey items consisted of 19 questions with 6 items on Attitude toward Mathematics, and 13 items on Attitude toward Web-Based Mathematic Learning and Assessment.
The examination of the scree plot and the eigenvalues depicted three factors underlay the postsurvey questionnaire. The first factor was associated with 43.23% of the total common variance, the second with 19.98% of the total variance before rotation, the third with 5.65% of the total variance before rotation; these three factors together being associated with 68.86% of the total common variance. Subsequent factors did not explain more than 5% of the total variance alone. Investigation of the rotated factor loadings further revealed that items were weighted high on one of three factors: (a) the general perception of learning math on computers, (b) positive affection of doing mathematics, and (c) the value of web-based mathematic instruction. Table 3 and Table 4 show the component (factor) transformation matrix and rotated component matrix of the postsurvey items for the WP group. Additionally, to validate the reliability of the presurvey, coefficient alpha for each factor was computed and gained the value of to be 0.93, 0.89 and 0.82, respectively.
GROUP COMPARISONS IN ATTITUDES TOWARD MATHEMATICS LEARNING AND COMPUTER USAGE
Prior to the Study
A multivariate analysis of variance (MANOVA) was computed to determine differences between the mean scores of attitudes toward mathematics learning and computer usage by group prior to the study. There was no significant difference between the WP and TP groups for the attitude variables in the presurvey (p value ranging from .065 to .965).
Group Comparisons in Attitudes toward Mathematics Learning After the Treatment
The descriptive statistics results showed that in the TP group, there were 15 (46%) females and 18 (54%) males, and 23 (70%) White, 6 (18%) Hispanic and 4 (12%) African American. In the WP group, there were 15 (37%) females and 26 (63%) males, and 25 (61%) White, 10 (24%) Hispanic and 6 (15%) African American. The multivariate analysis of variance (MANOVA) was conducted to determine differences between the mean scores of attitudes toward mathematics learning after the treatment. The results showed that there were no significant differences between groups, and no significant interaction between group and gender in the designated six dependent variables.
However, after the examination of the descriptive statistics, it was found that after the treatment while responding to the statement of "I am sure of myself when I do math," male students in the WP group (M=3.6, SD=0.21) yielded apparently higher scores than the males in the TP group (M=3.2, SD=0.25); yet, the WP group of female students' average rating score remained similar to the females in the TP group (M=3.2, SD=0.27).
Further, when responding to the statement of "I can do well in math," male students in the WP group (M=4.0, SD=0.19) yielded higher scores than the males in the TP group (M=3.7, SD=0.23); yet, the WP group of female students' average score showed lower score (M=3.6, SD=0.25) compared to the females in the TP group (M=3.8, SD=0.25).
Similarly, when asked to respond to the statement of "I think I could handle more difficult math," male students in the WP group (M=3.4, SD=0.23) yielded seemingly higher scores than the males in the TP group (M=2.83, SD=0.28); however, the WP group of female students' showed lower score (M=2.7, SD=0.31) compared to the females in the TP group (M=2.9, SD=0.31).
Concerning the statement of "I enjoy math problem solving," it was found that male students in the TP group (M=2.8, SD=0.28) yielded lower mean scores than the females in that group (M=3.1, SD=0.3). But in the WP group, male students (M=3.6, SD=0.23) had higher mean score than the females' (M=3.2, SD=0.3). Additionally, male students in the WP group (M=3.6, SD=0.23) yielded higher scores than the males in the TP group (M=2.8, SD=0.28); the WP group of female students' also showed higher score (M=3.3, SD=0.3) compared to the females in the TP group (M=3.1, SD=0.3).
Moreover, it was surprising to find that when asked to rate the statement of "I study math because I know how useful it is," males and females students in the WP group ([M.sub.male]=3.9, [SD.sub.male]=0.19; [M.sub.female]=3.7, [SD.sub.female]=0.25) both had lower mean scores than those in the TP group ([M.sub.male]=4.2, [SD.sub.male]=0.23; [M.sub.female]=3.8, [SD.sub.female]=0.25).
The WP Group' Attitudes Toward Web-Based Mathematic Learning and Assessment in Terms of Different Gender and Ethnicity
A multivariate analysis of variance (MANOVA) was computed to determine differences by gender and ethnicity for the 19 attitude variables shown in the postsurvey for the WP group after the treatment. The statistic result showed that there was a significant interaction between gender and ethnicity (F(20, 17) = 2.741, p< .05) (Table 5). In tests between-subjects effects by gender and ethnicity, it was found that there was a significant main effect by ethnicity in the attitude toward the dependent variable "I am sure of myself when I do math."
Further, the post hoc tests indicated that White and Hispanic students had significant difference in rating the statement of "I am sure of myself when I do math." The mean score yielded by the White students was 3.16 (SD=1.03) whereas the Hispanic students produced a significantly higher score of 4.20 (SD=0.79). Another significant difference existed between the White and Hispanic students in the dependent variable of "I think I could handle more difficult math." The mean score yielded by the White students was 2.84 (SD=1.21); on the contrary, the Hispanic students had a higher mean score of 4.10 (SD=0.88).
There was also a significant univariate interaction of gender by ethnicity on the following dependent variables: "I like to receive immediate scores on my math homework and quizzes from the computer," "Computer immediate feedback is useful for mathematics problem solving," "I have less anxiety in taking computer-based quizzes than paper-and-pencil quizzes," and "Computer-based math quizzes with immediate scoring help me evaluate my own understanding and performance" (Table 6).
The descriptive statistical results indicated that when asked to measure their attitude toward the statement of "I like to receive immediate scores on my math homework and quizzes from the computer," White female students (M=4.92, SD=0.29) produced higher scores than the African American females (M=4.00, SD=1.73). Conversely, White male students (M=4.69, SD=0.63) yielded lower scores than the African American students (M=5.0, SD=0).
Moreover, when asked to rate their attitude toward the statement of "Computer immediate feedback is useful for mathematics problem solving," White female students (M=4.42, SD=0.51) produced higher scores than that of the African American females (M=3.00, SD=0). However, Hispanic male students (M=4.20, SD=0.92) yielded the higher score compared to the African American (M=4.00, SD=0), and the White (M=3.69, SD=0.48) male students. While responding to the measurement of the attitude toward the statement of "I have less anxiety in taking computer-based quizzes than paper-and-pencil quizzes," White female students (M=4.83, SD=0.39) produced higher scores than that of the African American females (M=4.00, SD=0). Yet, African American males (M=5.00, SD=0) had the higher score than that of the White (M=4.77, SD=0.6), and the Hispanic (M=4.5, SD=0.85) male peers.
About the measure of WP students' attitude toward the statement of "Computer-based math quizzes with immediate scoring helping me evaluate my own understanding and performance," the result showed that White female students (M=4.50, SD=0.52) produced higher scores than that of the African American females (M=3.67, SD=0.58). But African American males (M=4.67, SD=0.58) generally produced the higher score than that of the White males (M=4.00, SD=0.82), and of the Hispanic males (M=4.40, SD=0.84).
Interviews were conducted with 12 students from the WP group on the last day of the study. The interview lasted 10 minutes for each student. Students were asked to further explicate their attitudes toward mathematics, attitudes toward using computers in learning, and perception of computer-based mathematics instruction. The interview was an informal conversation, which provided students opportunities to explain their written answers as well as to orally express their feelings about web-based mathematics learning and how the web-based instruction affected their mathematics achievement. Table 8 summarizes the interviewed students' demographic data.
The interviewed students' attitudes toward mathematics learning. Six out of seven male students believed that they liked mathematics courses. One White and one Hispanic male student said that they liked mathematics sometimes because they thought mathematics was challenging; nevertheless, it made them think harder. Five males perceived that they could do well in mathematics, even though they were not completely sure of their ability to do math problem solving. One White male student expressed that he did not like mathematics because it was unlikely for him to get an A in the math course even when he did his very best. He did not understand why every time when he checked the answers carefully before turning in his homework or tests, he still had the wrong answers and did not receive positive results. He was regarded as a careful and diligent student by the researcher during the time he was performing web-based tasks in the computer lab.
All of the male students believed that mathematics was useful and important for their future career. However, the majority of them did not know how mathematics learning could benefit them. One of the White male students said that mathematics would help him pursue engineering-related career. One African American male indicated that mathematics would help him to get into a college for a higher degree.
On the other hand, four out of the five female students said that they liked mathematics somewhat; and they knew that they would have to use mathematics a lot in the future; but they were unclear about the value of learning mathematics and its future application. One of the female students said that she liked mathematics because she liked to challenge herself to solve math problems. She believed that she could do well in math and was confident to solve math problems. One of the Hispanic female responded that her parents said that mathematics was good for her future; but she did not believe that she could do well in mathematics. One African American female expressed that she did not like mathematics because she usually got low scores in mathematics, and that she was not confident in doing mathematics.
The interviewed students' attitudes toward using computers in learning mathematics. This section summarized six males and five females' perceptions on advantages and disadvantages of using computer in learning. Generally, these students believed that computer supported their daily learning tasks. For example, they enjoyed typing papers, sending e-mails, playing games, listening to the music, and doing homework and practice questions on computers. Further, they believed that computers gave them opportunities to learn many new things, such as finding information about how to create animation pictures, getting information about nature, history, and movies. They could also use computers to convert units, find answers for their homework questions, checking weathers, print lyrics of some popular songs, and do some calculations with online calculators.
Also, they found that working with the computer made the learning process more enjoyable and stimulating. Lessons on the computer were believed to be more fun and easy to read because these lessons were more colorful, and highlighted with more tables and charts. The computer made it easier to type and erase when preparing their papers.
Nevertheless, one White male student held an attitude against computers use. He stated that "I don't like computer math. I don't learn anything from the computer. Computer math gives no help and no interaction. It's hard to work with the computer." He also said that he preferred to do math with paper-and-pencil, and had somebody sit next to him telling him how to do the work. He preferred human interactions; he believed that the computer was not smart enough to provide intensive and detailed feedback on how to solve problems. He also added that his parents did not want him to spend much time on the Internet, so he did not prefer using computers in learning.
The interviewed students' perception and evaluation on computer-supported mathematics instruction. An African American male student expressed that he enjoyed computer math because computer math made him feel that he was smarter and made the problem-solving process easier. He believed that he could get high scores in math if he was given more time to practice on the computer. Another African American male said that he liked computer math because it was exciting and challenging; but he did not think that computer homework could benefit his learning because he did not have computer access at home; therefore it was impossible for him to practice math questions on the computer at home.
Two Hispanic male students said that "computer math gives more clues, gives more information and more practice." They liked doing math on the computer because it was easy to type and erase. It was easy to identify the right or wrong answers; and it was exciting to receive immediate scores. They believed that computer math made them like mathematics more; and practicing mathematics on the computer would motivate them to learn more.
Moreover, two White males believed that the computer made mathematics easier because it offered more instruction, more examples, which were more readable. One of them liked the check button on the computer because it gave him an opportunity to correct his answers. Yet, he did not believe that computer math could increase his achievement scores because he could not ask questions on the computer. The other White male liked the computer scoring system because it told him how good or bad his performance was, and it allowed him to retake the homework in order to improve his scores.
On the other hand, five female students all believed that computer math was much more interesting than paper-and-pencil math since computers made learning fun; and computer provided examples, scoring, and questions solutions to help them learn and review at the same time. One African American female particularly expressed that computer math made her understand fractions better; and she liked the short answer homework, in which she could fill numbers in the box and solve each problem step-by-step. She also thought that computers made math problems look easier and clearer. Another Hispanic female student said that this was her first time to do math on the computer, and that she really liked it. She asked if she could have the website of the computer math course, so she could have more practices at home. She preferred to practice online and receive the solutions and scores immediately after finishing her assignments.
Students' Attitude Toward Mathematics Learning and Web-Based Assessment
The analysis of multivariate analysis (MANOVA) indicates that there was no significant difference between the two groups in students' preattitude toward mathematics learning. After three weeks of receiving treatments, statistic results showed that no significant difference between group and gender existed with respect to students' postattitude toward mathematics learning. Yet, the TP group students' postattitude toward mathematics remained the same, while the overall attitudes of WP students showed some improvement.
Further, the descriptive analysis and the interview results indicated that the WP students were generally enthusiastic about their web-based learning experience. Many of WP students reported that they enjoyed working with computer assessment, and preferred to have more computer math practice. This finding was consistent with the results reported in prior studies such as the studies by O'Callaghan (1998), and by Gretes and Green (2000).
In addition, after participating in the web-based mathematics assessment and practice, Hispanic students tended to be more confident with themselves in doing math than their White peers. Those Hispanic students demonstrated a stronger positive perception in believing themselves to be able to handle more difficult math than the Whites. Further, males in the WP group generally had more positive attitudes toward statements of "I am sure of myself when I do math," "I can do well in math," "I think I could handle more difficult math," "I enjoy math problem solving," than the males in the TP group.
Nevertheless, the WP group of females showed less positive attitudes toward the statements of "I can do well in math," and "I think I could handle more difficult math," than females in the TP group after the treatment. Seventh-grade White females were found to have much stronger agreements than their African American females on the statements regarding the usefulness of web-based immediate scores on their homework, the usefulness of web-based immediate feedback for problem solving, the feature of web-based practice to mitigate anxiety, and the web-based immediate scoring for helping their understanding and performance.
Comparisons Between the Web-Based and Paper-and-Pencil Drill and Practice
Before getting into an indepth discussion of the impacts of the web-based technology on students' learning, it was essential to summarize students' experience in computer using and perception. The data shown in this study indicated that the 21st century is no longer a "penmanship" century in the United States, which may not be surprising for most researchers. Even though the school participating in this study was located in the rural areas of Southern Texas, and the majority of the students were from low income families, 67% of students reported that they had computers at home, and 75% reported that they spent at least one hour on the computer per week. This finding matched with the report that "about two-thirds of all children 3 to 17 years of age lived in a household with a computer, and about one-third of all children used the Internet at home" by the National Postsecondary Education Cooperative (NPEC, [Phipps, 2004]). It is also true that schools with higher minority enrollments are provided more computer access than schools with lower minority enrollment (Phipps, 2004). Therefore, there should be no differences in computer using familiarity across minority and White students. The data in this study demonstrated that there was no difference in computer using attitude across all subgroups prior to the study: male and female; or African American, Hispanic, and White. It was shown that a majority of participants either agreed or strongly agreed that they enjoyed doing work on the computer and they preferred to have more lessons on the computer. Even though female and White students had the lowest minimum scores on the attitude toward computer using, the majority of students believed that it was important for them to learn how to use a computer on every learning subject.
In addition to students' computer using attitudes, it was essential to know that during the three weeks of working with web-based assessment and practice with the requirement of being familiar with the keyboard and web browser, there were no reports related to students' difficulties in computer and Internet manipulation. It was probably surprising to some researchers, but it was documented that "children and teenagers use computers and the Internet more than any other age group with 90% of children between the ages of 5 and 17 now using a computer" (Phipps, 2004). The interview notes indicated that students perceived lessons and assignments on the computer to be clearer to read, and that the mathematics problems on the computer appeared to be easier than on printed papers. Students also enjoyed the computer affordability; that is, easy to type and easy to erase. This finding was consistent with the results of comparison of computer-writing and paper-and-pencil writing in the study conducted by Russell in 1999.
Extending beyond the results from some previous studies such as studies by Russell and Haney in 1997, and by Zandvliet and Farragher in 1997, that considered the computer as merely a "glorified typewriter," this study showed that computer-based or web-based assessment and practice had positive and extraneous effects on students' mathematical learning processes. At the end of the study, students enthusiastically expressed that they felt smarter while doing mathematics on the computer, and gained more confidence once provided with opportunities for multiple practices and receiving immediate feedback for improvement. These effects reached beyond what paper-and-pencil could do for all students.
Web-based assessment and practice provided students with immediate feedback and automated scores that led students to have more control over their work and their effort. The students' survey results related to their perception and evaluation of the web-based assessment indicated that the immediate feedback and instant scoring appeared to be the most attractive features of the web-based learning. These features might also affect the students' success or failure in mathematics learning. This finding reinforced the presumption that students highly desire confirmation of their understanding and knowing their performance. Students need to recognize their mistakes as early as possible, so that they can have the chance to correct and adjust their understanding before they start to forget how they made those mistakes. The immediate feedback, even though for some particular questions, only telling students whether their answer is correct or incorrect, is of importance to let students recognize their misunderstandings and identify the learning areas that needed to spend more time and effort for improvement.
With the immediate feedback and instant scoring, the web-based assessment and practice not only plays the role of measurement or evaluation, but it also plays the role of instruction, reflection and reinforcement (Bransford, Brown, & Cocking, 1999; Thorndike, 1913). Students who take web-based assessment have opportunities to show their understanding; additionally, they can learn from their responses or mistakes to clarify, review, and reconfirm previous concepts, and finally integrate complex mathematics concepts from the multiple practicing.
Web-based assessment and practice offered students multiple practice opportunities that eventually encouraged students to spend more time on tasks and attain higher levels of achievement. The feature of the randomized item generation provided by the web-based assessment system offered students multiple versions of each homework set. Students could practice as many versions or as many times as possible when it is necessary. Along with the immediate feedback and automated scoring, the automatic item generation encouraged students (a) who highly desire a perfect performance, having an opportunity to reach the maximum scores, and (b) who are not confident with their understanding, confirming and reconfirming the mathematical concepts and procedures.
Even though some critics indicate that the benefits of drill-and-practice are only short-term memory (Carpenter & Lehrer, 1999), long-term retention can be gradually built if students are interested in their practicing, experienced with different setup items, and if they receive "score" awards for their persistence. Based on the results of this study, it was found that when students engaged in the web-based assessment and practice, they were willing to spend more time on tasks to gain understanding and to strive for better achievement.
Web-based assessment and practice improved students' confidence in math problem solving, reduced their anxiety in test taking, and motivated them to learn mathematics. "Computer math makes me smarter," an African American male student said. "Computer math makes me understand fractions better," stated by an African American female student. "Computer math gives more clues, more information and more practice," a Hispanic male student indicated. "Computer math gives more fun to learn," said a White male student.
Along with the ultimate positive evaluation and perception of web-based assessment on the written survey, these statements "Computer math makes me smarter," "Computer math makes me understand fractions better," "Computer math gives more clues, more information and more practice," and "Computer math gives more fun to learn" indicated that the WP group was very enthusiastic about their learning experience. Web-based assessment not only provided students with more practice, but also helped them build better self-confidence, self-efficacy, and self-motivation. Additional information from the survey questionnaire and interview results also demonstrated that web-based assessment with features of immediate feedback, clear instruction, and instant scoring gave students better guidance to direct their learning.
Also, web-based mathematics assessment and practice enabled participants, who had a high level of anxiety in learning math (several of the students were math phobic), to recognize their scores instantly; thus, they might have more control over their learning, and felt less anxious about doing math. It was apparent that the web-based multiple practices fostered students' self-assessment and self-regulation in which students could assess what they already know and what they need to improve, to make their knowledge base compatible with the learning goals. From this point of view, web-based assessment truly scaffolded students' learning processes (Bransford et al., 1999).
More research on the influence of web-based assessment and practice on students' performance should be conducted in different schools and at different grade levels to confirm the actual benefits of the online delivery system. In the future research, web-based assessment systems should be implemented for classroom assignments and practices, and should be assigned to students by their own teachers in addition to the traditional assessment over an entire school year. The measurement of the mathematics achievement and attitude should be the results of the long-term study across a diverse population (i.e., the future research should modify this present study by including more students from different ethnicity, such as Asian and Native American).
Technically, more web-based assessment tasks on various mathematical concepts and strands should be designed. The database system should be constructed to manipulate a large-scale data collection, and to generate item pools for different mathematical levels. An implementation of diagnostic systems for different mathematical strands is also suggested to enrich the automatic feedback to teachers and educators.
Aiken, L. R., & Dreger, R. M. (1961). The effects of attitudes on performance in mathematics. Journal of Educational Psychology, 52(1), 19-24.
Allen, G. D. (2001). Online calculus: The course and survey results. Computers in the Schools: The Interdisciplinary Journal of Practice, Theory, and Applied Research, 17(1/2), 17-30.
Allen, R. (1998). The web: Interactive and multimedia education. Computer Networks and ISDN Systems, 30(16-18), 1717-1727.
Anderson, J. R. (1995). Cognitive psychology and its implications, (4th ed), New York: W.H. Freeman and Company.
Bennett, R. E. (2001). How the Internet will help large-scale assessment reinvent itself. Education Policy Analysis Archives, 9(5). Retrieved on February 19, 2006, from http://epaa.asu.edu/epaa/v9n5.html
Bransford, J. D., Brown, A. L., & Cocking, R. R. (1999). How people learn. Washington, DC: National Academy Press.
Carpenter, T. P., & Lehrer, R. (1999). Teaching and learning mathematics with understanding. In E. Fennema & T. A. Romberg (Eds.), Mathematics classrooms that promote understanding (pp. 19-32). Mahwah, NJ: Lawrence Erlbaum.
Chi, M. T. H., Lewis, M. W., Reiman, P., & Glaser, R. (1989). Self-explanations: How students study and use examples in learning to solve problems. Cognitive Science, 13, 145-182.
Comrey, A. L, & Lee, H. B. (1992). A first course in factor analysis (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum.
Emerson, R. M., Fretz, R. I., & Shaw, L. L. (1995). Writing ethnographic field-notes. Chicago: University of Chicago Press.
Fennema, E., & Sherman, J. A. (1976). Fennema-Sherman mathematics attitude scales: Instrument designed to measure attitudes toward the learning of mathematics by females and males. JSAS Catalog of Selected Documents in Psychology, 6(31).
Fleischman, J. (2001, February). Approaches and strategies for assessment: Project-based learning using technology. Converge Journal, (pp. 38-40).
Galbraith, P., & Haines, C. (1998). Disentangling the Nexus: Attitudes to mathematics and technology in a computer learning environment. Educational Studies in Mathematics, 36(3), 275-290.
Gretes, J. A., & Green, M. (2000). Improving undergraduate learning with computer-assisted assessment. Journal of Research on Computing in Education, 33(1), 46-56.
Hart, L. E., & Walker, J. (1993). The role of affect in teaching and learning mathematics. Research ideas for the classroom. Reston, VA: National Council of Teachers of Mathematics.
King, T. (Ed.). (1997). Technology in the classroom: A collection of articles. Arlington Heights, IL: IRI/SkyLight Training and Publishing.
Knezek, G., Rhonda, C., Miyashita, K., Ropp, M. (2002). Instrument for assessing educator progress in technology integration. Institute for the Integration of Technology into Teaching and Learning (IITL). Retrieved February 20, 2006, from http://www.iittl.unt.edu
Kulm, G. (1994). Mathematics assessment: What works in the classroom. San Francisco: Jossey Bass.
Lappan, G., Fey, J., Fitzgerald, W. M., Friel, S. N., & Phillips, E. D. (1998). Connected mathematics. Menlo Park, CA: Addison Wesley Longman.
Liang, X., & Creasy, K. (2004). Classroom assessment in web-based instructional environment: Instructors' experience. Practical Assessment, Research & Evaluation, 9(7). Retrieved February 19, 2006, from http://PAREonline.net/getvn.asp?v=9&n=7
Linn, D. (2002). Using electronic assessment to measure student performance: Issue brief. Education policy studies division. Retrieved February 19, 2006, from http://www.nga.org/Files/pdf/ELECTRONICASSESSMENT.pdf
Lockwood, F., (2001). Innovation in distributed learning: Creating the environment. In F. Lockwood & A. Gooley (Ed.), Innovation in open and distance learning: Successful development of online and web-based learning (pp. 1-14). London: Kogan Page.
Merill, P. F., & Hammons, C. (1996). Computers in education (3rd ed.). Boston: Allyn and Bacon.
Middleton, J. A., & Spanias, P. A. (1999). Motivation for achievement in mathematics: Findings, generalization, and criticism of the research. Journal for Research in Mathematics Education, 30(1), 65-88.
Morgan, C., & O'Reilly, M. (2001). Innovations in online assessment. In F. Lockwood & A. Gooley (Eds.), Innovation in open and distance learning: Successful development of online and web-based learning (pp. 179-188). London: Kogan Page.
National Council of Teachers of Mathematics. (2000). Principle standards for school mathematics. Reston, VA: Author.
Nguyen, D. M., & Allen, G. D. (2006). Interactive mathematics QuizMaker and the online placement exams. Proceedings of the 17th International Conference on Technology in Collegiate Mathematics. Upper Saddle River, NJ: Addison-Wesley.
Nguyen, D. M., & Kulm, G. (2005). Using web-based practice to enhance mathematics learning and achievement. Journal of Interactive Online Learning (JIOL), 3(3).
Norusis, M. J. (1986). The SPSS guide to data analysis. Chicago: SPSS Inc.
O'Callaghan, B. R. (1998). Computer-intensive-algebra and students' conceptual knowledge of functions. Journal of Research of Mathematics Education, 29(1), 21-41.
Pemberton, M. R. (1996). The impact of Maple on science and engineering students. Paper presented at the Proceedings of the Second Biennial Australian Engineering Mathematics Conference, (pp. 511-519).
Phipps, R. A. (2004). How does technology affect access in postsecondary education: What do we really know? (NPEC 2004-831) Report of the National Postsecondary Education Cooperative Working Group on Access-Technology. Retrieved February 20, 2006 from http://nces.ed.gov/pubs2004/2004831.pdf
Reif, F. (1987). Interpretation of scientific or mathematical concepts: Cognitive issues and instrumental implications. Cognitive Science, 11, 395-416.
Russell, M. (1999). Testing writing on computers: A follow-up study comparing performance on computer and on paper. Educational Policy Analysis Archives, 7(20). Retrieved February 19, 2006, from http://epaa.asu.edu/epaa/v7n20/
Russell, M., & Haney, W. (1997). Testing writing on computers: An experiment comparing student performance on tests conducted via computer and via paper-and-pencil. Educational Policy Analysis Archives, 5(1). Retrieved on May 20, 2003 from: http://epaa.asu.edu/epaa/v5n3
Sanchis, G. R. (2001). Using web-forms for online assessment. Mathematics and Computer Education, 35(2), 105-113.
Steele, D. F., & Arth, A. A. (1998). Math instruction and assessment: Preventing anxiety, promoting confidence. Schools in the Middle, 7(3), 44-48.
Thorndike, E. L. (1913). Educational psychology (Vols. 1 and 2). New York: Columbia University.
Web-based Education Commission (WBEC). (2001). Retrieved on February 25, 2002 from: http://www.hpcnet.org/webcommission
Zandvliet, D., & Farragher, P. (1997). A comparison of computer-administered and written test. Educational Policy Analysis Archives, 29(4), 423-438.
DIEM M. NGUYEN
Bowling Green State University
YI-CHUAN JANE HSIEH
Ching Yun University
G. DONALD ALLEN
Texas A & M University
Table 1 Component (Factor) Transformation Matrix 1 (the value of 3 (the value of computer usage in 2 (positive affection of studying component learning) doing mathematics) mathematics) 1 .812 .472 .343 2 -.531 .841 .100 3 -.241 -.263 .934 Table 2 Rotated Component Matrix Variables Component 1 Component 2 Component 3 I enjoy doing things on a .760 computer I concentrate on a computer when .611 I use one. I would work harder if I could .695 use computers more often. I know that computers give me .584 opportunities to learn many new things. I enjoy lessons on the computer. .511 I believe that the more teachers .786 use computers, the more I will enjoy school. I feel comfortable working with a .830 computer. I think that working with a .850 computer is enjoyable and stimulating. I have a lot of self-confidence .752 when it comes to working with computers. I enjoy mathematics courses. .779 I feel at ease in mathematics. .790 I am sure of myself when I do .717 math. I know I can do well in math. .739 I think I could handle more .864 difficult math. I enjoy mathematics problem .664 solving. I will need mathematics for my .854 future work. I study math because I know how .721 useful it is. Knowing mathematics will help me .779 earn a living. Table 3 Component (Factor) Transformation Matrix 3 (the value of 1 (the general 2 (positive computer-based perception of learning affection of doing mathematic component math on computers) mathematics) instruction) 1 .801 .352 .485 2 -.415 .910 .025 3 .432 .221 -.874 Table 4 Rotated Component Matrix Variables Component 1 Component 2 Component 3 I like to do math on the .872 computer. Computer-based math tasks are .716 clear and easy to read. I like to receive immediate .875 scores on my math homework and quizzes from the computer. Immediate scores help me to be .743 aware of my performance. Computer immediate feedback helps .575 me to recognize my mistakes instantly. Computer-based math homework .599 gives me more chance to practice. Computer-based quizzes help me to .800 be less anxious in waiting for my scores. I have less anxiety in taking .633 computer-based quizzes than paper-and-pencil quizzes. Computer-based math quizzes with .677 immediate scoring help me evaluate my own understanding and performance. I like computer-based math .891 quizzes more than paper-and- pencil quizzes. I enjoy mathematics course. .781 I am sure of myself when I do .791 math. I know I can do well in math. .806 I think I would handle more .828 difficult math. I enjoy mathematics problem .836 solving. I study math because I know how .725 useful it is. I like the help and suggestions .539 on my math homework from the computer. Computer immediate feedback is .746 useful for mathematics problem solving. Computer-based mathematics .797 instruction helps me to review mathematics concepts. Table 5 Multivariate Tests Effect Value F Hypothesis df GENDER Pillai's Trace .523 .930 20.000 Wilks' Lambda .477 .930 20.000 Hotelling's Trace 1.094 .930 20.000 ETHNIC Pillai's Trace 1.058 1.010 40.000 Wilks' Lambda .213 .991 40.000 Hotelling's Trace 2.419 .968 40.000 GENDER *ETHNIC Pillai's Trace .763 2.741 20.000 Wilks' Lambda .237 2.741 20.000 Hotelling's Trace 3.224 2.741 20.000 Effect Error df Sig. Eta squared GENDER Pillai's Trace 17.000 .566 .523 Wilks' Lambda 17.000 .566 .523 Hotelling's Trace 17.000 .566 .523 ETHNIC Pillai's Trace 36.000 .490 .529 Wilks' Lambda 34.000 .515 .538 Hotelling's Trace 32.000 .544 .547 GENDER *ETHNIC Pillai's Trace 17.000 .020* .763 Wilks' Lambda 17.000 .020* .763 Hotelling's Trace 17.000 .020* .763 * p < .05 Table 6 Tests of Between-Subjects Effects Dependent Type III sum Mean Source variables of squares df squared ETHNIC ... ... ... ... I am sure of myself when I do math. 9.222 2 4.611 GENDER* ... ... ... ... ETHNIC I enjoy mathematics course. .919 1 .919 I am sure of myself when I do math. 3.257 1 3.257 I know I can do well in math. .110 1 .110 I think I would handle more .827 1 .827 difficult math. I enjoy mathematics problem .191 1 .191 solving. I study math because I know how .179 1 .179 useful it is. I like to do math on the computer. .368 1 .368 Computer-based math tasks are clear 1.088 1 1.088 and easy to read. I like to receive immediate scores 1.813 1 1.813 on my math homework and quizzes from the computer Immediate scores help me to be .879 1 .879 aware of my performance. I like the help and suggestions on 6.440E- 1 6.440E- my math homework from the computer. 02 02 Computer immediate feedback helps .704 1 .704 me to recognize my mistakes instantly. Computer immediate feedback is 3.596 1 3.596 useful for mathematics problem solving. Computer-based math homework gives .776 1 .776 me more chance to practice. Computer-based mathematics 7.558E- 1 7.558E- instruction helps me to review 02 02 mathematics concepts. Computer-based quizzes help me to .280 1 .280 be less anxious in waiting for my scores. I have less anxiety in taking 1.369 1 1.369 computer-based quizzes than paper-and-pencil quizzes. Computer-based math quizzes with 2.721 1 2.721 immediate scoring help me evaluate my own understanding and performance. I like computer-based math quizzes .272 1 .272 more than paper-and-pencil quizzes. Error ... ... ... ... Dependent Eta Source variables F Sig. squared ETHNIC ... ... ... ... I am sure of myself when I do math. 4.253 .022* 0.191 GENDER* ... ... ... ... ETHNIC I enjoy mathematics course. .973 .330 .026 I am sure of myself when I do math. 3.003 .092 .077 I know I can do well in math. .114 .738 .003 I think I would handle more .537 .468 .015 difficult math. I enjoy mathematics problem .117 .735 .003 solving. I study math because I know how .161 .691 .004 useful it is. I like to do math on the computer. .832 .368 .023 Computer-based math tasks are clear 2.331 .136 .061 and easy to read. I like to receive immediate scores 4.134 .049* .103 on my math homework and quizzes from the computer Immediate scores help me to be 1.942 .172 .051 aware of my performance. I like the help and suggestions on .116 .735 .003 my math homework from the computer. Computer immediate feedback helps 1.394 .246 .037 me to recognize my mistakes instantly. Computer immediate feedback is 8.468 .006* .190 useful for mathematics problem solving. Computer-based math homework gives 2.056 .160 .054 me more chance to practice. Computer-based mathematics .146 .705 .004 instruction helps me to review mathematics concepts. Computer-based quizzes help me to 1.007 .322 .027 be less anxious in waiting for my scores. I have less anxiety in taking 3.952 .054* .099 computer-based quizzes than paper-and-pencil quizzes. Computer-based math quizzes with 5.229 .028* .127 immediate scoring help me evaluate my own understanding and performance. I like computer-based math quizzes .852 .362 .023 more than paper-and-pencil quizzes. Error ... * p < .05 Table 8 Demographic Description of the Interviewed Students African American Hispanic White Male 2 2 3 Female 1 2 2
|Printer friendly Cite/link Email Feedback|
|Author:||Allen, G. Donald|
|Publication:||Journal of Computers in Mathematics and Science Teaching|
|Date:||Sep 22, 2006|
|Previous Article:||Computers and mathematical philosophies in educational trends.|
|Next Article:||Learning life sciences: design and development of a virtual molecular biology learning lab.|