Editor's choice: post-testing in developmental education: a success story.

Georgia's technical colleges and state university system have entered into an agreement that guarantees that five courses will transfer from technical colleges to the university system colleges. The agreement requires that all students requiring development work successfully test out of developmental studies before entering college-level courses in English and math. As initial scores at one technical college were dismal, a committee identified problems and created solutions. Results have shown marked improvement.

**********

McCabe (2003) states that half of the students entering community colleges enroll in one or more developmental classes and, of those, only half complete these developmental studies. Kozeracki (2002) found that 55% of community colleges reported that the number of students in developmental studies has increased over the previous 5 years. Roueche and Roueche (1999) report that the majority of community colleges require that students enroll in remedial courses and require that the work be completed before students can enroll in college-level courses. But, the authors say, "We still do not know very much about the actual success of remedial programs because colleges do not evaluate them very well. They frequently collect inappropriate or poor-quality data and use inappropriate criteria for measuring effectiveness. Community colleges must invest more time and look with a more critical eye at how they evaluate their remedial programs" (p. 27). McCabe (2003) writes, "Developmental education programs must continuously evaluate results as a basis for improvement. The task of developmental education is especially difficult, and unrelenting attention to improvement is essential" (p. 39).

In fall 2001, the Georgia Department of Adult and Technical Education (GDTAE) and the Georgia university system entered into an agreement called the "mini-core agreement." The agreement guarantees that five courses (college algebra, math modeling, precalculus, composition, and rhetoric I and II) transfer from any GDTAE college to any college or university in the Georgia university system. This was the first statewide transfer agreement of any kind between the two systems. The agreement requires students in the highest level of developmental studies courses to test out using the same test(s) and test scores as used for placement (ACT's ASSET or COMPASS). Post-testing is required in ENG 098, RDG 098 and/or MAT 099 before a student may be placed in ENG 191 (Composition and Rhetoric I), MAT 190 (math modeling), and/or MAT 191 (college algebra). Some students must take English and/or reading developmental courses and the exit exam(s) prior to taking college-level English, and some students must take developmental math courses and the exit exam prior to taking college-level math. Students may be taking any combination or all three developmental course subjects and thus may need one or more exit exams prior to taking college-level courses. The post-testing establishes "consistency between exit standards for remedial courses and entry standards for college-level courses" (McCabe 2003, p. 25), something McCabe says is one of the steps to an effective developmental education program. Sawyer and Schiel (2000) have suggested that students completing developmental mathematics and writing courses have a high probability of scoring at or above the post-test cutoffs and of being permitted to enroll in the subsequent college-level courses.

Chattahoochee Technical College (CTC) has the largest enrollment among the 34 GDTAE colleges, and it has one of the largest enrollments of developmental studies students in the state. CTC was named the fastest growing community college in the nation (Barter, 2002). With 6,169 students in fall 2003, it accommodates over 1,500 (duplicate count as students may be in reading, English, and/or math courses simultaneously) students in developmental studies courses each tall.

CTC's instruction and student services personnel first implemented the mini-core agreement with little notice in the winter of 2002. Results were dismal. Using the ASSET (the timed paper and pencil ACT placement test for two-year college students), students passed the exit test for ENG 098 at a rate of 37%, RDG 098 at a rate of 38%, and MAT 099 at a rate of 51%. College personnel quickly decided to load COMPASS software (the untimed computer-adaptive version of ACT's placement test for two-year college students) in a computer lab and to provide re-testing for students. After students were offered the re-test, scores increased overall to 64% for ENG 098 and to 63% for RDG 098. The math portion of the COMPASS malfunctioned during testing, so none of the re-test scores were used; therefore, the math score remained at 51%. Tables 1 and 2 show the exit test results of the first administration. Table 3 shows the combined results for both of these administrations.

The director of Arts & Sciences (A & S) quickly put together a committee whose purpose was to determine problems and potential solutions for the less than acceptable pass rates. In addition to the director of A & S, the committee members were the director of the ACCESS center and admissions, an assessment coordinator, the registrar, the director of information technology, the lead instructors for math and English, and student services personnel from two satellite campuses.

The committee identified the following problems: the age-old struggle among instructors to "teach to the test" or not to "teach to the test," variations in curriculum, many part-time faculty teaching in developmental studies, students' test anxiety (especially on a timed test), the test environment versus the normal classroom environment, students' computer anxiety or lack of skill (some did not know what a mouse was), and instructors' beliefs that an outsider was determining whether students passed or failed their courses. Other problems identified were the possibility of holding students back from completing a diploma or degree (especially those who had no intention of transferring) and the students' anger (there was plenty displayed). Because of the late notice about implementing the agreement, the new director of A & S discovered on her third day of work--a Friday--that she was to implement the agreement beginning with the new quarter that started 3 days later. The students were very angry when they discovered during the quarter that there was to be an exit test. When a large number of students did not pass the exit tests, some threatened to picket the college and report the situation to the local news stations.

Results and Discussion

Several actions were taken as a result of the committee's work. In student services, a new testing center was created with 25 computer stations so that now only COMPASS is used for testing, flexible testing hours were offered during the regular finals period (8 a.m. to 10 p.m. "on demand"), and a third day of testing was added for re-testing only. Academic advisors were on site to change schedules as necessary (pre-registration allows students to register for courses for which they do not have the prerequisites completed; therefore some students who failed the post-test would have to alter their schedules for the next quarter). Student services also provided information regarding COMPASS test-taking preparation on the college's Web site. McCabe (2003) affirms CTC's move to testing by COMPASS only with his list of computer-adaptive test advantages. Among the advantages he mentions are "greater flexibility regarding administration, immediate results, untimed nature better for nontraditional and older students, and easier to retest, as items will be different" (p. 36).

In instruction, many changes were made. A detailed memorandum regarding the exit testing was sent to all ENG 098, RDG 098, and MAT 099 instructors prior to each term. Instructors were required to include exit test information on their syllabi. The exit test requirement was added to the catalog course descriptions for the three courses. Instructors were also asked to take the portion of the COMPASS that relates to the course they are teaching. Instructors give grades of satisfactory (S) or unsatisfactory (U) for courses, and only those students who receive a satisfactory grade are allowed to take the exit exam. They bring a written "pass" from their instructor to the testing center. Students who test and re-test but still do not exit successfully are offered tutoring (live or online), or they may audit the course the next quarter at no cost. If they passed ENG 098 and/or RDG 098 but not the exit exam, they may move on to other courses with an "ENG 191 ready" prerequisite such as sociology or psychology so as not to hold them back from taking college-level courses.

The work of the committee yielded good results. Tables 4-8 show results for four subsequent quarters.

An 85 + % pass rate is quite acceptable. However, there is still work to do. Ideally, college personnel would like 100% of students to pass the course and 100% to pass the exit exam. While perfection may not be attainable, it is that for which CTC strives. And the question remains: What of the students who did not pass the course and thus did not take the exit exam? It is important to know what happened to each of those students.

Implications for Further Research

The instruction and student services personnel are quite happy with the results of the committee's work. It is clear that this type of research can be applicable to almost any developmental studies program including courses in math, writing, and reading. Interestingly, computer-adaptive tests appear to allow students a better chance at passing since they are untimed, and they allow for more frequent re-testing since different test items are generated each time.

Other research is needed to determine whether students who successfully complete developmental studies are successful in the subsequent college-level courses and, if so, are they successful at the same rate as students placed directly into the college-level courses? Hodges (1998) reported that CTC "can with confidence continue its placement and developmental studies efforts with the knowledge that it is as successful as other similar programs.... Over 50% of the students ... who are placed in developmental or regular level courses based on ASSET test scores satisfactorily complete the developmental studies classes and earn grades of C or higher in the regular courses" (p. 64). That study was based on data from 1995. Since several years have passed, the questions should be asked: Has the success rate changed? Are students still successful in college-level classes after exiting developmental studies? Now that the college has switched to COMPASS, are the placements still accurate, and are students successful where placed?

Since the beginning of the mini-core agreement implementation, CTC has also implemented computer-assisted instruction in MAT 099. Some students still take the traditional lecture method classes but are offered the option of the MAT 099 "C" sections that do have an instructor present, but most of the instruction is done via the computer software. While the students do seem to pass the exit exam at the same rate as those in the lecture classes, it is yet to be determined if there is a difference in how successful they are in the subsequent college-level courses of college algebra and math modeling. The results of such a study would be most helpful in determining the best methodologies for the transfer of learning among developmental studies students.

Table 1
Exit Test Results for Winter 2002 (ASSET)

Course    # who tested   # who passed   % who passed

ENG 098       173             64             37
RDG 098       126             48             38
MAT 099       107             55             51

Table 2
Exit Re-Test Results for Winter 2002 (COMPASS)

Course    # who tested   # who passed   % who passed

ENG 098        77          46             60
RDG 098        58          32             55
MAT 099        16           0 (a)          0 (a)

(a) Note. The math portion of the COMPASS malfunctioned
during testing, so none of the re-test scores were used.

Table 3
Combined Exit Test and Re-Test Results for Winter 2002
(ASSET & COMPASS)

Course    # who tested / re-tested   # who passed   % who passed

ENG 098           173 / 77               110           64
RDG 098           126 / 58                80           63
MAT 099           107 / 16                55           51 (a)

(a) Note. The math portion of the COMPASS malfunctioned during
testing, so none of the re-test scores were used; therefore,
the math score remained at 51%.

Table 4
Exit Test Results for Fall 2002

          # who passed   # who       # who         % who
Course       class       tested   passed test   passed test

ENG 098       219         146         125           86
RDG 098       115          90          83           92
MAT 099       115          94          79           84

Table 5
Exit Test Results for Winter 2003

          # who passed   # who    # who passed   % who passed
Course       class       tested       test           test

ENG 098       209         202         179             89
RDG 098       119         115         108             94
MAT 099       115          94          79             84

Table 6
Exit Test Results for Spring 2003

          # who passed   # who    # who passed   % who passed
Course       class       tested       test           test

ENG 098       139         132         106             80
RDG 098       127         101          91             90
MAT 099       136         113         102             90

Table 7
Exit Test Results for Summer 2003

          # who passed   # who    # who passed   % who passed
Course       class       tested       test           test

ENG 098        68          62          56             90
RDG 098        45          40          40            100
MAT 099        63          62          59             95

Table 8
Exit Test Results for Four Quarters Combined

          # who passed   # who    # who passed   % who passed
Course       class       tested       test           test

ENG 098        635         542         466            86
RDG 098        406         346         322            93
MAT 099        428         372         366            90

TOTALS        1469        1260        1124            89

References

Baner, S. (2002, December). With a boom and some flash. Community College Week, 15(9), 1-2.

Hodges, D. Z. (1998). Evaluating placement and developmental studies program at a technical institute: Using ACT's underprepared student follow-up report. Community College Review, 26(2), 57-66.

Kozeracki, C. A. (2002). ERIC review: Issues in developmental education. Community College Review, 29(4), 83-100.

McCabe, R. H. (2003). Yes we can! A community college guide for developing America's underprepared. Phoenix, AZ: League for Innovation in the Community College.

Roueche, J. E., & Roueche, S. D. (1999). High stakes, high performance: Making remedial education work. Washington, DC: Community College Press.

Sawyer, R., & Schiel, J. (2000). Posttesting students to assess the effectiveness of remedial instruction in college. Iowa City, IA: American College Testing.

Dawn Z. Hodges is the director of instruction of the School of Related Arts and Sciences at Chattahoochee Technical College, Marietta, Georgia. [e-mail]

Nichole H. Kennedy is the director of admissions and the ACCESS center at Chattahoochee Technical College, Marietta, Georgia. [e-mail]

* This article is published as an Editor's Choice selection. Editor's Choice articles are selected by the editorial staff of the Community College Review and have not gone through the peer review process.