A look at the new STAR: the state is moving closer to a seamless, standards-based assessment system in 2003. What will this mean for educators and students?

California has finally made the switch to a standards-based assessment system. The California Standards Tests--once referred to as the "augmented" questions--have become the main course in the Standardized Testing and Reporting (STAR) program. Beginning with schools' 2002 Base Academic Performance Index scores, reported in January 2003, CSTs will outweigh the nationally norm-referenced test in calculating the API. And, beginning in spring 2003, a new NRT (norm-referenced test) will have fewer questions and take less testing time.

In 1999 the state ranked schools and determined their eligibility for rewards or sanctions based on how well their students performed on the SAT-9, a nationally norm-referenced test not directly tied to state standards. Beginning in 2001, the role of the NRT in ranking schools has become increasingly less important.

Besides focusing more on a standards-based assessment approach, the state has also made some progress toward eliminating redundant tests and creating a seamless assessment system. In addition, the State Board of Education chose a new contractor, Educational Testing Service, to administer the CSTs and NRT in English.

State officials noted that ETS currently handles the California High School Exit Exam. Having one contractor for both, they said, gives the state an opportunity to improve test coordination and reduce testing redundancies, especially in high school testing. They also said they chose ETS because it appeared to be the contractor most capable of continuing the development of the standards-based tests.

While these underlying changes are significant, the 2003 STAR program will look and feel much the same as the 2002 program to students, teachers and district personnel.

STAR's basic ingredients and procedures remain the same

The STAR program still has three components:

* Tests based on the state's academic content standards, called California Standards Tests. In 2003, CSTs will make up more than 50 percent of the test items in the elementary grades and about 66 percent of the questions for grades 10 and 11. CSTs are multiple-choice, with the exception of a writing assessment for fourth and seventh graders.

* Norm-referenced, multiple choice tests of basic skills comparing pupil performance to a national sample. California Achievement Tests, Sixth Edition Survey (CAT/6) will be the NRT beginning in 2003.

* The Spanish Assessment of Basic Education, Second Edition (SABE/2) for Spanish-speaking English learners, which is written in Spanish and tests basic skills for Spanish speakers who have been enrolled less than 12 months in California public schools. Districts have the option of testing English learners who have been enrolled for more than 12 months. (English learners must also take the CSTs and CAT/6.)

Procedures remain similar

Administrative procedures will be similar, including test security procedures. Districts will continue to have the option of "pre-identifying" pupils by submitting their data to the testing contractor before testing begins in order to produce adhesive labels or pre-identified test booklets and answer documents.

Pupils in grades two to seven will have one multiple-choice test booklet for all tests. As in the past, students in grades eight to 11 will have separate test booklets for math and science tests because those tests are based on specific courses. However, student ethnicity will be identified by a code developed for the California School Information Services program, the state's voluntary student data system.

Testing time increases slightly

Total testing time for the CSTs and CAT/6 combined will increase slightly because of more standards-based questions and the inclusion of some field test questions. There will no longer be separate field tests at a different time of year. CSTs are not timed tests, though the state gives guidelines as to the amount of time needed by most students.

Based on those guidelines, in 2003 the shortest amount of testing time for CSTs and CAT/6 combined will be about 7-1/2 hours for second graders. The longest will be about 13 hours for grades 10 and 11. (These testing times include 10 minutes to distribute and collect each subtest, such as language arts of mathematics.)

Standards-based assessments

Norm-referenced tests generally measure a student's mastery of basic skills in comparison to a national sample of students and are typically reported as national percentiles. California's standards-based assessments, on the other hand, test students' knowledge of the state's demanding academic content standards. A score of "proficient" on those standards, state officials say, means that students are developing the skills necessary to succeed in college or the workplace.

The increased emphasis on standards-based tests could have an effect on schools' API scores. Initially the API, which was used to rank schools beginning in 1999, relied solely on the results of the Stanford-9 nationally norm-referenced test. Standards-based results were reported only as the number of items and the number of correct answers.

Not until 2002 was student performance on all CSTs reported by performance levels--far below basic, below basic, basic, proficient and advanced--with the goal of having every student perform at the proficient level of above.

The 2001 base API, which was released in January 2002, was the first time the English language arts CST was included in the API. In the 2002 base API, released in January 2003, the CSTs--in more than one subject--become the primary factor in a school's API store.

In the 2002 base API, the results of CSTs in English language arts and mathematics will count as 60 percent of the API calculation for elementary and middle schools and 36 percent of the API calculation for high schools. In addition, for the first time the CSTs in history/social studies for grades 10 and 11 will count as 20 percent in the API calculation for high schools.

The California High School Exit Exam, which covers English language arts and mathematics, is also based on state academic content standards. Results from that exam will count as 15 percent of the API calculation for high schools (10 percent for English language arts and 5 percent for math), meaning a total of 71 percent of a high school's API score will rely on standards-based test results (see chart above).

API to reflect addition of more CSTs

When the California Department of Education calculates the 2002 Base API (released in January 2003), it will take into consideration the impact of the new CSTs. If overall test scores from all the schools in the state--within elementary, middle and high school categories--either rise or fall compared to the previous year, CDE will give every school an adjustment (called a scale calibration factor) in its API store that reflects that change. In this way, the impact of the new tests will be mitigated. A similar method is used, for example, whenever new stocks are added to the Dow Jones Industrial Average.

Each year half of the CST questions are changed. In addition, a number of other changes, described below, will affect how students in the state are assessed. These changes include:

* In science and history/social science, the addition and/or elimination of new standards-based tests;

* A new alternate performance assessment for severely disabled students; and

* An effort to integrate CSTs with Golden State Exams.

Tests added: Science

In 2003 all fifth-graders will take a 20-item standards-based field test in science, in preparation for a planned official assessment in 2004. Individual results will not be reported to parents of schools, but schools will receive some information about school-level performance from the field test.

The state also plans to develop a comprehensive high school science test based on core science standards. The state must develop this test by 2007-08 to meet federal No Child Left Behind Act requirements. Currently, high school CSTs in science are subject-specific, such as biology or chemistry, and are given only to students enrolled in specific courses.

CDE is also making changes in the tests based on integrated science courses for students in grades nine toll. The State Board of Education, acting on recommendations of the CDE, the California Science Teachers Association and the Integrated Science Panel, adopted revised test "blueprints" for four integrated science courses. All four tests will include earth science, biology, chemistry and physics. These tests will be administered for the first time during spring 2003.

Tests added and eliminated: History/social science

The testing protocol for this subject will show the most change in 2003. For the first time, eighth-grade students will take a standards-based test in history/social science. The test will be an official assessment, with student and school scores reported. This test will not undergo field testing because the questions have already been found to be sound in previous tests of ninth-graders. The test will assess the history/social science standards for grades six, seven and eight.

In high school, there will be no CST for ninth-graders in history/social science because there are no history/social science standards for that grade level. (Ninth grade is an elective year for social studies, with different districts and schools offering different courses.) However, students in grades 10 and 11 will continue to take CSTs in this subject. In addition, the NRT social studies tests for grades nine to 11 will be eliminated in 2003 because they are not aligned to California curricula, according to the SBE.

New test: California Alternate Performance Assessment

Students with severe disabilities who are unable to participate in the STAR program, even with accommodations, will be tested with CAPA, an "on-demand" assessment aligned to a subset of California content standards. In the past, the state relied on students' portfolios developed as part of their Individual Education Program. In 2003 the students will instead take the CAPA test on a specific testing day. The test, which will be administered in March and April, will not be multiple-choice questions but will be open-ended, with teachers assisting in recording the answers.

Golden State Exams

For the first time, Golden State Exams will be used in combination with some of the standards-based tests. In the past, students choosing to take GSEs completed two portions: a multiple-choice section and a constructed-response section (a written essay or a written solution to a math or science problem). These voluntary tests were in addition to the required CSTs, which are multiple-choice exams.

In 2003 the GSE will be shorter. A combined CST/GSE score will be used to determine the GSE recognition, honors and high honors designation. The 11th-grade CST in English language arts will be combined with the Reading/Literature and/or Written Composition GSEs. The High School Summative Math CST will be combined with the High School Mathematics GSE. Additional CST/GSE combination tests are expected to be ready by 2006.

California also has a task force studying whether to use the merger of the CSTs and the GSEs for post-secondary placement and credit, and perhaps eventually as admissions criteria for state universities.

The CAT/6 vs. the Stanford-9

Even if the SBE had not changed contractors, the board would have adopted a new nationally norm-referenced test. The CAT/6 is published by CTB-McGraw Hill. Only the "survey form" (as opposed to the full-length version) of the CAT/6 will be used, resulting in fewer test questions. Instead, more CST questions will be asked.

Testing times across subjects for CAT/6 questions range from two hours and 20 minutes for second grade to close to three hours in high school. And no CAT/6 questions will be used to produce CST scores.

CAT/6 questions will differ from those on the Stanford-9, and the test booklets will look a little different in terms of layout and type fonts. Reporting of results will also be different. The CAT/6 percentile rankings will be based on more recent norms--2000 for CAT/6 vs. 1995 for Stanford-9.

The CAT/6 will use a single math test for grades two through eight instead of separating the test into "math procedures" and "math problem solving" subtests. Reports to students will show test scores in mathematics only. Although the CAT/6 will have a single test for English language arts, score results will be broken down into reading and language arts. CAT/6 results will also show spelling test scores in grades two through eight and science scores in grades nine to 11. The CST test results will continue to be broken into strands, such as writing strategies, reading comprehension, of exponents and logarithms.

Linking the SAT-9 and CAT/6

ETS will be completing a study to link the Stanford-9 and CAT/6 results in order to assure "maximum possible accuracy" in measuring the change in 2002-03 API scores. CDE is predicting that the study is likely to indicate how students would have scored on the CAT/6 in 2002 if they had taken it, as opposed to how they would have done on the Stanford-9 in 2003.

Administratively, there is likely to be a high degree of continuity in the transition, state officials say, because both ETS and the former test administrator use the same subcontractor for scoring and reporting of the STAR program. However, CDE says that district testing directors should not expect to receive equating tables or information that can be used to produce school of district trend data until August 2003.

Initial scores may drop

Although the linking study may be able to ensure an accurate comparison, it cannot guarantee that students' scores will not drop because of a lack of familiarity with a new test. Even changing the form of a test from the same publisher can have a negative impact on scores, warm Joan Herman, co-director of CRESST, the Center for Research on Evaluation, Standards and Student Testing.

Test scores tend to rise each successive year that the same test is given as teachers and students become more familiar with it, she says. Typically, when the new forms are introduced, scores initially drop and then begin to rise in subsequent years. Sometimes that initial drop can be dramatic.

Although CDE's adjustment would soften the impact of such a drop on a school's API store, the drop would still be reflected in individual student, school and district CAT/6 scores because they will not be adjusted.

New API scores will emphasize tests aligned with state standards

  Elementary & Middle Schools

      2001-02 API   2002-03 API

NRT       64%           40%
ELA       36%           60%

          High Schools

       2001-02 API   2002-03 API

NRT        76%           29%
CST        24%           56%
HSEE                     15%

In 2001-02, the Academic Performance Index (API) calculation included
the California Standards Test (CST) in English/language arts (ELA)
only; in 2002-03 it will include CST scores in ELA and math for all
schools, plus history/social science in high schools. In 2002-03, the
API for high schools will also include scores from the High School Exit
Exam.

California Department of Education
EdSource 10/02

Note: Table made from pie chart.

Ed Source is a not-for-profit, independent organization working to clarify complex K-12 education issues. More information is available at www.edsource.org.