Right from the start helping students pass the HSEE: students who fail to pass the high school exit exam the first time may be at risk of dropping out. New approaches to data analysis, now being used in the private sector, offer a more scientific method of targeting students for early intervention.

One of the facts of life for California public schools is the impending implementation of the High School Exit Examination. In spring 2001, many California public school ninth-grade students participated in the first administration of the California HSEE. Starting with this cohort of students, only those who receive a passing score on both the English/language arts and mathematics components will receive a high school diploma.

These students, and all those who follow, will have an opportunity in spring of their 10th-grade year, and at least three opportunities per year during the 11th and 12th grades, to pass the test. This is a fundamental change in the demands placed upon the public schools, because producing students who finish high school without a diploma is like General Motors producing cars that can't be driven. The public will be monitoring the pass rates of the schools and making judgments about the quality of public education based on these results.

Students will be making their own long-range plans based upon their success with this test. If they experience failure the first time they take the exam, and if they feel that they are unlikely to pass by the 12th grade, it may be their rational assessment that further time in high school is wasted time. It is not a stretch to predict that the High School Exit Exam will have a significant negative impact on the state's dropout rate.

As school districts grapple with the changes that are being put in place because of this exam, their best strategy may be to try to maximize the number of students who pass the exam the first time that they take it. If schools can increase the number passing initially, they can decrease the number of students who are likely to drop out.

Under California statutes, schools are required to offer and document remediation that will help students to pass the HSEE. In fact, high schools will be expected to make sure that students who have not passed the exam are enrolled in remedial classes and given additional opportunities to take the test.

The law requires that all California public high school students must be given at least four opportunities to take the exam before the end of 12th grade. By maximizing the number of students who pass the exam the first time, the school's resources will be freed to concentrate on those who have not yet achieved a passing score.

Most districts will probably try to identify students in the sixth - ninth grade who are least likely to pass the HSEE in order to target instruction and increase the initial pass rate. Over the years, districts have identified students who are "at risk" of academic failure in order to marshal their resources to improve student performance. The methods used for doing so have usually been simplistic.

Some districts identify at-risk students simply by listing all those who score below a certain score on a standardized or other test. Others may include, along with low standardized test scores, characteristics such as course-taking patterns (students in remedial classes), those who were not successful in algebra, low grades or absenteeism to choose students for early intervention. However, there is a better, more scientific method for choosing students to target.

Data mining

Decision tree analysis, a "data mining" process, is a method that offers great promise in education. Data mining methods are being used widely in private industry, but have yet to make their way into education. They are new approaches to data analysis that have only been made possible over the last decade because of the increased speed and power of computers, and the quantity and breadth of data that are stored in computer systems.

In traditional educational research, the investigator begins with a hypothesis, develops an experiment, and finds evidence to support or reject the hypothesis. In data mining, the researcher begins with the data, and looks for patterns that will lead to a conclusion.

Data mining processes are often considered to be "black box" techniques, because they are less concerned with hypothesis formation, or finding out "why things work." In some cases predictive relationships are established, but without the ability to say why the variables are related. However, in certain circumstances it is more important for the educator to find things that work, rather than worrying about why they work.

Here are some examples of applications of decision tree analysis that are currently being used in the private sector. They are probably even being used with you right now, although you may not be aware of it.

Data mining is being used to help direct marketers choose homes to target for mail solicitation. Marketers know something about the profile of people who have responded in the past to their solicitations. They can then use the technique to find other people with similar profiles. Thus, they can use their resources to focus on those people who are most likely to respond to their solicitations.

Mortgage lenders who need to judge whether families are good credit risks also use decision tree analysis. They do so by using the information they have about the families to whom they have given loans previously. They know whether or not families with certain profiles have been good credit risks in the past. They can then apply the model to prospective clients who are asking for loans. The analytic process helps to sift through the relationship of variables to identify those that make the best prediction.

Decision tree modeling

Decision tree analysis is the data mining process that has the potential to help schools choose students for targeting for early intervention. Here is the decision tree modeling process for the HSEE:

1. California public school students take the exam. Once the results are returned to the school district, it knows the scale score of all students in the English/language arts or math sections, whether they have achieved a score of 350 to pass that section, as well as their relative strengths and weaknesses in the strands.

2. For each student, the district will have a repository of information contained in the district's Student Information System. For example, most school districts' Student Information Systems may contain grades, attendance records, prior test scores, disciplinary interventions and other demographic information. Variables such as these are our "predictor variables," which we will use to establish a relationship between the variables themselves and passing or not passing the HSEE.

3. Using decision tree analysis, predictor variables are used to classify students into categories that have concentrations of students who passed or did not pass.

4. Once we have identified these relationships, we're ready to apply them to groups of younger students, for example, sixth- through ninth-graders. In this way, students who meet the profile of students who were successful or not successful are identified, by name.

5. At this point, the district has the ability to plan effective intervention strategies for students who fit the profile of those who have previously not passed the exam.

Using predictor variables

Predictor variables are used or not used according to the degree of relationship they have with the outcome variable (passing the High School Exit Exam). In this way, the strongest relationship between variables can be established, thus allowing for the best possible prediction. In the modeling process, the investigator uses the mathematical process to establish the best prediction, rather than the haphazard manner that has been used in the past.

There are a number of variables that could be related to passing the High School Exit Exam, but they can only be used if they are being captured electronically in some way. They fall into two broad categories -- "controllable," or those over which the schools may have some control or impact, and "fixed," or those that may be related to passing the test, but over which the school has no control.

The researcher needs to make a philosophical and practical decision as to the inclusion of types of variables. The inclusion of fixed variables may help to create a stronger model, but there is nothing the school district can do to affect them. On the other hand, by including only controllable variables in the model, the district can set goals to improve these areas, which may have a commensurate effect on the result on the HSEE.

Accuracy will improve over time

This modeling process is state of the art, and is the approach that the private sector would probably take in this situation. It is likely to be far more accurate, or predictive, than the approaches that a school district might normally take.

This type of modeling approach will likely become more and more accurate as more and more students take the HSEE. With a larger number of students included in the model, the data mining process can be fine-tuned.

Getting students over the bar

We have a responsibility to do all we can to get every one of our students over the bar so they will conclude their public school careers with a high school diploma that represents a high standard of quality, and one that the public will recognize as such. The future of public education in California may well depend upon our efforts.

Phil Morse is director of research and assessment for the Orange Unified School District. He may be reached at [e-mail].