Performance assessment in real time: Harnessing today's technology tools to support data-driven instruction.

Most of us remember the early days in our teaching careers: exciting new opportunities, bounds of energy, high expectations for ourselves and our students, and piles and piles of assignments to grade. We knew as young teachers that the best way to help our students learn was to assess their progress and assess it often. We quickly learned that this meant many nights, weekends and holidays grading and planning. We also knew that the sooner student work was returned and reviewed, the greater the opportunity for learning.

It is here that one of the greatest dilemmas in teaching is borne: How do I plan for effective instruction, individualize learning, and create exciting and engaging opportunities for students while managing the enormous amount of information required to assess the progress of each student? And more importantly, how do I adjust my teaching strategies for the individual needs of students without having accurate information immediately to drive those decisions? Today technology can profoundly change how we manage the teaching and learning process through online assessments and real-time data collection. Too often student assessment is viewed as a final analysis, a grade at the end of a unit of study that evaluates what a student knows, but has nothing to do with the future experiences they have in a classroom. By the time assessment information is available, the teacher has moved on and a learning opportunity has been lost.

Instruction often is driven by coverage of content instead of mastery of concepts. Students who master content quickly are brought through repetitive activities that do nothing to add to their repertoire of skills, while those who struggle miss the opportunity to fully master the content. And technology is used to simply manage grades on assignments and tests in order to calculate a final grade.

One of the greatest leadership challenges today is the development of an effective student assessment strategy. This strategy includes comprehensive information about student progress, analysis of assessment information so specific instructional strategies can be developed, individualized assessments and interventions, connection to district and state standards, and all of this in a time frame that is as close to real-time as possible.

The more refined and accurate the student assessment strategy, the more effective our teachers will be in focusing on specific student needs and providing them with the resources they really need. Until now, this strategy wasn't possible due to the enormous amount of information required. But today's technology used properly has potential to profoundly affect the student assessment process that moves beyond summative data for a final grade. Rather, it can inform the instructional process and provide students with learning opportunities customized for their needs.

A Vision Outlined

Imagine a classroom where students have learning opportunities customized to their own skills and abilities. A teacher introduces a new concept to the class and during the discussion she checks for understanding by posing a question. Students are able to respond with their wireless personal digital assistants and the teacher quickly adjusts the lesson for where the students are and where gaps still exist. Instead of favoring only those students who are verbally strong, she quickly gains a picture of the entire class. The data are automatically compiled in the teacher's database and classified as an in-class exercise.

After the short lesson, a group of students goes to the library to do research that will contribute to their class project. Some resources already are identified for them but they need more. As they work, they pose a question to the teacher online that will likely be answered later in the day or possibly the next day.

After finishing their work in the library, they complete a quick four-question online self-assessment on how much progress they made and a reflection of how hard they worked and why. They also review their requirements for the class and check their long-term learning plan. This information is compiled in the teacher's database and classified as reflection. They send their work to a digital locker so they can retrieve it later.

A second group works collectively with the teacher for a question-and-answer session, discussing salient points of the topic. After they are done they take time to gather their thoughts and generate possible questions.

The third group moves to another part of the room where they work through an online simulation. Data are pulled from the exercise and sent to the teacher's master database. Additional exercises are provided for the students as they master concepts, based on their ability.

When the students come back together later in the day, the teacher decides to give a quiz that counts. Students take the quiz on their personal digital assistants, and the information is compiled in the teacher's database. Students are assigned a writing task verbally, and it is sent home via e-mail. The writing assignment will be placed in the electronic inbox when complete.

At the end of the day the teacher begins analyzing the data before her. She assesses progress on the concepts and, from the analysis, draws from a number of exercises in the online curriculum resource bank. The system suggests a few activities based upon the criteria she establishes and finds a match. She e-mails parents of students in the lower quartile with some suggestions for those students. She also contacts all parents of students in the lower quartile with some suggestions for those students. She also contacts all parents electronically and asks them to do a quick assessment of their child's progress on the class project. The parent information is compiled in the database and classified as parent reflection.

As she thinks about the day, the teacher doesn't grade papers but rather dives deeply into student progress, making decisions about what activities would be most appropriate for her students. Because she already has aligned the activities in her class to the district standards, she generates a report to determine she might be over-teaching or under-teaching a particular standard for a given student. She adjusts accordingly.

Our teacher is tired. It's been a full day of assessing, teaching and planning. But the work is different. There is less grading and managing and more thinking and planning. Students are interested and engaged, the process is individualized, and she has more students than ever meeting district and state standards. Later in the week she will disaggregate her data to see if she can establish some trends among her students and prepare them for the state tests.

Data-Centric Strategies

Does this sound impossible? A dream? We don't think so. Technology tools today are more powerful, flexible and scalable than ever before. These tools coupled with strong leadership, a vision for data-driven instruction and some creativity can fundamentally change how students experience school. There is no silver bullet and no single technology tool will provide the answers. But a combination of strong technology tools will pave the way for an organization that is driven by demonstrated student learning and the individual needs of students.

In both Lake Washington (www.lkwash.wednet.edu) and Vancouver school districts (www.vansd.org), major components of a data-centric learning strategy are now in place. A great deal still needs to be done, but with current building blocks each organization is seeing fundamental shifts in how instruction is managed and delivered. The following strategies are critical to building a data-centric learning agenda.

* Develop a comprehensive student assessment philosophy and strategy.

To effectively use real-time data, an organizational shift must take place that is assessment and mastery centric. Teachers and administrators will need to change the way they think about assessment and the teaching and learning process to take full advantage of the tools now available.

Assessment should be viewed as a part of the instructional process that informs instruction instead of another thing to do that strictly summarizes performance. Real-time data should drive daily practice, linking that practice to district standards. Of the strategies outlined here, this is the most challenging. This is about changing culture and well-rooted behavior. But these changes, consistent with research on good instruction, will fundamentally change the nature of the organization.

Lake Washington and Vancouver have addressed this challenge through significant investments in technology, professional development, and the collaboration of departments that focus on real-time data and data-driven decision making. The commitment to a data-centric focus is pervasive in the organization from the classroom teacher to the superintendent.

* Invest in a scalable and flexible student assessment system with an open architecture.

The core of the data-centric learning agenda is the student assessment system. Today's assessment system needs to be considered as central to the school district as the fiscal and human resources system or the student information system. As with other core technology systems, these are complex and can be resource intensive, but they are the foundation for all other real-time data applications.

In Vancouver schools, the first step was to develop an assessment system internally using Oracle development tools. The Phase I project provides access to student performance data and can be disaggregated by several factors, but doesn't yet support real-time entry and access by the classroom teacher. Phase II of the project will address these issues as the education portal is developed with Hewlett Packard.

Lake Washington, on the other hand, has invested in a commercial product, Virtual Education by Edmin.com, which is a standards-based student assessment system. With this system teachers can correlate any class work, assignment or test to district and state standards and then disaggregate the data immediately. With their data disaggregated and analyzed, teachers can identify students who aren't meeting their standards, locate curricular resources and establish trends that will influence instructional practice. Parents and students can access data, and student work can be stored in the virtual locker. Building and district administrators can access aggregate data to help drive program decisions.

In both cases, it has been critical that the student assessment system is scalable and appropriately defines the information each user needs to inform instruction. Further, an open technical architecture is essential. This architecture ensures that data bases are compatible with one another.

* Invest in technology solutions that assess student progress and report that progress to the district student assessment system.

Technology in schools is most exciting when it saves time and when it can individualize the learning process for students. For many years systems have been available that deliver content to the individual student, and in some cases student responses will generate customized content. These are valuable systems when used appropriately as part of a thoughtful instructional process.

As school districts grapple with the implementation of systems, the obvious curricular questions must be considered. But it is more important than ever to insist that student assessment is built into the system and drives the activities for students. Secondly, the system must be able to automatically report into the core student assessment system.

In Lake Washington and Vancouver, several systems are used that have assessment built in and directly report. These include Reality Based Learning and Mindsurf Networks. Reality Based Learning (www.rblc.com) is an example of a Web-based system with built-in assessments that drive the activities for students (early literacy). In addition, this system is working with vendors like Edmin.com to ensure that users can directly upload student performance information.

Mindsurf Networks has taken another approach. Through the use of wireless handheld devices, students in a Mindsurf classroom are able to interact with the teacher providing real-time responses through their Discourse product and now their testing software. With these systems, student feedback is immediate and content delivery can be customized for the student.

* Extensive training using data to inform instruction that includes the integration of technology, assessment and curriculum.

Most teachers we know are gifted individuals. They work hard, handle a multitude of tasks and emotions in a classroom and are artful about their practice as teachers. For decades, the art of teaching has been the centerpiece of professional development, teacher evaluation and teacher education programs.

Unfortunately, the science of teaching has not been a strong focus, and most of our teachers would not claim that data analysis is one of their core skills. With a continued emphasis on standards and student performance, we must focus our training efforts on the effective use of performance data in order to drive instruction. While technology literacy is important, this training is more about understanding assessment and connecting it to instructional practice than technology.

A balance of the science of teaching with the art of teaching will provide teachers with the tools they need to function in a data-driven classroom.

* Provide access to technology tools to automate as much information as possible.

Access to technology tools has improved significantly over the past decade in schools nationwide. The student-to-computer ratio is approaching 5:1 in most states, and communities continue investing substantially in technology for students. We are finding, however, that pervasive student access continues to be a significant issue. Until students have available access to technology tools anywhere and anytime, the ability to quickly and efficiently collect electronic data will be limited.

Networkable electronic devices continue to mature and will eventually provide the ability for every student to have access. Whether a personal digital assistant, laptop computer or something else, individual access to a network device will ultimately be required to capture the potential of data-driven student systems.

Better Use

As organizations transform their beliefs and practice around the use of data to inform instruction, school districts will discover they are more efficient and more effective in meeting student needs and preparing them for the future. The gifts that teachers bring to the classroom will be more fully utilized and students will be more fully engaged because of the personalized experience schools are able to provide.

Chip Kimball is assistant superintendent for information services, Lake Washington School District, 16250 N.E. 74th St., Redmond, WA 98073. E-mail: [e-mail]. Tom Cone is assistant superintendent for educational services in the Vancouver, Wash., School District.