A Technology Checkup.

A Plan for Assessing the Effectiveness of Technology in Your School District

As school leaders commit more time and money to their districts' technology needs, they are facing some tough questioning from parents and other taxpayers about its use.

This is particularly the case if the school district is planning to ask its constituents for additional fiscal resources through a public referendum to fund technology.

Before embarking on such an initiative, superintendents would do well to examine thoroughly the district's current state of technology. A technology audit can provide that information.

Common Device

School districts regularly conduct financial, energy, instructional, or other audits. Annually, a district undergoes an audit of its financial state and business practices. These audits serve as an integrity review of finance and payroll systems, as well as a review of the procedures for controlling access to programs and data, site security, and disaster recovery. Such audits address some of the issues related to the management of a district's information systems.

Technology audits are similar. The purpose is to conduct an external examination of how effectively school district technology resources are being planned for, used, and managed. The audit provides a snapshot of the schools' current use of technology, including an assessment of staff skills with technology-based products.

Objective View

A technology audit compares a school district's plans and expectations for technology with actual deployment and use. The audit facilitates continuous improvement by providing an objective reflection of the school district's direction in technology compared to observed activities, best practices in schools, and technology trends.

In May 1995, the Orono, Minn., School District undertook a technology audit. Superintendent Tom Mich compared the experience to an accreditation visit, pointing to the value of applying outside expertise and objectivity to his school system's use of technology. The findings, he said, are helping the district revise its plan for how it uses technology and to "determine our standing with respect to other comparable school districts."

As superintendent, Mich said the audit helped him "obtain an objective view of what my school district is doing with respect to technology so that I can better provide needed encouragement and leadership."

Our technology consortium, consisting of 45 Minnesota school districts, has developed a framework and conducted technology audits in more than 10 districts.

A technology audit addresses the following: information systems, operational environment, administrative application, systems (student, finance, human resources), technology, infrastructure (telecommunications, networks), office automation and productivity tools, teaching and learning applications, staff development, and technical support.

Each of these areas is reviewed in the context of the following questions:

* To what degree are work activities consistent with planned directions and expected outcomes?

* How are technology resources, programs, and staff managed?

* In what ways is the use of technology measured to ensure continuous improvement?

Dramatic Changes

An audit recognizes school districts face different situations, politics, and implementations of technology. Nevertheless, the technology audit uses common themes, processes, and approaches that optimize information systems and technology.

An outsider's look at technology in a school district provides a third-party view of the environment. Such a view tends to neutralize political and personal forces that can dominate technology discussions. In addition, an audit can be the springboard for leveraging change in work processes.

Daryle D. Russell, superintendent of the International School of Islamabad in Pakistan, identified a change that occurred in technology-related discussions in an article he wrote for the November 1995 issue of The International Educator.

"Faculty and colleagues have long heard me rant, rage, and vent my frustration when discussing school computer implementation fiascoes," Russell said. "I detest lengthy, longwinded meetings of shared ignorance and rambling commentary. I easily tire listening to Apple or IBM zealots argue their fanatic allegiances. My heart and blood pressure can no longer tolerate disproportionate computer budget demands, faulty planning assumptions, the quick obsolescence of last year's hardware purchases, the hurt feelings, the bruised egos and blank stares when administrators or board members voice concerns about cost effectiveness and/or program accountability."

He noted a dramatic change following the technology audit in the way his school's technology committee members interacted. "All agenda discussions are focused and extend from recommendations cited in the audit report. ... No more turf battles or blind rushes to implement whizbang ideas where debate is too often limited to what computers can do, rather than what they should do. Instead, teachers, administrators, board members, and parents now work with common purpose in a quest to reach planning consensus on all the school's short- and long-term technology needs."

Process Steps

The actual process of conducting a technology audit involves several activities. It I essential to have a clear, written understanding of what areas will be included in the technology audit, what pre-audit information materials are needed, what survey instruments will be used, what will be done during the audit team's site visits, and what will be included in the audit report. Leaders of both the audit team and the school district need to be identified to plan and coordinate the audit activities.

The following steps comprise a typical audit process:

* Agree on scope of the audit.

* Confirm activities, contract days, logistics, and financial arrangements.

* Collect pre-site visit information materials, including mission and goals of school district technology plans; curriculum incorporating technology use; inventory of technology resources; telecommunications infrastructure; staffing structure for technology; and technology training.

* Conduct surveys.

* Conduct site visits with interviews, observations, and focus groups.

* Develop a preliminary technology audit report, including description of the process, observations and issues, recommendations, and a summary of the audit results.

* Review and comment on the preliminary audit report.

* Present final audit report to school district.

Audit Tools

Our service agency uses different tools to examine various technology areas. These tools not only guide the audit interaction, but allow for summary representations of audit results.

These tools include surveys, configuration matrices, and a research-based learning model.

We developed a survey instrument, known as the "Continuous Improvement Profile for Technology in Schools," which is completed by selected school district staff members prior to the site visits. The survey results are used as a basis for focus group discussions. The instrument has 40 survey items grouped in the areas of leadership/planning, information systems, staff development! support, and teaching and learning.

TIES also has developed "Innovation Configuration Matrices" for the areas of technology infrastructure and instructional technology. The technology infrastructure matrix incorporates the components of voice, video, data, and districtwide area networking. The instructional technology matrix has the components of curriculum integration, types of use, connectivity applications, and multimedia applications.

A model for using technology in learning is described in Plugging In: Choosing and Using Educational Technology, developed by the North Central Regional Educational Laboratory.

The laboratory's model uses research-based findings about effective learning and effective use of technology and graphically relates learning and technology. It is based on the premise that the key measure of technology use is the degree to which student-engaged learning is supported.

The International Society of Technology in Education developed educational technology standards in conjunction with the National Council for Accreditation of Teacher Education. The 12 standards were used by our agency in developing a self-assessment instrument. Survey respondents indicate their level of expertise on a continuous five-point scale.

For each audit tool, results are compiled onto graphs that serve as a powerful way to visually represent and summarize audit results in the different areas. These graphs assist the school district staff in focusing its efforts on those items that will enable movement from where they are to where they want to be.

Key Question

Technology and information systems can help enable change within a school district. No longer is it a question of whether technology is used, but how effectively information systems and technology are used to achieve the school district's mission.

The technology audit is a tool to assist school leaders in answering that question.

James Sydow is director of student learning and technology for Technology and In formation Educational Services. Clark Kirkpatrick is the agency's executive director.

Recommended Resources

Here are a few resources recommended by James Sydow and Clark Kirkpatrick for readers who want to learn more about auditing technology in a school district:

* Plans and Policies for Technology in Education: A Compendium, a 1995 report from the Institute for the Transfer of Technology to Education. To order, contact National School Boards Association, 1680 Duke St., Alexandria, Va. 22314.

* Plugging In: Choosing and Using Educational Technology, a 1995 report from the Council for Educational Development and Research. To order, contact North Central Regional Educational Laboratory, 1900 Spring Road, Suite 300, Oak Brook, Ill. 60521.

* Tools for Change: Restructuring Technology in Our Schools, a 1993 guide for decision making. To order, contact Institute for Effective Educational Practice, 637-B S. Broadway, Suite 302, Boulder, Colo. 80303.