Digging deeper into data mining.

Data mining--the process of poring through various databases looking for hidden patterns and relationships--is alive and well, despite controversy raised by projects such as the Defense Advanced Research Projects Agency's (DARPA's) Terrorism Information Awareness (TIA) program. In fact, 52 government agencies are using or are planning to use data mining for projects ranging from detecting criminal activities to improving service, according to a study by the General Accounting Office (GAO).

The Department of Education mines data looking for fraud and waste, for instance, while NASA analyzes scientific and research information.

Of the 199 data-mining efforts identified by the GAO, 122 are using personal information, and many of these efforts use sensitive private-sector data such as credit reports or bank account numbers.

Opposition to government data-mining efforts related to terrorism in February 2003 led the Secretary of Defense to appoint the Technology and Privacy Advisory Committee (TAPAC) to look at Defense Department data-mining efforts. TAPAC has just issued a set of recommendations to help ensure that these efforts "do not compromise the privacy of U.S. persons."

According to a recent TAPAC report, "TIA was a flawed effort to achieve worthwhile ends. It was flawed by its perceived insensitivity to critical privacy issues, the manner in which it was presented to the public, and the lack of clarity and consistency in which it was described."

The TAPAC report goes on to say that "DARPA stumbled badly in its handling of TIA, for which the agency has paid a significant price in terms of its credibility in Congress and with the public."

These public-relations missteps aside, the TAPAC report maintains that data mining is "a vital tool in the fight against terrorism," and the study offers a dozen ways to make future programs more acceptable. Recommendations include safeguarding the privacy of U.S. citizens when using data mining; getting written authorization by an agency head to authorize the data mining; and establishing a policy-level privacy officer. The report also advises the Department of Defense to look to Congress and the President, to for example, establish a "framework of legal, technological, training, and oversight mechanisms" to protect privacy.

Privacy issues receive a lot of attention, with the committee authors discussing the meaning of privacy, constitutional and other protections of informational privacy, as well as non-U.S. privacy protections.

The privacy risks posed by government data mining are examined as well, with particular emphasis on "chilling effect and other surveillance risks," false positives and mission creep.

But the report indicates just how contentious the data-mining issue is, with two members issuing separate statements outlining their specific disagreements. Committee members William T. Coleman, Jr., a former Secretary of Transportation and Floyd Abrams, an attorney specializing in First Amendment cases sparred in these statements over the degree to which privacy should be balanced with security in light of the war on terrorism.

@ Safeguarding Privacy in the Fight Against Terrorism is available at SM Online.