Access to electronic health information for the public: analysis of fifty-three funded projects.

ABSTRACT

IN 2000 THE NATIONAL LIBRARY OF MEDICINE (NLM), a component of the National Institutes of Health, funded fifty-three consumer health outreach projects through the National Network of Libraries of Medicine (NN/LM). The goal of all projects was to improve access to electronic health information for consumers. Drawing on experience gained in the NN/LM public library pilot projects undertaken in 1998-1999, the projects involved medical and public libraries in partnership with a wide range of community organizations, including public health departments, schools, churches, and local professional associations. The projects provided training in the use of MedlinePlus and other health information resources and support for Internet access in a variety of settings. The projects used an array of approaches over an eighteen-month funding period. This article presents descriptive information about the projects, highlights common barriers, and provides an analysis of the effectiveness of methods and approaches used.

INTRODUCTION

There has been considerable research on the nature of consumer health information (CHI) and the frequency of needs for it. Reflecting on this research, Deering and Harris (1996) note that CHI encompasses patient information, which includes medical instructions and decision support information, as well as health education, self-care information, "quality-in-care information," and alternative medicine. Reviewing previous studies of the demand for CHI, Deering and Harris conclude that CHI requests account for a substantial portion of reference questions raised in public libraries. They suggest that as many as fifty-two million health-related requests are raised in public libraries every year.

Since Deering and Harris's review much attention has focused on CHI available from Internet-based resources. Based on a survey conducted in 2001, Harris Interactive (Taylor, 2001) estimates that 100 million adults in the United States consult online sources for health information. This figure represents three-quarters of adults who use online resources and approximately 47 percent of adults in the United States. Among adults who seek health information online, the average frequency is slightly more than three times per month.

Pifalo, Hollander, Henderson, DeSalvo, and Gill (1997) studied the impact of a CHI service on individuals. Surveying users of a CHI service provided by the Delaware Academy of Medicine, Pifalo and her colleagues found cognitive, affective, and behavioral impacts. The majority of respondents (94 percent) reported that they learned more about an illness or health issue. Slightly more than half (52.1 percent) reported that the information reduced anxiety. Approximately half (51.3 percent) indicated that the information led them to address questions to their health care providers.

Information about the motivations associated with seeking CHI online emerged from a random survey conducted in 2003 of users of MedlinePlus, a leading Web-based source of consumer health information. Results of this survey indicate that the most common reason for visiting the site was to find information on a specific condition (62 percent). The next most common reason for visiting MedlinePlus, to find information about a specific medication, was selected by 29 percent of survey respondents (Backus, 2003).

Libraries have developed a wide array of CHI resources, collections, and services, but Rees (1982) summarizes the essential role of libraries as coordinators of access to health information: "The role of the library has begun to emerge with some clarity. Increasingly, the library (public and hospital) is called on to coordinate health information access at a local level, utilizing the many information and educational resources available locally, regionally, and nationally" (pp. 37-38).

Interest in coordinated access to health information is evident in the collaborations of the earliest libraries in the United States. In a study of health-related collections in public libraries, Wannarka (1968) reports that Boston Public Library held the earliest such collection. Primarily the result of physicians' gifts, this collection totaled 28,604 volumes in 1864, exceeding the Boston Medical Library's collection of 20,285. In 1904 Boston Public Library transferred 21,000 volumes to the Boston Medical Library, which merged with Harvard s Medical Library in 1965 to form the Francis A. Countway Library of Medicine. Wannarka notes that the 21,000 volumes originally transferred in 1904 remain the property of Boston Public Library.

A series of papers presented fifty years ago at the annual meeting of the Medical Library Association explored issues emerging from CHI services provided by health sciences libraries. Later published in the Bulletin of the Medical Library Association, these papers consider policies related to collection access for members of the general public as well as opportunities for collaborations with public libraries. Representing the Armed Forces Medical Library, Jacqueline Chambers (1955) notes that "It is important that public and medical libraries cooperate with one another wherever this is feasible, and it is to their mutual advantage to divide the responsibilities which should be met" (p. 260).

Collaborations between public and health sciences libraries were features of a handful of projects funded by Library Services and Construction Act (LSCA) Title I grants in the late 1970s (Hollander, 1996). Among these was the Consumer Health Information Network (CHIN) project in Cambridge, Massachusetts. A collaboration of Mount Auburn Hospital and six public libraries in the surrounding community, the CHIN project provided for training of public library staff, cooperative collection development, interlibrary loans, and reference assistance (Gartenfeld, 1978).

Health sciences libraries also have collaborated successfully with community-based organizations in addition to public libraries to promote access to health information. Sligo and Jameson (2000) underscore the importance of community engagement for the successful dissemination of health information. In their report of a study of Pacific islanders' participation in cervical screening services, Sligo and Jameson note that this population "strongly favored sources of information that were mediated through their community groups" (p. 858). Community assessment and community engagement are also central ideas in Burroughs's Measuring the Difference (2000), a manual guiding the planning and evaluation of health information outreach projects.

Community engagement is a guiding theme in a number of projects sponsored by the National Library of Medicine (NLM). A project targeting Native American and Alaska Native communities is known as the Tribal Connections project. Coordinated by the University of Washington, this initiative has resulted in improvements in Internet connectivity in sixteen communities. Community assessments and participation were key to the success of this effort (Wood et al., 2003). Another NLM-sponsored project conducted by George Washington University has resulted in collaborations with a consortium of nonprofit clinics in the District of Columbia. Interventions at each clinic are determined by the needs of that clinic's staff and patients (Partners for Health Information, 2001).

Other collaborations between health sciences libraries and community-based organizations have led to a number of practical findings. With partial support from the National Library of Medicine, the University of Illinois at Chicago's Library of the Health Sciences collaborated with a variety of community-based organizations to extend access to HIV/AIDS information. In their report of the project Martin, McDaniels, Crespo, and Lanier (1997) note the importance of identifying community representatives who can serve as liaisons between the targeted community and the library. They also note the value of the library's repeated contacts and communications with the targeted community.

Also with support from the National Library of Medicine, a subsequent project conducted by the University of Illinois at Chicago focused on reducing asthma and lead poisoning among children through improved access to environmental health information resources. This project entailed collaborations with seven community-based organizations: two public schools, three community action groups, and two public health organizations. In her report of the project, Scherrer (2002) highlights the need for community assessments when planning interventions as well as a theoretical framework to inform the planning and evaluation of the project. Scherrer also notes significant cultural differences between academic health sciences centers and community-based organizations, and she underscores the importance of cultural competence among project staff.

A host of public health projects have relied on community partnerships and collaborative efforts. Notable among these is the Turning Point initiative. Funded by the Robert Wood Johnson Foundation and the W. K. Kellogg Foundation, the Turning Point initiative seeks to improve the public health system in the United States by making it more community-based and collaborative. Since its inception in 1997, Turning Point has resulted in 23 state and 41 community-level partnerships aimed at improving public health through community collaborations (Turning Point National Program Ofrice, 2003).

Collaborations such as those fostered by the Turning Point initiative bring together institutions seeking to realize a common goal that may not be attained separately. In a report of their efforts to develop a collaboration to promote breast cancer education among rural and Hispanic migrant and seasonal farmworker women, Meade and Calvo (2001) summarize a wide array of previous community-academic coalitions targeting specific health concerns. Community-academic collaborations are also a hallmark of the urban research centers funded by the Centers for Disease Control and Prevention since 1995 (Metzler et al., 2003) and have led to a growing body of research known as community-based participatory research (CBPR) (Israel, Schulz, Parker, & Becker, 2001). At the NewYork Academy of Medicine's Center for the Advancement of Collaborative Strategies in Health, Lasker and Weiss (2003) have framed a model for community partnerships and collaboration. Lasker and Weiss observe that public health concerns often "cannot be solved by any person, organization, or sector working alone" (p. 15) and propose a model of community-level problem solving that relies on empowering individuals, bridging social ties, and creating synergy among diverse participants. Green and Kreuter (1999) have also explored the ideas of social capital and community coalitions to address public health concerns, noting the complexities of power sharing among coalition participants.

NLM's efforts at improving access to health information have been informed by the related work of other libraries and public health organizations, only a fraction of which is represented in the previous summary. NLM has a long history of providing health care professionals with timely, up-to-date information. A vigorous outreach effort to health care professionals began with the 1989 NLM Board of Regents special panel report, Improving Health Professionals' Access to Information. The report encouraged NLM to develop an outreach program to reach health professionals who did not have easy access to recent scientific and biomedical information. It also noted the importance of the National Network of Libraries of Medicine (NN/LM) in helping NLM reach health professionals throughout the United States and making them aware of the resources and services available from the NLM and the NN/LM. In the five years following the Board of Regents' report, NLM sponsored approximately 300 outreach projects targeting health professionals. These projects engaged more than 500 institutions, often in close collaboration with NN/LM network members (Wallingford et al., 1996).

With the introduction of free MEDLINE searching on the Internet through PubMed in 1997, a new wave of users began to access NLM's database. The increased interest in MEDLINE searching by the general public led to the development of MedlinePlus and provided the impetus for NLM's consumer health focus to provide health information for the public. The NN/LM and its network members were again key to NLM's efforts to improve access to information, now for the general public as well as health professionals. In 1998 NLM began to explore ways to reach consumers through collaborations with public libraries and to foster partnerships between NN/LM network members and public libraries (Wood et al., 2000). From the beginning of this initiative, there was clear recognition that hospital, academic health sciences, and public libraries are important partners in reaching the public.

NLM has recognized the importance of supporting library and community partnerships in order to reach the public. In an effort to encourage and enhance community partnership building, NLM issued a request for proposals from NN/LM network members in April 1999. The request was for projects that would focus on improving electronic access to health information for a variety of groups, including consumers, underserved and minority populations, health professionals serving underserved and minority populations, public health workers, public libraries, and community-based and faith-based organizations. Up to $10,000 was available for each project being conducted by a single institution, and up to $40,000 was available for projects that entailed formal institutional collaborations. The request for proposals encouraged collaborations among NN/LM member libraries and public, state, and school libraries as well as health information resource centers and community- and faith-based organizations.

Proposals were received in July 1999 and were reviewed by multiple panels of reviewers representing health sciences libraries, health care professionals, public health workers, public libraries, state libraries, and community- and faith-based organizations. Fifty-three projects were funded. Figure 1 is a map representing the geographic distribution of the projects in thirty-four states and the District of Columbia. All projects were administered as subcontracts through the NN/LM Regional Medical Libraries and were implemented over an eighteen-month period.

METHODS

The authors relied on the quarterly and final reports of the projects as well as interviews with project directors to gain an understanding of the projects. Interviews with project directors were conducted between October 2002 and January 2003. The semi-structured telephone interviews were guided by a set of questions that addressed project activities, target populations, approaches taken, methods of publicity, the project's impacts on the target populations, and lessons learned. The interviews also provided an opportunity to verify summary information obtained from each project's reports.

The authors used NVivo software from QSR International to analyze the interview data. NVivo provided a system for coding, linking, searching, and organizing the qualitative information obtained during the interviews. A taxonomy was developed to identify categories for coding the interview documents in NVivo. This taxonomy was based on the prominent and recurring themes that emerged from a review of the data.

FINDINGS

Eleven single institutions received funding up to $10,000, and forty-two multitype projects received support of up to $40,000. Among the forty-two projects that entailed institutional collaborations, the most frequent partnerships were among academic health sciences libraries, public libraries, hospital libraries, and community-based organizations. Although many projects relied on collaborations spanning more than two types of organizations, the following list identifies the most common institutional partnerships formed as a result of the projects:

* 18 academic health sciences library-public library partnerships

* 16 hospital library-public library partnerships

* 9 academic health sciences library-community-based organization partnerships

* 8 hospital library-community-based organization partnerships

* 4 academic health sciences library-hospital library partnerships

Target Populations

Many projects targeted multiple populations in their efforts to improve access to electronic consumer health information. Thirty-eight projects worked directly with members of the general public, often targeting specific populations such as racial and ethnic minorities (9 projects) and seniors (7 projects). Projects also targeted a variety of professional populations, including public librarians (29 projects) and health professionals (13 projects). Projects targeting health professionals typically aimed to increase health professionals' awareness of resources available to support patient education.

Training

Forty-five of the fifty-three projects included training as part of the intervention. Approximately 820 training sessions were conducted across these forty-five projects, reaching an estimated 13,750 individuals. Almost all the projects that included a train-the-trainer approach highlighted the success of this approach. However, one project noted a lack of success with this approach in preparing members of support groups as trainers. In their reports and interviews, project directors emphasized the value of personal contact, site visits, and "putting a face to a name" as contributors to the success of their training and outreach efforts. In their reports and interviews, project directors pointed to a variety of other factors influencing the success of the training, including the scheduling and location of the training sessions, the materials used to support the training, and an accurate assessment of participants' computer skills.

Several project directors who provided training in public library branches and made accommodations for hands-on practice with the resources observed that these approaches were particularly effective. They also noted the importance of flexibility in the scheduling of training events and training locations. Most projects underscored the value of providing training in small-group settings in participants' own environments. Other project directors suggested, however, that participants who were trained in their own work settings were more likely to be distracted by work-related responsibilities.

Many projects discovered the importance of pretesting all translated materials and customizing the content to the needs and interests of the targeted community. This was found to be particularly useful when training specific populations. Issues related to cultural competence were particularly significant for projects that targeted non-English-speaking communities. A project targeting the hearing impaired reported that there are many medical terms for which signs are not available in American Sign Language.

Project directors discovered significant variability in trainees' computer skills, and one noted that self-reported data about computer proficiency may not be reliable. Other methods may be needed for ensuring that participants in a session are at a common level of proficiency. For members of a targeted community with limited computer proficiency, it may be necessary to provide training on basic computer skills as a foundation for training on computer-based health information resources.

Web Site Development

The development of Web sites and pages was a major component of thirty-eight projects. Among these projects, several added new Web pages to their organization's existing Web site. Project directors with dedicated information technology (IT) staff reported that this was advantageous for the development of the project's Web presence. Multiple project directors commented on the value of having staff with varied experience to help develop and create the project's Web presence. Project directors also underscored the benefit of identifying a single staff member with primary responsibility for technical difficulties.

As part of the development of a project's Web presence, directors noted the value of ensuring the availability of usage statistics. They also highlighted the value of a simple, streamlined design to enhance a site's usability. Collaborations in developing information for the project's Web presence were noted as particularly helpful. Projects that undertook usability testing reported the benefit of this effort in the site's development.

Publicity and Marketing

All projects were aware of the importance of promoting their program. More than half of the projects developed and distributed project-specific promotional materials. These products included bookmarks, information prescription pads, flyers, posters, displays, videos, and screen sweeps. Promotion through newspapers and newsletters was also popular. Participation in health fairs and exhibits was effective in thirteen projects, and "word of mouth" was considered effective in twelve projects. Word of mouth included communications by phone, staff contacts, personal contacts, and at meetings. Other methods included the use of Web sites, listservs, e-mail, intranet pages, links on local Web sites to the organization's Web site, and announcements in professional journals. Most projects employed more than one marketing approach.

Project Evaluation

A manual published by NN/LM, Measuring the Difference (Burroughs, 2000), provides a six-stage framework for planning and evaluating health information outreach projects:

1. Identify the target community and conduct a community assessment

2. Establish goals and objectives

3. Develop activities and strategies based on audience assessments

4. Establish evaluation objectives and develop methods of data collection

5. Carry out planned outreach and evaluation activities

6. Disseminate results of the evaluation

The fifty-three projects implemented a variety of approaches to evaluation, many of which correspond to components of the six-stage framework presented in Measuring the Difference. It should be noted that most of the fifty-three projects funded as part of the Access to Electronic Health Information for the Public program were planned prior to the widespread distribution of Measuring the Difference. Two project directors reported that they did not undertake a structured approach to evaluation but relied instead on informal methods and unsystematically gathered anecdotal evidence of the project's impact. Three projects conducted a structured needs assessment in tandem with their outreach activities. Many project leaders conducted some form of community assessment prior to the project as part of developing the project's funding proposal.

Surveys and measures of use were the most common strategies employed in the fifty-one projects that undertook a systematic approach to evaluation. Written surveys were administered as part of training sessions in thirty-six projects. These included training session evaluation or satisfaction surveys (thirty projects) and pre- and post-tests of participants' knowledge (thirteen projects). Some projects administered pre- and post-tests of participants' knowledge as well as training evaluation surveys.

Other projects surveyed participants through questionnaires and interviews to assess the project's impact. In nineteen projects, questionnaires were administered to participants at a time apart from a training event. Six projects that developed Web sites gathered evaluation data through an online feedback survey. Five projects surveyed participants through in-person or telephone interviews to gather evaluation data.

Web site usage statistics were gathered in ten of the thirty-eight projects that developed a Web resource. Two projects monitored the frequency of health-related questions at a library's reference desk, and one project measured patients' use of a hospital library.

Less frequently used methods of evaluation data collection included focus groups (two projects) and reviews of Web sites by content experts (two projects). Five projects also systematically gathered qualitative data, primarily anecdotes related to project impacts. Although the long-term impacts of outreach in a community may be difficult to ascertain, one project pointed to the establishment of a new branch library in the county hospital as an indicator of the project's success.

Measuring the Difference

Measuring the Difference differentiates formative evaluation, in which findings are used to refine ongoing outreach activities, from summative evaluation. While other projects may have used evaluation data to modify their outreach activities, two project directors explicitly noted in their final reports or follow-up interviews that evaluation findings were used to refine ongoing outreach activities. Both of these projects relied on data gathered from training session evaluation questionnaires for their formative evaluations.

Project evaluation may also consider whether outreach activities are sustained beyond the period of time for which external funding is available. Reflecting on the sustainability of activities, Rees (1982) observed a common pattern among CHI projects:

   There would appear to be a sequence of events in the development of
   structured CHI programs. Under the initial impetus provided by one
   or more persons, funding is secured from local, state, and/or
   federal sources for the initiation of the program. After
   successful promotion, demonstration, and marketing, the program
   is absorbed into regular library operations as a result of the
   buildup of expectations, resources, and expertise. In this
   manner, successful CHI programs "self-destruct" as they become
   part of the parent library system. The desired objective of
   funded CHI programs is, therefore, to catalyze, develop, extend,
   demonstrate, evaluate, and refine innovative services that will
   then be integrated into regular library operations (p. 38).

When interviewed, the majority of project directors indicated that project activities had continued beyond the period of funding. Ongoing activities typically include training and promotion of MedlinePlus and the project's Web site. Many of these activities are being conducted through an ongoing collaboration with other organizations. Thirty of the thirty-eight Web sites developed as part of the projects were still available at the time the interviews with project directors were conducted. Project directors responsible for twenty-one of these sites reported that they were being updated on a frequent basis.

Key Lessons Learned

The interviews with project directors provided an opportunity for them to highlight significant lessons learned during the course of their projects. The themes that emerged related to partnerships, leadership, commitment, communication, and decision making.

Involving the targeted community in planning and designing activities increases each group's investment in the project. For collaborative partnerships to be successful, it is essential to collaborate with representatives who are familiar with their organizations' constituents. Consulting with others who have collaborated with similar groups may provide useful information during the planning of a project. Including members of the target population with diverse expertise in an advisory role for planning and needs assessments can serve as not only a strategic function but as an information portal. Those projects focusing on Hispanic populations found that having Hispanic community leaders involved in the project from inception, in pretesting materials for cultural and medical appropriateness and in providing feedback, was extremely valuable to the success of the project.

Successful partnerships require an evaluation of potential partners' resources, including staff and time availability. Interest and enthusiasm, especially among an organization's leaders, are also key attributes to consider among potential partners. One project focusing on training of library staff found that the most effective strategy in implementing the project was to have the support of the library's administrators. Having administrators require participation among the staff proved to be a successful strategy for another project.

Potential project partners should also be able to devote the time and resources necessary to honor commitments that contribute to the success of the project. One project director noted that collaborations are valuable but can be time consuming. Multitype organizational collaborations may be difficult as a result of differences in missions and levels of commitment.

Project directors repeatedly noted the importance of communication in the planning and execution of their projects. E-mail did not always facilitate communication for some projects. One project targeting gay men and lesbians reported that electronic and e-mail communication was a major factor in the success of their project. Nearly all review and discussion was done electronically, saving considerable time for everyone. However, telephone and face-to-face communication worked well for several projects and were ways to ensure that communications reached appropriate individuals. Project directors also noted the benefit of frequent communication with administrators and community leaders, one of the key findings from a review of public health outreach projects conducted between 1998 and 2001 (Rambo et al., 2001).

Reflecting on their decision-making processes, a number of project directors reported that working with a diverse committee could make reaching consensus difficult. Political and bureaucratic challenges emerged in the course of multiple projects. A project to develop a regional health information network reported that collaborations are complicated because of the participation of distinct organizations accustomed to relying on different approaches. When planning how decisions will be reached, project directors remarked that it may be helpful to identify one organization as having final authority. This is often the organization with financial control.

CONCLUSIONS

Project directors reported significant benefits from their projects, in spite of minor setbacks and unexpected problems. These projects led to increased awareness of health science libraries and the recognition that medical libraries are a source for assistance in locating health information for the users of public libraries. As a result of collaborations between health science libraries and public libraries, the public library began to perceive hospital libraries and academic libraries as resources for medical information. This type of collaboration fostered linkages between the collaborating organizations, leading to the sharing of resources and reference requests. Within many organizations, institutional administrators gained a greater appreciation of the library as a source of funding and in fulfilling the community service mission of the organization. Many organizations were able to add consumer health information to their Web sites, raising the library's visibility. Several project directors reported that the project expanded personal networks that crossed professional boundaries.

Undertaking a new project within an organization that is already utilizing its staff and resources to its fullest extent is not without challenges. Some of the project directors felt that their library staff were uninterested or overwhelmed by the additional responsibility of providing consumer health information. Insufficient funding was another concern that project directors raised. Occasionally, because of underbudgeting or unexpected expenses, they were unable to complete the project with the funded amount, resulting in the lead organization having to use its own resources to accomplish the project's objectives.

The goal of the Electronic Access to Health Information projects was to improve access to electronic health information for consumers. The results of this first round of funding served as a starting point for NLM to focus special funding through the NN/LM for projects to promote electronic access for the general public. The projects reached a large number of individuals in many areas of the United States. The implementation of the projects enhanced existing partnerships and collaborations and created new ones. The participating libraries reaped positive benefits from their participation. The results convinced NLM that providing special funding for such efforts is an opportunity for NN/LM network members to expand their outreach to ensure that members of the public are aware of and have access to quality health information. The results of these efforts also highlight the need for involvement of the community in identifying, planning, and implementing activities that address their health information needs. In addition to providing funding for another round of these projects, the NN/LM also is piloting a set of Community Outreach Partnership Planning Projects, which provide funding to libraries to support intense involvement of community partners in the assessment of community information needs and the planning of community outreach projects. The hypothesis is that these planning awards will yield proposals that will have the full support of all partners involved.

Table 1. Project Institutions, Collaborations and Target Populations

                                                          Project
                                                          Leadership:
                                     Lead                 Single or
                                     Institution Types    Multiple
Lead Institution             State   ([dagger])           Institution

University of Alabama at     AL      AHSL                 Multiple
  Birmingham
University of Arizona        AZ      AHSL                 Multiple
University of Arkansas       AR      AHSL                 Multiple

Planetree Health             CA      Consumer Health      Multiple
  Resource Center                    Library

University of California     CA      AHSL                 Multiple
  at San Diego
Valley Care Foundation       CA      Hospital Library     Single
St. Francis Hospital and     CT      Hospital Library     Single
  Medical Center
University of Connecticut    CT      AHSL                 Multiple
  Stowe Library
George Washington            DC      AHSL                 Multiple
  University
Halifax Medical Center       FL      Hospital Library     Single
Tifton Tift County Public    GA      PL                   Single
  Library
Emory University School      GA      AHSL                 Multiple
  of Medicine, Health
  Sciences Center
  Library
Three Rivers AHEC            GA      AHEC                 Multiple
Hawaii Medical Library       HI      AHSL                 Multiple

Idaho State University,      ID      AHSL                 Multiple
  Idaho Health Sciences
  Library
Kootenai Medical Center      ID      Hospital Library     Multiple
  Library
Rush-Presbyterian-           IL      AHSL                 Multiple
  St. Luke's Medical
  Center
Loyola University Health     IL      AHSL                 Multiple
  Sciences Library
Trinity Medical Center,      IL      Hospital Library     Multiple
  Health Sciences
  Library
State Library of Iowa        IA      State Library        Multiple

University of Kentucky,      KY      AHSL                 Single
  Chandler Medical
  Center
Louisiana State              LA      AHSL                 Single
  University, Shreveport,
  Health Sciences Center
  Library
Maine General Medical        ME      Hospital Library     Multiple
  Center Library
Western Maryland AHEC        MD      AHEC                 Multiple

Eastern Shore Regional       MD      PL                   Multiple
  Library
University of Massachu-      MA      AHSL                 Multiple
  setts Medical School,
  Lamar Soutter Library

Massachusetts General        MA      Hospital Library     Multiple
  Hospital, Treadwell
  Library
Holland Community            MI      Hospital Library     Single
  Hospital
Mid-Missouri AHEC            MO      AHEC                 Multiple
University of Missouri,      MO      AHSL                 Single
  Columbia
Truman Medical Center        MO      Hospital Library     Single
  East
ICON: Omaha Area             NE      Library Consortium   Multiple
  Health Information
  Consortium
Las Vegas Clark County       NV      PL                   Multiple
  Library District
Highlands Regional           NJ      Library Consortium   Multiple
  Library Cooperative
University of New            NM      AHSL                 Multiple
  Mexico, Health
  Sciences Center
  Library
ViaHealth Rochester          NY      HL                   Multiple
  General Hospital
SUNY Health Science          NY      AHSC                 Multiple
  Center at Syracuse
South Central Regional       NY      Library Consortium   Multiple
  Library Council
Samaritan Medical            NY      Hospital Library     Single
  Center
University of North          NC      AHSL                 Multiple
  Carolina, Chapel
  Hill, Health Sciences
  Library
Mountain AHEC                NC      AHEC                 Multiple

Oregon Health Sciences       OR      AHSL                 Multiple
  University
University of Pittsburgh,    PA      AHSL                 Multiple
  Falk Library of the
  Health Sciences
Medical University           SC      AHSL                 Multiple
  of South Carolina,
  Libraries and Learning
  Resource Centers
Rapid City Regional          SD      Hospital Library     Multiple
  Hospital Health
  Sciences Library
University of Texas          TX      AHSL                 Multiple
  Health Science Center
  at San Antonio
University of Utah, Eccles   UT      AHSL                 Multiple
  Health Science Library
Mary Washington              VA      Hospital Library     Multiple
  Hospital, Medical
  Library
Public Health, Seattle       WA      Public Health        Single
  and King County                    Department
Yakima Valley Memorial       WA      Hospital Library     Multiple
  Hospital
University of Wisconsin,     WI      AHSL                 Multiple
  Madison
Medical College of           WI      AHSL                 Multiple
  Wisconsin
Wyoming Medical Center       WY      Hospital Library     Multiple

                             Collaborations
                             Resulting from        Specified Target
Lead Institution             Project ([dagger])    Populations

University of Alabama at     AHSL/PL               Consumers
  Birmingham
University of Arizona        AHSL/PL               Public Librarians
University of Arkansas       AHSL/PL/AHEC/SL/HL/   Public, Hospital,
                             PHD/CBO                 and Academic
                                                     Librarians; Public
                                                     Health Personnel;
                                                     Cooperative
                                                     Extension Agents;
                                                     Teachers;
                                                     University
                                                     Students
Planetree Health             Consumer Health       Minority
  Resource Center            Library/CBO/            Populations,
                             Community Health        Hispanics, Low-
                             Center                  Income
                                                     Individuals,
                                                     Health
                                                     Professionals
University of California     AHSL/SL               Low-Income
  at San Diego                                       Individuals,
                                                     Youth, Hispanics,
                                                     African Americans
Valley Care Foundation       HL/PL/SL              Families of Special
                                                     Needs Children
St. Francis Hospital and     HL/Senior Center      Seniors, Low-Income
  Medical Center                                     Individuals,
                                                     Minorities
University of Connecticut    AHSL/SL/PL            Public Librarians,
  Stowe Library                                      Hospital
                                                     Librarians,
                                                     Members of
                                                     Community
                                                     Organizations
George Washington            AHSL/Community        Urban Clinics,
  University                 Health Centers          Minorities,
                                                     Hispanics, HIV/
                                                     AIDS Patients,
                                                     Uninsured
                                                     Individuals
Halifax Medical Center       HL/PL                 Public Librarians
Tifton Tift County Public    PL/CBO                Public without
  Library                                            Internet access
Emory University School      AHSL/PL/CBO           African Americans,
  of Medicine, Health                                Urban Clinics,
  Sciences Center                                    Minority
  Library                                            Populations,
                                                     Hispanics, HIV/
                                                     AIDS Patients,
                                                     Uninsured
                                                     Individuals
Three Rivers AHEC            AHEC/HL/PL            Public Librarians,
                                                     Consumers
Hawaii Medical Library       AHSL/PL/SL/CBO/       Public Librarians,
                               Senior Center         Consumers, Native
                                                     Hawaiians, HIV/
                                                     /AIDS Patients,
                                                     Seniors, ESL
                                                     Students
Idaho State University,      AHSL/PL               Public Librarians,
  Idaho Health Sciences                              Consumers
  Library
Kootenai Medical Center      HL/PL                 Public Librarians,
  Library                                            Consumers
Rush-Presbyterian-           AHSL/CBO              HIV/AIDS Patients,
  St. Luke's Medical                                 Inner-City
  Center                                             Residents, Low-
                                                     Literacy
                                                     Individuals,
                                                     Hispanics
Loyola University Health     AHSL/PL/HL            Public library
  Sciences Library                                   Patrons, Nurse
                                                     Educators
Trinity Medical Center,      HL/Senior Center      Seniors
  Health Sciences
  Library
State Library of Iowa        State Library/PL/HL   Public Librarians,
                                                     Teachers, Health
                                                     Professionals,
                                                     Consumers
University of Kentucky,      AHSL/CBO
  Chandler Medical                                 Women
  Center
Louisiana State              AHSL/PL               Public Librarians
  University, Shreveport,
  Health Sciences Center
  Library
Maine General Medical        HL/CBO                Rural Residents
  Center Library
Western Maryland AHEC        AHEC/PL               Public Librarians,
                                                     Community Center
                                                     Staff, Parish
                                                     Nurses
Eastern Shore Regional       PL/PL                 Public Librarians,
  Library                                            Consumers, Rural
                                                     Residents, Seniors
University of Massachu-      AHSL/PL/CBO           Consumers
  setts Medical School,                            Clinicians, School
  Lamar Soutter Library                              Nurses, Community
                                                     Outreach Workers,
                                                     Health Educators,
                                                     Public Librarians,
                                                     School Librarians
Massachusetts General        HL/CBO/PHD            Public Librarians,
  Hospital, Treadwell                                Youth at Boys' and
  Library                                            Girls' Clubs
Holland Community            HL/PL/CBO             Health Care
  Hospital                                           Professionals,
                                                     Patients
Mid-Missouri AHEC            AHEC/PL/CBO/PHD       Public Librarians,
                                                     Consumers
University of Missouri,      N/A                   Consumers
  Columbia
Truman Medical Center        N/A                   Public Librarians,
  East                                               Nursing
                                                     Coordinators
ICON: Omaha Area             AHSL/Library          Public Librarians,
  Health Information           Consortium/           Consumers,
  Consortium                   Medical Society       Seniors, Hispanics
Las Vegas Clark County       PL/HL/CBO             Public, School,
  Library District                                   Academic, and
                                                     Special Librarians
Highlands Regional           PL/SL/Academic        Consumers, Public
  Library Cooperative          library/ Special      Librarians
                               library
University of New            AHSL/PL               Consumers, Public
  Mexico, Health                                     Librarians
  Sciences Center
  Library
ViaHealth Rochester          HL/CBO                Consumers, Public
  General Hospital                                   Librarians
SUNY Health Science          AHSL/CBO              Seniors
  Center at Syracuse
South Central Regional       Library Consortium/   Public Librarians,
  Library Council              HL/PL                 Hospital
                                                     Librarians
Samaritan Medical            HL/PL/CBO/County      Public Librarians,
  Center                       Social Service        School Librarians,
                               Agency                Consumers
University of North          AHSL/PL               Consumers,
  Carolina, Chapel                                   Librarians, Public
  Hill, Health Sciences                              Health Workers
  Library
Mountain AHEC                AHEC/PL/CBO           Public Librarians,
                                                     Community
                                                     Coalition
                                                     Coordinators,
                                                     Health Educators,
                                                     Public Health
                                                     Nurses
Oregon Health Sciences       AHSL/Public Schools   School Nurses
  University
University of Pittsburgh,    AHSL/PL               Librarians, School
  Falk Library of the                                Nurses, Students,
  Health Sciences                                    Faculty
Medical University           AHSL/PL/SL            Minorities, African
  of South Carolina,                                 Americans, Inner-
  Libraries and Learning                             City Residents,
  Resource Centers                                   Rural Residents,
                                                     Low-Income
                                                     Individuals
Rapid City Regional          HL/PL                 Public Librarians
  Hospital Health
  Sciences Library
University of Texas          AHSL/PL/HL/CBO        Librarians,
  Health Science Center                              Consumers, Health
  at San Antonio                                     Professionals
University of Utah, Eccles   AHSL/PL/CBO/AHEC/     Hispanics
  Health Science Library       PHD
Mary Washington              HL/PL                 Public Librarians,
  Hospital, Medical                                  Other Librarians
  Library
Public Health, Seattle       N/A                   Lesbians, Gay Men,
  and King County                                    Transgendered
                                                     Persons
Yakima Valley Memorial       HL/PL/CBO/PHD         Consumers, Health
  Hospital                                           Professionals,
                                                     Hispanics
University of Wisconsin,     AHSL/PL/State         Public Librarians
  Madison                      Library
Medical College of           AHSL/PL/HL            Consumers
  Wisconsin
Wyoming Medical Center       HL/PL/PHD             Librarians, Health
                                                     Department Staff

([dagger]) Institution types are abbreviated as follows: AHEC: Area
health education center; AHSL: Academic health sciences library; CBO:
Community-based organization; HL: Hospital library; PHD: Public health
department; PL: Public library; SL: School library; N/A: Not
applicable--no collaboration undertaken as part of project

REFERENCES

Backus, J. (2003, May). MLA Sunrise Seminar--MEDLINEplus. Paper presented at the Annual Meeting of the Medical Library Association, San Diego, CA. Retrieved May 28, 2004, from http://www.nlm.nih.gov/psd/presentations/medlineplusmla2003sunrise.ppt.

Burroughs, C. (2000). Measuring the difference: Guide to planning and evaluating health information outreach. Seattle: National Network of Libraries of Medicine, Pacific Northwest Region. Retrieved September 14, 2004, from http://nnlm.gov/evaluation/guide/.

Chambers, J. L. (1955). Service to the lay public: What services should we give the lay public? Bulletin of the Medical Library Association, 43(2), 257-262.

Deering, M.J., & Harris, J. (1996). Consumer health information demand and delivery: Implications for libraries. Bulletin of the Medical Library Association, 84(2), 209-216.

Gartenfeld, E. (1978). The community health information network: A model for hospital and public library cooperation. Library Journal, 103(17), 1911-1914.

Green, L. W., & Kreuter, M. W. (1999). Health promotion planning: An educational and ecological approach. Mountain View, CA: Mayfield Publishing Company.

Hollander, S. (1996). Consumer health information partnerships: The health science library and multitype library system. Bulletin of the Medical Library Association, 84(2), 247-252.

Israel, B. A., Schulz, A.J., Parker, E. A., & Becker, A. B. (2001). Community-based participatory research: Policy recommendations for promoting a partnership approach in health research. Education for Health, 14(2), 182-197.

Lasker, R. D., & Weiss, E. S. (2003). Broadening participation in community problem solving: A multidisciplinary model to support collaborative practice and research. Journal of Urban Health, 80(1), 14-60.

Martin, E. R., McDaniels, C., Crespo, J., & Lanier, D. (1997). Delivering health information services and technologies to urban community health centers: The Chicago AIDS Outreach Project. Bulletin of the Medical Library Association, 85(4), 356-361.

Meade, C. D., & Calvo, A. (2001). Developing community-academic partnerships to enhance breast health among rural and Hispanic migrant and seasonal farmworker women. Oncology Nursing Forum, 28(10), 1577-1584.

Metzler, M. M., Higgins, D. L., Beeker, C. G., Freudenberg, N., Lantz, E M., Senturia, K. D. et al. (2003). Addressing urban health in Detroit, New York City, and Seattle through community-based participatory research partnerships. American Journal of Public Health, 93(5), 803-11.

Partners for Health Information. (2001). Welcome to our Website! Retrieved September 14, 2004, from http://www.gwu.edu/~partners.

Pifalo, V., Hollander, S., Henderson, C. L., DeSalvo, P., & Gill, G. P. (1997). The impact of consumer health information provided by libraries: The Delaware experience. Bulletin of the Medical Library Association, 85(1), 16-22.

Rambo N., Zenan, J. s., Alpi, K. M., Burroughs, C. M., Cahn, M. A., & Rankin, J. (2001). Public Health Outreach Forum: Lessons learned. Bulletin of the Medical Library Association, 89(4), 403-406.

Rees, A. M. (1982). The role of the library in consumer health information services. In A. M. Rees (Ed.), Developing consumer health information services (pp. 34-41). New York: R. R. Bowker.

Scherrer, C. S. (2002). Outreach to community organizations: The next consumer health frontier. Bulletin of the Medical Library Association, 90(3), 285-289.

Sligo, E X., &Jameson, A. M. (2000). The knowledge-behavior gap in use of health information. Journal of the American Society for Information Science, 51(9), 858-869.

Taylor, H. (2001). Cyberchondriacs update. Harris Poll #19. Retrieved September 16, 2004, from http://www.harrisinteractive.com/harris_poll/index.asp?PID=229.

Turning Point National Program Office. (2003). What is Turning Point? Retrieved September 16, 2004, from http://www.turningpointprogram.org/Pages/about.html.

U.S. National Library of Medicine, Board of Regents. (1989). Improving health professionals' access to information. Bethesda, MD: U.S. Department of Health and Human Services. Retrieved September 14, 2004, from http://www.nlm.nih.gov/pubs/plan/ih/ih_intro.pdf.

Wallingford, K. T., Ruffin, A. B., Ginter, K A., Spann, M. L., Johnson, F. E., Dutcher, G. A. et al. (1996). Outreach activities of the National Library of Medicine: A five-year review. Bulletin of the Medical Library Association, 84(2 Supplement), 1-60.

Wannarka, M. (1968). Medical collections in public libraries of the United States: A brief historical study. Bulletin of the Medical Library Association, 56(1), 1-14.

Wood, E B., Lyon, B., Schell, M. B., Kitendaugh, E, Cid, V. H., & Siegel, E. R. (2000). Public library consumer health information pilot project: Results of a National Library of Medicine evaluation. Bulletin of the Medical Library Association, 88(4), 314-322.

Wood, F. B., Sahali, R., Press, N., Burroughs, C., Mala, T. A., Siegel, E. R. et al. (2003). Tribal connections health information outreach: Results, evaluation, and challenges. Journal of the Medical Library Association, 91(1), 57-66.

Angela B. Ruffin, Head, NN/LM National Network Office, National Library of Medicine, 8600 Rockville Pike, Bethesda, MD 20894, Keith Cogdill, Outreach Librarian, NN/LM National Network Office, National Library of Medicine, 8600 Rockville Pike, Bethesda, MD 20894, Lalitha Kutty, Librarian, Interlibrary Loan, National Library of Medicine, 8600 Rockville Pike, Bethesda, Maryland 20894, and Michelle Hudson-Ochillo, Second-Year NLM Associate Fellow, Arizona Health Sciences Library, 1501 North Campbell Ave, P.O. Box 245079, Tucson, AZ 85724-5079