Evaluating the Women's HIV/AIDS Resources Project as a model for establishing transnational community-based partnerships.

Abstract: The Women's HIV/AIDS Resources Project (WHARP) Partnership utilizes collective resources to facilitate a bi-directional exchange of information between South African and U.S. HIV/ AIDS Community-based organizations (CBO's). WHARP also serves as a model for establishing community-based collaborations among non-traditional organizations in an international setting. The evaluation of the WHARP project focused on the relationships between members of the partnership, highlights the accomplished/unaccomplished goals and objectives and the "Lessons Learned" from this partnership.

**********

As resources become increasingly competitive, he HIV/AIDS prevention community has become acutely aware of the need to demonstrate the value of its prevention efforts (Day, 1999). Many international community-based organizations (CBO's) strive to provide effective services that focus on HIV/ AIDS prevention and education in their communities. Accomplishing this goal is a challenge for many organizations because insufficient resources limit their prevention efforts. Public-private partnerships involving governments, foundations, private companies, and NGO's have proven to be effective in fighting the infectious diseases and epidemics that ravage the world's poorest and most vulnerable communities (Cook, 2001).

According to the National Minority AIDS Council (NMAC) funding for HIV programs have not increased proportionally according to the level of need in communities of color. In response to global health challenges the Centers for Disease Control and Prevention (CDC) and the Agency for Toxic Substances and Disease Registry (ATSDR) have prepared global health strategies for working with partners to improve global health. CDC/ATSDR acknowledge their commitment to a) the increasing influence of determinants arising outside the country on U.S. health; b) the mutual benefits of improving the health of other countries; c) the advantages of sharing U.S. knowledge and public health expertise with international partners; and d) the need to respond to the health consequences of international emergencies.

By engaging in rigorous evaluation of prevention efforts, organizations can assess whether change in attitudes, beliefs and behaviors has occurred, and whether the change can be attributed to a specific prevention effort (Day, 1999).

WHARP was a pilot HIV prevention project designed to provide support in the form of training, technical assistance and resource development to enhance the capacity of the partnering organizations in South Africa and the U.S. The Partnership is committed to promoting and strengthening resources for those infected and affected by HIV/AIDS through effective international, national, and local NGO collaboration that is based upon the principles of mutual trust, transparency, and respect among participants. The overall goal of the project was to establish a transnational partnership in order to exchange information and share resources that can be used to enhance HIV/AIDS prevention efforts among women's NGOs. The goals of this project included establishing linkages between SL and the NCHRE, two U.S. based grassroots organizations, and PWN, TAP, and SWAA-SA the regionally based organizations, to address HIV prevention and women's rights in South Africa. Other goals included sharing resources to further HIV/AIDS prevention efforts among women, and to pilot a replicable program framework for other community-based transnational partnerships.

This goal was accomplished by providing assistance in the three focus areas that were identified as priorities: training, resource development and technical assistance. Through this project, assistance was provided to organizations working with South African women living in the Gauteng province infected by the epidemic. Assistance that developed programs unique to the needs of the South African women similar to the way SL was able to build its unique programs for African-American women living in the southern U.S. who were also infected by the epidemic (SL, 1999).

OUTCOMES AND PERFORMANCE MEASURES

Indicators define the project's attributes that pertain to the focus of the evaluation (CDC, 1999). Indicators translate general concepts about a project and its expected effects into, specific, measurable parts (Center for the Advancement of Community-Based Public Health, CA-CBPH, 2000). For this evaluation the major indicators included:

* Levels of participation and satisfaction

* Achieved and understood goals and objectives

*Improvement in organizational capacity

* Project sustainability

Process evaluation was ongoing throughout the project cycle and included field observations, pre and post-tests, debriefing participants, and community feedback. This was done in order to monitor the project's indicators.

METHODOLOGY

GOALS

To identify and document the goals, processes, benefits, and outcomes resulting from the implementation of the WHARP Partnership as an international community based collaboration conducted through linkages between NGO's in the U.S. and in SA and to determine the impact of the Partnership on the HIV/ AIDS epidemic.

OBJECTIVES

The objectives of the evaluation are:

* To determine the effective strategies, activities, and interventions that were implemented to strengthen the capacity of the Partnership.

* To identify factors that served as barriers to or enhanced the implementation of WHARP processes and project objectives.

* To determine the challenges and opportunities for each key partner, as a participant in the Partnership.

* To address and identify the mechanisms and lessons learned for promoting sustainability of the Partnership.

EVALUATION DESIGN

The overall evaluation design consisted of three consecutive study phases. Phase one included the period of dialogue, planning, and discussion of the Partnership agreement and needs assessment (this initial phase started in December of 1999). Phase two was composed of strengthening the formal Partnership and revision of the formal agreement via a MOA in addition to a preliminary evaluation based on the program interventions (Healthy Love Parties, Training of Trainers Workshops, etc.) and strategic planning activities. Phase three activities included the final meetings (Reflection, Debriefing, etc.) and evaluation activities (the key partner interviews). Phase three of the evaluation process began in January of 2002.

EVALUATION PROCESSES AND ACTIVITIES

PHASE I

The activities being evaluated for phase one included a period of dialogue, planning, and discussion of the Partnership agreement and needs assessment (this initial phase started in December of 1999). During this phase a briefing session was conducted with all of the key stakeholders (including representatives from the funding agencies) to familiarize each member with the project and discuss the formal agreement.

PHASE II

Other evaluation activities were based on the completion of processes, which included, the redefining the objectives, strengthening the formal Partnership and revision of the formal agreement via the MOA in addition to a preliminary evaluation based on phase one activities including some program interventions (Healthy Love Parties, Training of Trainers Workshops, etc.) Also during this phase a series of strategic planning meetings were held and a strategic plan was developed.

PHASE III

The final phase of the evaluation included site visits, key and local partner interviews, meetings, and a review of the processes of WHARP for Years 02-03 activities. The culminating pilot project activities also included a Partnership Reflection Meeting, which was held in Johannesburg, South Africa on October 1, 2002. Participant post feedback evaluations were administered to the participants at the Reflection Meeting.

DATA COLLECTION

Qualitative methods where used as the primary technique for data collection. The multi-methods of data collection used for this evaluation included interviewing, document review/analysis, and observational analysis. Data collection was performed during January 2002-April 2003.

LOCAL SA PARTNER INTERVIEWS

Local SA partner interviews were conducted and were limited to only personnel from the lead agency representing all the local SA partners. These interviews were held after the Reflection Meeting. There was a total of two documented interviews with the local partners' representative agency. This activity was conducted to assess the local SA partner's views and understanding of the WHARP Partnership, the strategic planning outcomes and the Reflection Meeting. The participant feedback questionnaire was designed by the evaluation team an administered to the key partners of WHARP.

SA KEY PARTNERS' INTERVIEWS

A member of the evaluation team conducted interviews with the SA key partners after the Reflection Meeting. There were a total of five documented interviews from the SA key partners and affiliates, This activity was conducted to assess the SA key partner's views and understanding of the WHARP Partnership, the strategic planning outcomes and the Reflection Meeting. The participant feedback questionnaire was designed by the evaluation team and administered to the key partners of WHARP.

U.S. KEY PARTNERS' INTERVIEWS

The U.S. key partner interviews were arranged with four members from the two U.S. partnering agencies. These interviews were conducted by telephone and focused on the WHARP processes. During the final data collection and analysis process, telephone interviews were conducted over a two-week period during February 2003. A preview of the questionnaire was sent via email to the participants prior to the telephone interviews (January 2003). A follow up call was made to participants to schedule their individual interviews (February 2003). Of the four participants scheduled only three participated in the interview process. The fourth participant thought that her knowledge and involvement of WHARP was insufficient to provide accurate descriptions of WHARP's activities. The interview responses were recorded (with verbal permission from the participants) and transcribed for analysis. This activity was conducted to assess the U.S. key partner's views and understanding of the WHARP Partnership, the strategic planning outcomes and the Reflection Meeting. The participant feedback questionnaire was designed by the evaluation team and administered to the key partners of WHARP.

DATA ANALYSIS

PHASE I

Document analysis was the procedure used to analyze the data collected during phase one of the evaluation. The steps utilized for this process included:

(1) Reviewing all WHARP related documentation identified for phase one (specified in the types of data and information section of the methodology) and

(2) Categorizing and extracting relevant themes and constructs related to the project planning and partnership activities.

PHASE II

Document and observational analysis were the procedures used to analyze the data collected during phase two of the evaluation. The steps utilized for this process included:

(1) Reviewing all WHARP related documentation identified for phase two (specified in the types of data and information section of the evaluation methodology).

(2) Categorizing and extracting relevant themes and constructs related to the strategic planning and preliminary evaluation.

PHASE III

Document, transcription, interviews, and observational analysis were the procedures used to review and analyze the data collected during phase three of the evaluation. The steps utilized for this process included:

(1) Reviewing all WHARP related documentation identified for phase three (specified in the types of data and information section of the evaluation methodology).

(2) Categorizing and extracting relevant themes and constructs related to the culminating activities of the partnership.

(3) Transcribing the data from the U.S. interviews.

(4) Reviewing the data from the local and SA key partners interviews.

(5) Coding and categorizing the data collected from the interviews.

(6) Reviewing all notes, meeting minutes and evaluators comments.

RESULTS/INTERPRETATIONS

The community needs assessment was developed and implemented during the first quarter of the funding cycle. It assessed a number of activities, behaviors and attitudes within the community and the community's views on appropriate interventions.

The original program design consisted of dividing the initial program activities into four quarters over the first year of the grant with training, community outreach, support services, and income-generating activities. These program activities were continued throughout the second year.

Training consisted of selected peer counselors to carry out HIV prevention educational activities in South African communities, to address the lack of knowledge identified by the South African NGOs. Training was used to engage South African men in addressing the rights of women. Providing a safe space where HIV-infected women can receive emotional support and learn job skills will also address the issue of women's status. The South African NGOs were also provided computers, Internet access and fax capabilities, which contributed towards capacity building. The South African NGOs were responsible for assuming the majority of the responsibility for the continuation of the Project's in-country activities while all partnering organizations remained linked through monitoring and information exchange. Table 1. provides a summary of the interview findings.

The partners were asked to identify both enablers and barriers of the WHARP processes during the partnership project. The most common response for enablers was the key stakeholders. The most common barriers were communication and lack of resources. Table 2 identifies these enablers and barriers and how they barriers were overcome.

DISCUSSION

The Impacts and Outcomes of WHARP as an International Partnership are seen through several notable accomplishments, which were identified by the partners. They included:

* Strengthening the individual capacities of each of the organizations

* Increased visibility for all of the organizations and increase visibility and opportunities to get support for the program and the organizations beyond the partnership

* SL refocused to strengthen internal capacities as an NGO doing transnational work in the area of HIV

* NCHRE developed skills in terms of conflict resolution and conflict management and understanding that that's a key critical piece of partnership, but being able to provide that technical assistance in the form of training and bringing that human rights perspective

* Development of a how-to-manual that will basically outline based on the experiences of the NGO pilot partnership.

CHALLENGES OF THE WHARP PARTNERSHIP

Some of the challenges within the partnership were related to lack of availability of financial resources as well as information and material resources. Some of the U.S. partners believed that lack of resources and information posed particular challenges for the South African partners. One U.S. key stakeholder noted that these challenges could have been related to the impact of the African patriarchal society on South African women NGOs ability to access information and material resources.

It was also mentioned that although the exchanges of tools and training may have helped organizational efforts, one of the major challenges encountered in the partnership was effective communication between all participating organizations. There was also a lot of confusion about what was going on, when, and where. This was an indication that more time needed to be dedicated to planning and communicating by email, by phone or by fax before conducting training and other activities. Translation of training material also was a challenge for all parties because SA partners was burdened with translating the training materials, and some of the themes and principles of the English language are not translatable into Zulu principles.

There were also challenges in coordinating activities from the U.S. that occurred in SA as a result of the communication difficulties. The U.S. partners emphasized the need for a better-developed in-country communication process between the U.S. partners as well as more frequent contact and communication with the SA partners. Many activities took a lot more effort than was needed because of lack of communication. The U.S. partners indicated that there was lack of consistent feedback from all partners regarding partnership development and ongoing activities.

Some of the U.S. partners indicated that there were more benefits than difficulties because SA was not unfamiliar territory for them and because many of the U.S. partners had participated in many different activities within the country. However, one U.S. key stakeholder suggested that participants in WHARP were involved in the partnership for different reasons, which may have created conflicts between SA partnering organizations and posed some challenges in meeting all the project objectives. Another challenge was not getting in any conflicts between the local SA partners.

Problems developed in maintaining similar levels of accountability between individuals and other partnering organizations. It was difficult for one person to do HIV/AIDS community level work and raise the awareness of related issues without having the organizational capacity or support. Local Partnership participants that were not representing any particular partnering organization did not have the financial and material resources to support their efforts to participate in the partnership to the maximum extent.

BENEFITS OF THE WHARP PARTNERSHIP

The U.S. partners indicated that one of the benefits gained from participating in WHARP was developing relationships and sharing personal experiences with South African women. One U.S. participant expressed how rewarding it was to meet and work with other HIV+ women who are committed to the fight against AIDS despite the unavailability of medication.

In addition, several of the U.S. partners indicated that the benefits were more practical and educational in nature. One of the benefits of the partnership was learning how to develop human rights education that focuses on HIV. Another benefit was learning how to deliver human rights training in a different environment. An example of this process was developing the pre and post questionnaires and trying to translate phrases and concepts from English to other indigenous languages, which proved difficult at times. Often times some of the ideas did not translate. As a result, the U.S. partners indicated that they learned how to construct questionnaires appropriate in other languages such as Zulu.

Another U.S. key stakeholder indicated that an immediate benefit of the partnership to the U.S. participants was conducting human rights training in a different cultural setting. This allowed them to see what changes needed to be incorporated into curriculum that would speak to the needs of the population served. In addition, the U.S. partners discovered some commonalties between those affected by HIV in SA and those in the U.S. as a result of the training. The U.S. partners also discovered that there was a module. Building from the concept of doing women's HIV/ AIDS human right education for men was another benefit since it had not been a priority for the U.S. partnering organizations in the past. Participating in WHARP also helped to accelerate the efforts to meet the need to educate men on women's human rights.

LESSONS LEARNED

Communities and Community Based Organizations (CBO's) can hopefully utilize these lessons in addressing issues related to the AIDS pandemic across the Diaspora. The Lessons Learned from the WHARP Partnership were most appropriately stated when the partners were asked if they had to build a new partnership what they would do differently. Their responses were: (a) reducing number of key partners, (b) clearly state the budget and have adequate staff to monitor financial capacity, (c) provide funding for equal travel between the U.S. and SA, or between all collaborating parties, this would increase the ability to share resource and provide support, (d) include board as organ of governance, (e) provide specific roles for all participants from the beginning of the project (f) establish board of trustees, (g) allow local and community input in the initial planning process, and (j) establish operational guidelines to ensure proper methods of communication.

Additional lessons were to allocate sufficient funding to have a coordinating center from the beginning to be as transparent and clear on issues such as, budgeting, responsibilities, defining the inequalities as well as the equalities of the partnership.

Establish procedures to ensure that the organizations selected for the partnership have similar backgrounds and have equal levels of ability. Not only in terms of the work and services they provide, but the populations that they serve, the missions that they share, and the capacities of the organizations to be engaged in the partnership.

RECOMMENDATIONS

As a result of this evaluation it is recommended that additional research be conducted to determine the actual impact of the partnership on the HIV/AIDS epidemic. Even though the infrastructure of the organizations providing services needs to be strong to be most effective, assessments should be conducted to see how specific capacity building activities enhance services provided directly to those infected and affected by HIV/AIDS.

It is also recommended that because this was the first project of this type to be explored and implemented the outcomes, lessons learned, and best practices should be published and they should serve as a template for establishing future international collaborations.

CONCLUSION

This evaluation describes the SL/WHAKP Pilot Partnership Project and highlights the goals and objectives of the WHARP project activities and the Partnership accomplishments. Public health work would not be as effective without partnerships that unify and represent the diversity of today's society. Community-based grassroots organizations play a major role in enhancing the community's knowledge of HIV/ AIDS. They also provide support and resources to those that are affected by the epidemic. Past efforts have proven that combating the HIV/AIDS epidemic without addressing the global health issues will only increase the amount of work and increase the potential for less effective interventions. This partnership serves as a model became it represents the dedication, courage, and ambition of women's grassroots community based organization need in order to change health outcomes. It also addresses the need for appropriate and specific consideration when focusing on the HIV/ AIDS needs of women of color in the Diaspora.

Table 1. Summary of Key Partner Interview Findings

                           Local                        SA
                          Partners                   Partners

   Level of        * High                    * Mobilizing NGO's
  Involvement      * Organized the           * Locating trainees
                     organizations           * Activities coordination
                     affiliated with         * Organizing venues
                     WHARP(database)         * Planning meetings
                                             * Documentation review
                                             * Full participation
                                             * Training
                                             * Formal Agreements

 Satisfaction      * Not Satisfied           * Okay
     Level           (lack of resources)     * Satisfactory
                                             * Not always satisfied
                                             * Not so good
                                             * 40% (okay)

    Project        * Yes                     * Yes
  Usefulness       * Provided training       * Exchange of informa-
                   * Provided a process        tion
                     for establishing        * Provided a learning
                     future partner-           experience
                     ships                   * Opportunity to share
                   * Helped each               resources
                     CBO/NGO assess          * Gained information
                     their SWOT                on collaboration

    Clearly        * Yes                     * Yes
  Stated and                                 * Written objectives clear,
  Understood                                   actual thoughts and
  Objectives                                   implementation was
                                               not

Satisfaction of    * No, Not with            * Objectives provided
    Chosen           male involvement          encouragement
  Objectives       * Yes, Information
                     exchange
                   * TA/ Resource
                     sharing
                   * Strengthening
                     local HIV/AIDS
                     prevention

  Understand-      * Exchange infor-         * Exchange information,
 ing of Goals/       mation                    Share resources, informa-
  objectives       * Learn from expe-          tion, training/
                     rience                    knowledge, TA
                   * Prepare for the         * Eradicate the impact of
                     future                    HIV/AIDS through
                                               education, prevention,
                                               support, and human
                                               rights advocacy
                                             * Create an International
                                               community based
                                               partnership
                                             * Capacity building
                                             * Self sustainability of
                                               all organizations

  Achievement      * Yes, (TA/equip-         * Yes, (Men's Focus
 of Objectives       ment were not             Sharing of informa-
(not achieved)       met fully)                tion was not
                   * Skills Transfer-          consistent)
                     Technical               * Lack of resources
                     Governance              * Time restrictions
                     documentation
                   * Public Speaking

   Level of        * Yes, project was        * Yes, commitment of
    Project          successful                the partners
    Success        * Brought local           * Project sustainability
                     and international       * Establishing the
                     organizations             Partnership
                     together                * Implementation not
                   * Strengthened              completely successful
                     support groups          * Not the Male focused
                   * Provided basic            activities
                     HIV training
                   * Established
                     effective
                     communication
                   * Documentation

                             US
                          Partners

   Level of        * Comprehensive
  Involvement      * 100 % participation
                   * Responsible for implementation
                   * Activities coordination
                   * Key partner meetings
                   * Correspondence follow-up
                   * Communication
                   * Funders
                   * Documentation tracking
                   * Responsible for MOU/MOA
                     agreement
                   * Provided technical assistance
                   * Monitored resource allocation
                   * Provided training

 Satisfaction      * Satisfied for the most part
     Level         * Yes satisfied, responsibility as an
                     employee
                   * 9.5, on a scale of 1-10, tremen-
                     dously satisfied

    Project        * Yes
  Usefulness       * Definitely
                   * Gained lessons/experiences
                   * Gained new information
                   * Strengthened each organizations
                     capacity

    Clearly        * Yes, evolved over time(changed
  Stated and         after community needs assessment)
  Understood       * Not understood by all partners
  Objectives       * Redefine/modify objectives
                   * MOA development

Satisfaction of    * Not 100%
    Chosen         * Some objectives were not realistic
  Objectives         (time/funds)
                   * Responsibilities made in MOA

  Understand-      * Implementation of a pilot
 ing of Goals/       partnership
  objectives       * Determine outcomes of a
                     transnational partnership among
                     grassroots organizations
                   * Exchange information
                   * Share resources, TA, best practices,
                     information, training/ knowledge,
                     designing interventions

  Achievement      * 85% - 90%
 of Objectives     * 90% - 95%
(not achieved)     * (male involvement, creating a local
                     resources directory guide)

   Level of        * Yes
    Project        * Met goal/objectives
    Success        * Outcomes
                   * Sustainability
                   * Community impact
                   * Functioning staff in SA
                   * Funding from Gates foundation

Table 2. Enablers/Barriers of WHARP Processes

                   Local Partners          SA Partners

Enablers         * Key Stakeholders      * Program Directors,
                 * Evaluation team         managers, and site
                                           coordinators
                                         * CDC
                                         * Local Partners
                                         * SL-kept communi-
                                           cation channels
                                           open

Barriers         * Not understand-       * Discussing issues
                   ing roles             * Communication with
                 * Cultural                local partners
                   differences           * Disrespect of time
                 * Overtime              * Insufficient resources
                 * Delayed funding       * Unclear roles/
                                           responsibilities

How Barriers     * Meetings with         * Focusing on
Were Overcome      Stakeholders            similarities as
                 * Ongoing                 people of color
                   communication         * Improving
                                           communication
                                           networks

                   U.S. Partners

Enablers         * Key Partners
                 * Friendship established prior to
                   the Partnership
                 * SL International programs
                   manager
                 * Transparency

Barriers         * Communication
                 * Implementation of commitments
                 * Additional demands from the
                   funders
                 * Lack of resources

How Barriers     * Reassigning tasks
Were Overcome    * Readjusting budgets
                 * Dialogue and communication
                 * Formal agreements
                 * Transparency

REFERENCES

Center for the Advancement of Community Based Public Health (CA-CBPH). An Evaluation Framework for Community Based Programs. 2000.

CDC/ATSDR, Global Health. Working with Partners to Improve Global Health: A Strategy for CDC and ATSDR. September 2000.

CDC. Framework for Program Evaluation in Public Health. MMWR 1999; 48(No. RR-11): 1-35.

Cook, Joseph A. Public-Private Partnerships Are Key to Fighting Disease. New Statesman. Issue: Oct, 15 2001. Available at www.findarticles.com; Accessed April 16, 2003.

Day, Charlene. Evaluation: Maximizing your prevention efforts. National Minority AIDS Council. 1999.

Bill & Melinda Gates Foundation. The Bill & Melinda Gates Foundation Awards U.S. $150,000 Grant to the Gauteng Partnership for Grassroots AIDS Action. 2002. Available at www.gatesfoundation.org. Accessed on April 18, 2003.

Rodney, P. and Nu, S. "WHARP Evaluation Report Year 01" Adanta, Georgia. SisterLove, Inc. 2001.

Sewell, Meg. The use of Qualitative Interviews in Evaluation. Cyfernet Evaluation. Accessed at http:// ag.arizona.edu/fer/fs/cyfar/Intervu5.htm; Accessed March 31, 2003.

SisterLove, Inc. "WHARP Grant Proposal". Atlanta, Georgia. 1999.

SisterLove, Inc. "International Programs Concept Paper". Atlanta, Georgia. 2002.

SisterLove, Inc. "WHARP Quarterly Progress Reports (Years 01,02, and 03)". Atlanta, Georgia. 1999-2002.

SisterLove, Inc. WHARP Final Performance Report. Atlanta, Georgia. 2003.

SisterLove, Inc. "WHARP Strategic Plan Report". Atlanta, Georgia. 2001.

SisterLove, Inc. "WHARP Cooperative Agreements (Year 03)". Atlanta, Georgia. 2001-2002.

ACRONYMS

NGO--Non-governmental organization

SL--SisterLove Inc.

NCHRE--National Center for Human Rights Education

PWN--Positive Women's Network

TAP--Township AIDS Project

SWAA-SA--Society for Women Against AIDS in Africa-South Africa Chapter

MOA--Memorandum of Agreement

MOU--Memorandum of Understanding

SA--South Africa

SWOT--Strengths, Weaknesses, Opportunities, and Threats

TA--Technical Assistance

HEALTH EDUCATION RESPONSIBILITY AND COMPETENCY ADDRESSED

Responsibility IV--Evaluating Effectiveness of Health Education Programs

Competency D--Infer implications from findings for future program planning.

Subcompetency 3--Apply findings to refine and maintain programs.

Address all correspondence to Patricia Rodney, M.P.H., Ph.D., Morehouse School of Medicine, Department of Community Health and Preventive Medicine, Master of Public Health Program, 720 Westview Drive, Southwest, Atlanta, Georgia 30310, PHONE: [phone], FAX: [phone], E-MAIL: [e-mail].