Project Performance Assessment Report on Kenya Sexually Transmitted Infections Project.
Currency Unit = Kenya Shilling (Ksh)
44.6 Ksh = US$ 1 At Appraisal
75 Ksh = US$1 At Project closing
US$ .0224 = 1 Ksh at Appraisal
US$ .0133 = 1 Ksh at Project closing
Abbreviations and Acronyms AIDS Acquired immune deficiency syndrome CBO Community Based Organization CEDA Canadian International Development Agency DACC District AIDS Control Council DARE Decentralized ADDS and Reproductive Health Project DASCO District AIDS and STI Coordinator DfID Department for International Development (United Kingdom) DHS Demographic and Health Survey DIAC District Intersectoral AIDS Committee HPAC HIV/AIDS Prevention and Care Project HIV Human Immunodeficiency Virus HSV-2 Herpes Simplex Virus 2 (genital herpes) ICR Implementation Completion Report GoK Government of Kenya KADRE Kenya AIDS Disaster Response Emergency Project MAP Multi-country AIDS Project MOH Ministry of Health MOU Memorandum of Understanding MTR Midterm Review NACC National AIDS Control Council NASCOP National AIDS and STI Control Program NGO Nongovernmental Organization OED Operations Evaluation Department PMU Project Monitoring Unit PPAR Project Performance Assessment Report PSI Population Services International STI Sexually transmitted infection(s) USAID United States Agency for International Development VCT Voluntary counseling and testing WHO World Health Organization
Government: July 1-June 30
Director-General, Operations Evaluation: Mr. Gregory K. Ingram
Manager, Sector and Thematic Evaluation: Mr. Alain Barbu
Task Manager: Mr. Timothy Johnston
MEMORANDUM TO THE EXECUTIVE DIRECTORS AND THE PRESIDENT
SUBJECT: Project Performance Assessment Report on Kenya Sexually Transmitted Infections Project
Attached is the Project Performance Assessment Report prepared by the Operations Evaluation Department (OED) on the Kenya Sexually Transmitted Infections (STI) Project. The project's three major objectives were to: (i) strengthen institutional capacity at the national and district levels to design, implement, monitor and evaluate STI/HIV interventions; (ii) promote preventive measures to reduce the risks of STI/HIV transmission; and (iii) enhance both health sector and community provision of physical and psychological care and develop strategies to mitigate the social and economic consequences of AIDS. The project was financed through EDA Credit No. C2686 for US$40 million, with a government contribution of $4 million, and donor cofinancing of $25.5 million. It was approved in March 1995, and closed June 30, 2001, with US$0.9 million of the remaining credit cancelled.
The original project design was relevant, but gave insufficient attention to implementation arrangements. Implementation progress was slow in the initial years, but the Bank and government subsequently gave increased attention to resolving implementation bottlenecks at the national and district level. Following the midterm review (1997), the Bank gave increased attention to building political support for a comprehensive HIV/AIDS control program. Project monitoring and evaluation was weak, making it difficult to assess the impact--and in some cases the output--of project activities.
Outcome is rated moderately satisfactory. The project made substantial contributions to strengthening national political commitment to addressing STIs/HIV, and to strengthening capacity for STI/HIV treatment and prevention, particularly in the original 15 pilot districts. But the project made only modest contributions to the capacity of the National AIDS and STI Control Program (NASCOP), and negligible contributions to strengthening the capacity of nongovernmental organizations (NGOs). Under objective (ii), drugs and training financed by the project appear to have improved treatment of bacterial sexually transmitted infections (which may have a role in facilitating HIV transmission). The project also provided over $6 million for information, education, and communication (IEC) activities--to promote awareness and behavior change regarding STIs and HIV--but there is limited evidence regarding the output or impact these activities. Objective (iii) was only partially achieved. The project financed pharmaceuticals to treat opportunistic infections, but failed to channel originally committed project funds to communities or local NGOs in support of care and mitigation activities. Sustainability is rated as likely, but with variation among components. The government's commitment to addressing HIV/AIDS remains strong, yet neither the Bank nor government adequately planned for sustaining the STI drug supply following project closure. Institutional development is rated substantial. Bank and borrower performance are both rated satisfactory.
Project experience also suggests several other findings and confirms some OED lessons. First, strong monitoring and evaluation programs--clear and measurable indicators that are systematically collected over time--are essential to assess program impact and to improve the effectiveness of future interventions. Second, the socio-political environment has a strong influence on the effectiveness of national HIV/STI prevention programs. The Bank and partners can contribute to strengthened political commitment by promoting dialogue with and among political leaders and other stakeholders, but this tends to be a long-term process. Third, decentralizing project activities to district level can improve local implementation, but needs to be accompanied by capacity building and institutional reforms to give districts greater autonomy over planning and budgeting. Finally, projects planning to finance NGO/community activities need to develop clear modalities for doing so, and develop strategies to ensure that high-risk groups are targeted and that mechanisms are in place for monitoring, evaluation, and sharing lessons.
OED Mission: Enhancing development effectiveness through excellence and independence in evaluation
About this Report
The Operations Evaluation Department assesses the programs and activities of the World Bank for two purposes: first, to ensure the integrity of the Bank's self-evaluation process and to verify that the Bank's work is producing the expected results, and second, to help develop improved directions, policies, and procedures through the dissemination of lessons drawn from experience. As part of this work, OED annually assesses about 25 percent of the Bank's lending operations. In selecting operations for assessment, preference is given to those that are innovative, large, or complex; those that are relevant to upcoming studies or country evaluations; those for which Executive Directors or Bank management have requested assessments; and those that are likely to generate important lessons. The projects, topics, and analytical approaches selected for assessment support larger evaluation studies.
A Project Performance Assessment Report (PPAR) is based on a review of the Implementation Completion Report (a self-evaluation by the responsible Bank department) and fieldwork conducted by OED. To prepare PPARs, OED staff examine project files and other documents, interview operational staff, and in most cases visit the borrowing country for onsite discussions with project staff and beneficiaries. The PPAR thereby seeks to validate and augment the information provided in the ICR, as well as examine issues of special interest to broader OED studies.
Each PPAR is subject to a peer review process and OED management approval. Once cleared internally, the PPAR is reviewed by the responsible Bank department and amended as necessary. The completed PPAR is then sent to the borrower for review; the borrowers' comments are attached to the document that is sent to the Bank's . Board of Executive Directors. After an assessment report has been sent to the Board, it is disclosed to the public.
About the OED Rating System
The time-tested evaluation methods used by OED are suited to the broad range of the World Bank's work. The methods offer both rigor and a necessary level of flexibility to adapt to lending instrument, project design, or sectoral approach. OED evaluators all apply the same basic method to arrive at their project ratings. Following is the definition and rating scale used for each evaluation criterion (more information is available on the OED website: http://worldbank.org/oed/eta-mainpage.html).
Relevance of Objectives: The extent to which the project's objectives are consistent with the country's current development priorities and with current Bank country and sectoral assistance strategies and corporate goals (expressed in Poverty Reduction Strategy Papers, Country Assistance Strategies, Sector Strategy Papers, Operational Policies). Possible ratings: High, Substantial, Modest, Negligible.
Efficacy: The extent to which the project's objectives were achieved, or expected to be achieved, taking into account their relative importance. Possible ratings: High, Substantial, Modest, Negligible.
Efficiency: The extent to which the project achieved, or is expected to achieve, a return higher than the opportunity cost of capital and benefits at least cost compared to alternatives. Possible ratings: High, Substantial, Modest, Negligible. This rating is not generally applied to adjustment operations.
Sustainability: The resilience to risk of net benefits flows over time. Possible ratings: Highly Likely, Likely, Unlikely, Highly Unlikely, Not Evaluable.
Institutional Development Impact: The extent to which a project improves the ability of a country or region to make more efficient, equitable and sustainable use of its human, financial, and natural resources through: (a) better definition, stability, transparency, enforceability, and predictability of institutional arrangements and/or (b) better alignment of the mission and capacity of an organization with its mandate, which derives from these institutional arrangements. Institutional Development Impact includes both intended and unintended effects of a project. Possible ratings: High, Substantial, Modest, Negligible.
Outcome: The extent to which the project's major relevant objectives were achieved, or are expected to be achieved, efficiently. Possible ratings: Highly Satisfactory, Satisfactory, Moderately Satisfactory, Moderately Unsatisfactory, Unsatisfactory, Highly Unsatisfactory.
Bank Performance: The extent to which services provided by the Bank ensured quality at entry and supported implementation through appropriate supervision (including ensuring adequate transition arrangements for regular operation of the project). Possible ratings: Highly Satisfactory, Satisfactory, Unsatisfactory, Highly Unsatisfactory.
Borrower Performance: The extent to which the borrower assumed ownership and responsibility to ensure quality of preparation and implementation, and complied with covenants and agreements, towards the achievement of development objectives and sustainability. Possible ratings: Highly Satisfactory, Satisfactory, Unsatisfactory, Highly Unsatisfactory.
Principal Ratings PPAR ICR Outcome Moderately Satisfactory Satisfactory Sustainability Likely Likely Institutional Development Substantial Substantial Borrower Performance Satisfactory Satisfactory Bank Performance Satisfactory Satisfactory * The Implementation Completion Report (ICR) is a self-evaluation by the responsible operational division of the Bank. Key Staff Responsible Project Task Manager/ Division Chief/ Country Director Leader Sector Director Appraisal Vulimiri Jacob van Francis X. Colaco Jagdish Lutsenburg Maas Completion Andrew Follmer Dzingai Mutumbuka Harold E. Wackman
This is a Project Performance Assessment Review (PPAR) for the Kenya Sexually Transmitted Infections (STI) Project. The project was financed through IDA Credit No. C2686 for US$40 million, with a government contribution of $4 million, and donor cofinancing of $25.5 million. It was approved in March 1995, and closed June 30,2001, with US$0.9 million of the remaining credit cancelled.
The findings of this assessment are based on an Operations Evaluation Department (OED) mission to Kenya in February 2002. The mission also discussed with stakeholders the emerging experience with the Decentralized AIDS and Reproductive Health (DARE) Project and the Kenya AIDS Disaster Response (KADRE) Project, which are currently under implementation.
The OED mission met with government officials (including Ministry of Health, NASCOP, and the National AIDS Council; district and provincial staff); donor representatives; local and international researchers; nongovernmental and religious organizations; and health providers. The mission made field visits to several districts, including Nyeri and Kerinyaga (Central Province), Embu (Eastern Province) and Kisumu (Nyanza Province). OED distributed a short questionnaire to assess the views of government officials and stakeholders regarding the strengths and limitations of the project, as well as the Bank's HIV/AIDS programs more generally. Key documentary sources included the project's Implementation Completion Report (Report No. 22897, September 2001), the Staff Appraisal Report (Report No. 13385-KE, February 1995), project files, audit reports, and evaluations and reports from other donors, international agencies, and nongovernmental organizations (NGOs). The PPAR also reviewed available research literature regarding STIs and HIV in Kenya.
The author expresses appreciation to all those who made time for interviews and provided documents and information, including past and present officials of the Ministry of Health, staff of the STI Project Management Unit (PMU), the National AIDS and STD Control Program (NASCOP), the National AIDS Control Council (NACC), Provincial AIDS Control Councils (PACCs), donor representatives, nongovernmental and civil society organizations.
Following standard OED procedures, copies of the draft PPAR were sent to the relevant government officials and agencies for their review and comments. A number of observations were made, which have been incorporated into the text, and are attached to the PPAR as Annex D.
BACKGROUND AND SECTOR CONTEXT
1. During the 1970s and 1980s, Kenya made substantial gains in the health status of its population. By the late 1980s, health and fertility indicators were better than those of most other countries at similar levels of development. (1) Socioeconomic factors played a major role in these changes, but the rapid expansion of government-financed health and family planning programs during this period clearly contributed. The World Bank and other international donors provided substantial support to those programs, but by the early 1990s, many donors began reducing their support out of concern with the overall governance environment in Kenya and persistent project implementation difficulties. (2) This, plus a 10-percent decline in income per capita from 1990 to 1998 and continued inefficiencies in public resource allocation, put increased financial pressure on the government health system. The Ministry of Health (MOH) introduced cost recovery for health services in 1991, but additional revenues from cost recovery were offset by declines in government per capita health spending.
2. By the early 1990s, HIV/AIDS threatened to erode many of the health gains of the 1980s and place a growing burden on the health system. HIV prevalence--the percentage of adults living with HIV infection or AIDS--rose to about 12 percent in 1993 and 15 percent by 1999. These averages masked variations by region, gender, and age cohort. For example, prevalence rates in Kisumu district exceed 30 percent among sexually active adults. And HIV prevalence among girls aged 15-19 is five times higher than for boys of the same age (Glynn and others 2001). Sentinel surveillance data suggest that prevalence has declined in some districts in the past several years, but continues to increase in others. There are no reliable national data on changes in HIV incidence, however. (3)
3. Sexually transmitted infections (STIs) were relatively widespread in Kenya at the time of STI Project design: 5-10 percent of women attending antenatal clinics tested positive for syphilis or gonorrhea in the early 1990s, with higher rates for high-risk groups such as truck drivers and commercial sex workers. The human immunodeficiency virus (HIV) that causes AIDS is primarily a sexually transmitted disease in Sub-Saharan Africa and is largely preventable through the same precautions used to prevent other STIs--condom use, reduction in the number of sexual partners, and abstinence. In addition to being painful and potentially contributing to chronic health problems and infertility, by the early 1990s a growing body of evidence suggested that the presence of other STIs facilitated the transmission of HIV. Moreover, research in Kenya and elsewhere suggested that early diagnosis and treatment of STIs might reduce the risk of HIV transmission (see Annex B). As a result, the World Health Organization (WHO) advocated STI treatment as a cost-effective strategy to prevent HIV transmission. Subsequent research suggests, however, that the impact of STI treatment on HIV transmission may depend of a number of factors, including the type of STI, whether the STI is ulcerative, sexual behavior patterns, the quality and coverage of STI treatment programs, and the extent to which sexual (Fleming and Wasserheit 1999; Rottingen and others 2001).
4. At the inception of the Sexually Transmitted Infections Project, MOH strategy called for decentralizing responsibility for budgets and service delivery to the district level, but continued central control over health service planning and budgeting constrained the ability of districts to respond to local health needs. The other key players in health delivery in Kenya are mission hospitals, which provide nearly 40 percent of care in rural areas, and private practitioners, who have grown in both urban and rural areas over the past decade. The number of nongovernmental organizations (NGOs) active in the health/AIDS sector has also increased significantly in the past decade, reflecting an increasing role of civil society and increased availability of donor funding.
Project Design and Strategic Context
5. The Kenya Medium Term Plan (U) on HIV/AIDS 1992-96 established the strategic framework for government and donor programs. The key priorities were to: (i) prevent the sexual transmission of HIV; (ii) prevent HIV transmission through blood; (iii) mitigate the social and economic impact of AIDS; (iv) enhance surveillance; and (v) coordinate research and AIDS control activities across sectors. The STI project sought to support this strategy through its three major objectives: (i) strengthen institutional capacity at the national and district levels to design, implement, monitor, and evaluate HrVVSTI interventions; (ii) promote preventive measures to reduce the risks of HIVVSTI transmission; and (iii) enhance both health sector and community provision of physical and psychological care and develop strategies to mitigate the social and economic consequences of AIDS. The project had three major components, parallel to the project's main objectives (see Box 1).
Box 1. STI Project Components Component (1) aimed to strengthen the institutional capacity at the national and district levels to design, implement, and evaluate interventions (US$13.9 million) through supporting and strengthening (a) national capacity to provide adequate policy, planning, coordination, supervision, and technical support related to STIs; (b) district capacity to plan, coordinate, implement, and evaluate integrated multi-sectoral HIV/AIDS activities; (c) national and district STD surveillance systems; (d) research grants to improve interventions supported by the project; (e) monitoring and evaluation of the degree to which the project's objectives are being met; (f) support for innovative nongovernmental (NGO) and community-based (CBO) organization programs; and (g) project management. Component (2) sought to promote preventive measures to reduce the risks of STI transmission (USDS26.7 million) through the following sub-components: (a) the provision of accessible, acceptable and effective clinical management of STDs; (b) the development and implementation of information, education, and communication (DEC) activities for STIs and HIV; and (c) the supply and distribution of condoms to districts, municipalities, NGOs, and to those providing health services to the military, police, and security services as well as to refugees. Component (3) sought to enhance both the health sector and the community provision of physical and psychological care, and to develop strategies to mitigate the socioeconomic consequences (US$19.4 million) through the: (a) support of measures to control Tuberculosis (TB); (b) treatment of opportunistic infections; (c) support of District-based NGO/CBO, home-based care and counseling for people with HIV/AIDS; and (d) promotion of occupational safety activities and minimization of risk exposure to health workers.
6. The STI project was designed and approved rapidly--in about nine months. The Project was an important component of the government's wider STI/HIV program, but was complemented by a range of other activities by government and other partners. In addition, three Bank-financed projects were under implementation in the health and population sector at the time of STI Project approval. (4) Shortly after STI Project approval, the United Kingdom (formerly ODA, now DfJD) agreed to provide $12.5 million in parallel financing for the HIV/AIDS Prevention and Care (HPAC) Project, which provided technical assistance and training to NASCOP and districts to support STI Project implementation--as well as support for NGO activities in Nyanza province. German KfW ($4 million) also provided parallel financing for the national program. While not official parallel financers, Canada (CIDA) and Belgian cooperation supported STI research and pilot testing of STI treatment in Nairobi and several other cities since 1991 (see below). Other donors, such as US AID, financed HIV/AIDS activities, much of it channeled through international NGOs. In all, donors provided about $20 million in parallel financing for the national HIV/STI program during the project implementation period. The Kenya HIV/AIDS Working Group, established in 1995 and convened by NASCOP, sought to mobilize funds for the national program and improve coordination among donors and government (responsibility for convening the Working Group shifted to the National AIDS Coordinating Council (NACC) in 1999).
7. To evaluate progress, the project also established targets to be achieved by the end of project implementation:
* 75 percent of the target population able to cite two ways to protect themselves from HIV/AIDS;
* 30 percent increase in reported condom use among the target population;
* 20 percent decrease in reported sexual partners; and
* Increase the appropriate case management of STIs to 70 percent.
8. Project design had several weaknesses, however. First, although project objectives were stated in terms of quantitative benchmarks for sexual behavior and STI treatment, no baselines were established, nor were there adequate mechanisms to collect the data necessary to track progress toward stated objectives. Second, project design gave insufficient attention to implementation arrangements or the capacity of implementing agencies to carry out proposed activities. Third, while the design document emphasized the importance of targeting high-risk groups (security personnel, truck drivers, commercial sex workers), the project did not establish specific mechanisms to target these groups (except for STI patients and military personnel). Targeted interventions for commercial sex worker, truck drivers, and other groups at high risk were undertaken in collaboration with other partners (e.g., USAJD, CIDA, WHO, UNICEF), but no overall strategy or monitoring program was put in place to ensure adequate coverage of these groups.
PROJECT IMPLEMENTATION--INITIAL DELAYS, SUBSEQUENT IMPROVEMENTS
9. The project encountered implementation problems initially, and disbursed less than $5 million in the first two years of effectiveness. (5) The reasons were several. First, to avoid procurement problems that had plagued previous projects, the project agreement required that the MOH use an independent procurement agent for all major procurements, and use of Letters of Credit as a payment method. These arrangements eventually proved successful, but the contract with the procurement agent was not finalized until mid-1997--resulting in a two-year delay in planned commodity procurements. Second, while the government agreed (for the first time) to decentralize authority to incur expenditures for HIV/AIDS activities to district medical officers, liquidity constraints and other bottlenecks soon emerged that constricted the flow of project funds for district activities. (6) Third, project design called for establishment of extensive partnerships with NGOs and community organizations, but did not establish clear guidelines or mechanisms for doing so, resulting in little direct funding for NGOs.
10. Efforts by the Bank and project staff to address these problems contributed to improved implementation performance in the latter years--with the exception of NGO activities. Following an intensive dialogue with the Bank and other partners, the government agreed--in 1999--to allow establishment of separate project accounts at the district level. (7) This was widely viewed as one of the most important reforms catalyzed by the project. Procurement of pharmaceuticals and other commodities proceeded smoothly once the agent was in place, and the use of Letters of Credit may have increased the number of companies bidding on contracts (since it assured payment). Procurement agents and Letters of Credit subsequently became standard feature of other Bank projects in Kenya. But because bottlenecks in resource transfers to districts persisted through most of the project, only 10 percent of project resources went directly to districts (against a planned 20 percent). In contrast, although the government agreed shortly after project approval to develop a Memorandum of Understanding (MOU) with the NGO community regarding use of project funds for NGOs, it was not finalized until 1997 and contained stipulations that effectively excluded most small NGOs (such as the requirement that NGOs produce a bank draft to guarantee any project funding received). In the end, the majority of financial support for NGOs came in the form of pharmaceuticals given to mission hospitals (approximately 20-25 percent of drugs procured).
11. The National ADDS and STI Control Program (NASCOP), a department in the MOH, was responsible for overall program management, while day-to-day project implementation responsibilities rested with a Project Management Unit (PMU). The PMU was proactive in implementing most project activities and actively engaged in the government's wider HIV/AIDS program. A project launch workshop brought together a wide range of stakeholders to discuss HIV/AIDS, suggesting a participatory approach to project implementation. But the original level of participation was not sustained: a number of donors, clinicians, and NGOs expressed concern regarding a perceived lack of consultation in the management of the project. In addition, financial management was poor in the initial years of project implementation. It improved following the secondment of a financial management specialist (financed by DfID) to NASCOP, but most annual audit reports by the Kenya Auditor General included qualifications. The expenditure tracking systems did not facilitate disaggregated reporting against specific project activities, which contributed to concerns among donor partners and local stakeholders regarding the transparency of project expenditures.
12. Prior to the midterm review in August 1998, it became evident to the Bank team that, despite progress in project implementation, the project's overall objectives were unlikely to be achieved without significant improvements in political and social context for the national HrV/AIDS program. The Bank therefore used the midterm review (MTR) to highlight the importance of improving the wider operating environment for HIV/AIDS activities, and to engage a wide range of actors inside and outside government and across sectors in discussions regarding STI and HIV strategies and programs. The project also sponsored a series of meetings and workshops to engage political leaders and civil society in discussions of HIV/AIDS. These efforts contributed to improvements in political support and in the operating environment for Hr//AIDS activities (see Box 2).
Box 2. Strengthening Political Commitment to Combat HIV/ATDS Perhaps the project's most important legacy is its contribution to strengthening the national HIV/STI policy framework and to improving the political environment for HIV/STI activities, in collaboration with other donors. One of the project's early activities was to support the MOH and NASCOP in the development of a Parliamentary Sessional Paper on HIV/AIDS to define a clear policy and legal framework. A secretariat led by the STI Project director drafted the paper, which was submitted to Parliament in 1996, and passed in 1997 as Parliamentary Sessional Paper No. 4. The paper established legal and regulatory provisions for issues such as blood screening and rights for persons with AIDS and called for establishing a multisectoral prevention and control strategy, enhancing resource mobilization, and establishing a National AIDS Council "... to provide leadership at the highest level possible." (MOH 1997) Two years later, however, recommendations of the Sessional Paper, including establishment of an intersectoral National AIDS Control Council, remained unimplemented. Government budgetary allocations for HfV/AIDS and efforts by central ministries to resolve constraints to HIV/STI program implementation continued to be weak. Political leaders rarely spoke out publicly about HIV/AIDS, and when they did it was in the context of calls for "moral uprightness." The leadership of the Catholic Church took a strong stand against condom distribution and sexual education for adolescents, including sponsoring the public burning of condoms and of "family life education" books in 1996-97. The President's relative silence was seen as particularly problematic, given that in the Kenya political system, government and political leaders largely took their cues from him. In 1998, the project and several donor partners sponsored a meeting among professional and political leaders in Nyanza Province, at which the President issued his first strong public statement regarding HIV/AIDS, including a call for increased condom use (Elderkin 1998). The PMU also helped sponsor (originally with counterpart funds) a special two-day 1999 Parliamentary session in Mombasa at which the President declared AIDS a National Disaster, a watershed event. The project, together with UNICEF and other partners, sponsored other regional meetings for political and civil society leaders, as well as a series of district-level constituency meetings. The Bank and donor partners also made clear to GOK that both increased efforts on HIV/AIDS and improved governance were preconditions for future support. Interviews and press clippings suggest that following these events, political leaders at both national and district levels began to speak out more regularly on HIV/AIDS. Those interviewed by OED widely agreed that the environment for HIV/AIDS activities has improved as a result, compared to the silence that prevailed when the project began. The National AIDS Control Council (NACC) was finally established in 1999, although it took about two years to become fully operational.
13. Relevance--the extent to which the project design was consistent with borrower and Bank strategies--is judged to be substantial. The original project design was consistent with the Bank's Country Assistance Strategy (CAS) and the government's Medium Term Plan on HIV/AIDS. It was consistent with research findings, in Kenya and internationally, regarding STI treatment and prevention. But project design gave inadequate attention to implementation arrangements, and initially underestimated government commitment to combating HIV/AIDS. These imbalances were partially remedied prior to and following the midterm evaluation.
14. Project efficacy--the extent to which stated objectives were achieved--varied among the major project objectives. The institutional capacity component and prevention components each substantially achieved their objectives, although each with shortcomings. The project made only a modest contribution to care and mitigation of the consequences of HIV/AIDS, however. The section below describes the project's contributions to improving STI treatment, behavior change, and care and mitigation for AIDS patients. The contributions of the capacity building components are assessed in the section on institutional development. Overall efficacy is rated as substantial.
Improving Treatment of STIs
15. The project helped improve the quality and coverage of treatment for STIs through development and dissemination of national guidelines for STI treatment, provider training, and procurement and distribution of STI drugs. This may in turn have contributed to a reduction in treatable STIs. National coverage for STI drugs was only achieved from late-1999, however, due to procurement delays. Drug shortages reemerged in mid-2001 following project completion.
16. STI treatment protocols. The project supported the development of national guidelines for syndromic treatment of STIs--in which providers diagnose and treat STIs using standard protocols based on symptoms (see Box 3). The project also financed the printing and widespread dissemination of a chart summarizing STI treatment protocols. (OED observed the chart on the wall of nearly every health facility visited.) These guidelines helped standardize STI treatment and is reportedly used by both public and private providers. The national protocol was consistent with WHO recommendations and drew on earlier pilot testing of syndromic management in Kenya (Moses and others 2001). It was further modified based on an initial procurement of drug kits financed by DflD (see below). Some disagreement persists, however, as to whether two injectable drugs that were added to the protocol (for first-line treatment of urethral discharge, and for second-line treatment of pregnant women) represented the most cost-effective options, and whether the decision to modify the protocol was based on adequate research and consultation.
17. Training in STI treatment. The project supported extensive training for syndromic management training, beginning in the 15 pilot districts and later expanding nationwide. The training covered STI treatment guidelines, patient counseling, provision of condoms, and partner tracing, and complemented training provided by other partners (including CIDA, Belgian cooperation, and DflD) in about a dozen other districts. The coverage of training appeared to be higher in the 15 pilot districts, where it was initiated earlier. The ICR reports that "several staff' in each district have received training in STI case management, but there is no available documentation regarding the total number or percentage of staff trained in STI treatment and counseling--either by the STI Project or other donors--nor are there systematic data regarding the impact of training on treatment quality. Although some districts reportedly included participants from the mission and private sectors in training, the project gave relatively little attention to training private providers, despite evidence that many clients prefer private care for STI treatment.
Box 3. Syndromic management of STIs One of the project's key objectives was to improve the quality and coverage of treatment for Sexually Transmitted Infections (STIs) nationwide. STIs also can be painful, and can lead to infertility and chronic health problems. But by the early 1990s, a growing body of biomedical research also suggested that the presence of other STIs (such as syphilis or gonorrhea) facilitated the transmission of HIV, and that treatment of STIs might reduce the risk HIV transmission (Fleming and Wasserheit 1999). Because laboratory testing is frequently unavailable in developing countries, WHO developed a syndromic approach to STI treatment, in which clinicians diagnose and treat STIs using standardized protocols based on symptoms (Dallabetta and others 1998). Beginning in 1990, Canadian CIDA and Belgian Cooperation provided support for a program at the University of Nairobi to pilot syndromic management of STIs in Nairobi city clinics, and to promote STI/HIV interventions for commercial sex workers in the Nairobi region (Moses and others 2001). The STI project sought to scale up this syndromic approach nationwide through development and dissemination of a national STI protocol, provider framing in STI treatment, purchase and distribution of standardized STI drug "kits," and communications campaigns to encourage clients to seek treatment for STIs.
18. Improved availability of STI drugs. The project improved the availability of drugs for treating sexually transmitted infections (at a cost of US$12 million). Due to delays in hiring the procurement agent, however, the two-year supply of STI drugs procured by the project did not arrive until December 1999. To bridge this gap, DflD agreed in 1997 to finance an emergency procurement of STI drugs, which were distributed directly to 500 health facilities nationwide (out of over 2000). Following the distribution of project-financed drugs, STI drugs were widely available from early 2000 to the middle of 2001, at both hospital and clinic level. In late-2000, 80 percent of health centers surveyed reported an adequate supply of STI drugs in previous three months, and 94 percent of clinics sampled had received STD drugs (Nganda and others 2001). About one-fourth of the drugs purchased by the project were given to mission hospitals, helping to improve STI drug availability in these facilities. German KfW also provided funding for STI drugs (about US$3 million). Increased availability of drugs appears to have to a sharp increase in the number of patients seeking care at government health facilities. Available data from NASCOP suggest that the reported number of STI cases treated in government health facilities increased from about 150,000 in 1999 to 260,000 in 2000. These data do not include about 40 percent of government health facilities (which failed to file reports), or NGO or Mission health facilities, so NASCOP estimates that the actual cases treated were more than double these amounts (300,000 in 1999, and 640,000 in 2000).
19. In addition to the initial procurement delays, several other issues arose regarding the procurement and distribution of drugs financed by the STI project (see Annex C). First, the decision to use a drug "kit" system--rather than to procure and distribute drugs individually--helped ensure that a complete set of drugs are available, and was probably appropriate given the prevailing weaknesses in the national drug procurement and distribution system. Each kit contained the complete set of STI drugs, which were to be delivered directly to each hospital or clinic. But the kit system is inherently supply-driven rather than demand-driven. Despite efforts to adjust drug quantities based on experience, some drugs were supplied in excessive quantities (Probenicid) and others undersupplied (Amoxycillin) relative to clinical needs. In addition, the project did not adequately anticipate the challenges involved with storage and distribution of such a massive shipment of drugs, contributing to difficulties in drug distribution, particularly to rural clinics in districts with inadequate transport. Third, although the contract with the supplier specified that the drugs were to have at least a two-year shelf life, some of the drugs procured arrived within nine months of their expiration dates. (8) In response, the MOH issued a circular permitting facilities to use drugs procured through the project to treat other ailments. Finally, sustaining the STI drug supply remains a challenge. During the final year of the project, the Bank, government and donors sought to develop a more coordinated approach to the procurement and sustainability of pharmaceuticals and medical supplies. The Bank agreed to finance several large procurements under the DARE project--including for condoms--with the expectation that donors would finance other priorities, including STI drugs. Yet despite these efforts, wide-spread stock-outs of STI drugs emerged beginning in the latter half of 2001. Emergency procurements from donors and the Bank are expected to bridge the gap in 2002, but the Bank has conditioned further financing of STI drugs on the development by MOH of a medium term expenditure and financing plan for essential drugs and supplies.
20. 577 case management may have improved. There are few evaluations and no reliable trend data on appropriate case management of STIs, so it is unclear if the project's original 70 percent target for appropriate case management has been met. According to records reviewed in a sample of districts, 88 percent of patients treated for STIs did not return for further treatment (Nganda and others 2001). This could suggest a high cure rate, but patients could have returned to other providers (private or traditional) or self-medicated. In terms of health education, recent reviews suggest that despite training, providers are not providing adequate counseling on behavior change or condom use. (9)
21. Prevalence of treatable STIs may have declined. The limited available evidence, together with anecdotal reports from clinicians, suggests that prevalence of bacterial STIs--including syphilis and gonorrhea--may have declined over the course of the project. Although behavior change could have contributed, it is plausible that improved STI treatment played a role (see Annex B). These changes in turn cannot be attributed specifically to the STI Project, but as a major source of support for the national STI program, it probably contributed. Results from research in Nyanza Province suggest, however, that while the prevalence of many bacterial STIs is relatively low, the prevalence of genital herpes (HSV-2)--an ulcerative STI that is strongly associated with HIV transmission, but cannot be cured--is very high. In Kisumu, two-thirds of randomly selected women tested positive for genital herpes, and 39 percent of young women aged 15-19 (Weiss and others 2001). While genital herpes does not respond to treatment with antibiotics, the risk of transmission can be reduced through reduction of sexual partners and increased condom use--which also reduce the risk of acquiring HIV. In addition, the prevalence among women in Kisumu of trichomaniasis infection--which can be treated with antibiotics but is often asymptomatic--was also high (29 percent) (Buve and others 2001).
22. Uncertain impact of STI treatment on HIV transmission It is possible that the Kenya STI treatment program contributed to a modest reduction in HIV incidence, through a reduction in treatable STIs. But the impact on HIV transmission was probably reduced by variations in the quality and coverage of STI treatment among districts; the high prevalence of genital herpes; mixed success in partner tracing; uneven coverage of the highest risk groups; and possibly by the high prevalence of trichomoniasis (see Annex B).
Encouraging Behavior Change
23. The project sought to increase awareness of HIV/STIs and encourage behavior change through providing US$7 million in financing for information, education, and communication (IEC) activities, sponsored at the national and district levels. These activities may have contributed to increased awareness of HIV/AIDS and STIs, and to changes in sexual behavior (see below). But with the exception of the audiovisual component (over US$2 million), the cost and outputs of these IEC activities are poorly documented. According to MOH estimates, about US$2.5 million was spent for HIV/AIDS/STI awareness raising at the district level, schools, and communities; about US$1.3 million was spent on printing and dissemination of IEC materials, and US$1.3 million for IEC targeted on improving provider and patient awareness regarding syndromic STD care. The project also financed the development of an EEC strategy for a health education strategy for the MOH (MOH 1998). Yet the overall IEC campaign did not seem to be based on a consistent strategy, nor did it give adequate attention to the efficiency or effectiveness of project activities. In the messages produced, the emphasis on behavior change compared with encouraging clients to seek treatment for STIs varied. Overall, OED is concerned that the output of the IEC component does not appear commensurate with the expenditures.
24. National and district IEC. The project financed posters, pamphlets, and billboards, and publications such as the annual AIDS in Kenya report produced by NASCOP (MOH 2001), and also sponsored events such as World AIDS Day. Most of the printed materials were produced in the early years of the project, and supplies are now mostly exhausted. (10) Posters seem to be of acceptable graphic quality, but the messages tended to be generic. Printed materials and bill boards were reported to have been translated into 11 local languages, but OED found no documentation regarding pre- or post-testing of printed materials or billboards. (11) A number of national and district stakeholders expressed concern about the relatively poor quality and location of billboards. District staff used funds from the project for local IEC and outreach activities. The PMU estimates that 150,000 teachers, health workers, district development committee members, and community, church, and youth leaders were "sensitized" through the project. In addition, an estimated 7,500 health workers received DEC training and materials related to STD treatment. While many of these various activities and materials were undoubtedly useful--and may have contributed to increased awareness of HIV/STIs--the cost, coverage, and quality of IEC activities are not well documented..
25. Multimedia campaign. The multimedia contract was bid through the procurement agent--thus was not awarded until 1998. The radio and TV programs, and videos produced appeared to be of good production quality, and were disseminated through a number of channels, including public broadcast and mobile videos to all districts. The private sector company responsible for the contract produced a detailed report documenting outputs, focus group pretesting and post-testing, and dissemination activities. (12) Copies of videos were available in district libraries, and are reported to be used by various groups, although lack of equipment constrained showing of videos in many districts. But due to delays in this contract, most of the dissemination activities took place in 1999, and were not subsequently continued due to lack of funding.
26. HIV/AIDS curriculum development and youth activities. The development of a sexual health curriculum was hindered during most of the project's life by opposition from religious groups, and a lack of champions within government. UNICEF took a lead role in this area, but the Bank sought to encourage progress, particularly after the MTR. A curriculum now has been developed, and some teachers' guides distributed, but distribution of student texts and teacher training remain to be implemented. The project sponsored few activities that targeted youth outside of school settings, except for a few ad hoc activities at the district level.
27. Targeting of IEC. Bank and project staff reported having extensive discussions regarding how and whether to target EEC messages, including the appropriate balance between focusing on those most likely to infect others, compared to those at highest risk of becoming infected. But of the EEC messages and materials reviewed by OED, none seemed to be targeted at groups at groups at highest risk of transmitting the HIV virus--such as commercial sex workers, truck drivers, or migrant workers. Moreover, because of the ad hoc nature of NGO support, the project missed an opportunity to significantly scale up peer counseling and other interventions among high-risk groups, which have been shown to be effective in changing behavior and reducing STIs (Rakwar others 1999; Moses and others 2001). An important exception was the security forces. The project conducted seminars for the security forces on condom promotion, peer education, STI treatment, and HIV counseling and testing. Training in HIV/AIDS and condom use has been made a requirement in the military, and a reported 30,000 condoms per months were distributed to the armed forces during the project.
28. Impact on behavior? Awareness of HIV/AIDS is now nearly universal: according to a survey completed at the close of the project, the percentage of adults able to cite two or more ways to prevent HIV transmission exceeded 90 percent (Nganda and others 2001). Increased awareness has been slow to translate into behavior change, however. Since the mid-1990s, there have been modest reductions in risky behavior, including reductions in the number of casual partners and increased condom use with non-regular partners. But risky behavior remains common, particularly among youth (See Annex B). Results from household surveys, interviews with STI clients, and observations of provider behavior suggest several findings. First, while EEC activities may have contributed to awareness and behavior change, the impact was probably reduced by inadequate targeting, pre-testing, and dissemination. Second, despite training, health providers are not yet providing adequate HIV prevention education and counseling to STI patients or other clients. For example, an analysis of the 1997 DHS found no correlation between learning about AIDS from a health provider and condom use (Waithaka and Bessinger 2001).
Increased public discussion of HIV/AIDS by national and local political leaders in the late-1990s may have contributed to increased awareness, although there is no way to assess the impact on behavior.
Enhancing Care for AIDS Patients
29. The project sought to enhance care and mitigate the social consequences of AIDS through several means, including financing antibiotics for treatment of opportunistic infections, and supporting activities by community based organizations (CBOs) and NGOs. Although the project allocated US$19 million for care and mitigation, its major financial contribution to care for AIDS was the procurement of $4 million in drugs and supplies to treat opportunistic infections (remaining funds were reallocated to other project activities). These drugs (including anti-fungals, anti-diarrhoeals, antibiotics for skin and eye infections) were distributed to hospitals and clinics nationally, including mission hospitals (about 20 percent of the drugs). Some clinicians and partners raised concerns, however, regarding inadequate training and treatment protocols for use of the drugs (particularly the anti-fungals). There are no data or evaluations available regarding the actual use or clinical effectiveness of these drugs. The project also financed $200,000 for anti-TB drugs; programmatic support for TB interventions was financed by other partners.
30. A consequence of the limited support for NGOs/CBOs was that community and home-based mitigation and care activities went mostly unfounded. In the 1999 district survey fewer than 15 districts reported sponsoring home-based care activities. The HP AC Project sponsored a training of trainers session for home-based care in 1998, but no further training was completed due to lack of funds. The project's financial reporting system did not separately track funding for NGOs, but it appears that the project's contributions were minimal.
31. Although it is not possible to estimate a rate of return for proj ect investments, the project's efficiency can be considered from two perspectives. First allocative efficiency is the extent to which the project (in the context of the national program) focused on the highest impact investments, compared to alternatives. STI treatment and behavior change communications financed by the project can be cost effective if effectively designed and implemented, but might have been more cost effective if they had focused more explicitly on behavior change and STI treatment interventions among groups at highest risk of transmitting HIV. Further, neither the project nor national program gave adequate attention to promoting behavior change among young people under age 19. Cost efficiency is the extent to which project investments represented good value for money. The use of the procurement agent for all the major commodity procurements resulted in substantial cost savings over the purchase on the local market. But these savings may have been reduced by (i) sub optimal use of some drugs (due to problems in distribution and shelf life); and (ii) purchase of some patented drugs when generics might have sufficed. Finally, the substantial expenditures on information, education, and communications did not appear to have been used efficiently. Efficiency therefore is rated as modest.
32. The project's capacity building component had mixed success in strengthening institutional capacity for design, implementation, and evaluation of HIV/STI programs (see below). But the project made substantial contributions to changing the "rules" for project implementation and budgetary transfers to districts, and to strengthening political commitment for combating HIV/AIDS (see Box 2). Institutional development therefore is rated substantial.
33. Modest contributions to strengthening national capacity. Much of the direct support for capacity building at the national level was financed through the DfTD-sponsored HP AC project. The HP AC project financed construction of a new headquarters for NASCOP--completed in 1999--which brought NASCOP staff under the same roof for the first time. Both the STI and HP AC projects provided training and technical assistance in program planning and management, and financial management. But for several reasons these project investments led to only modest improvements in NASCOP's ability to plan, implement, coordinate, and evaluate HIV/STI programs. First, staff turnover and sometimes limited interest on the part of counterpart staff reduced the impact of training. This explains, for example, the limited progress in building financial management capacity in NASCOP. Second, in late 1997 NASCOP was demoted from a Department of the MOH to a division under the Department of Preventive and Promotive Health Services, which undercut its ability to lead the national HIV/STI response. Third, although the STI Project was technically under NASCOP, 90 percent of all NASCOP activities were financed through the STI Project. The tendency of the STI project unit to take on policy and implementation responsibilities complicated efforts to strengthen NASCOP. (13)
34. Substantial strengthening of district capacity. The project contributed to strengthening the response to HIV/STIs at the district level--starting with 15 pilot districts and expanding in 1997 to all 69 districts--although the impact appears to have been greatest in the pilot districts. District Health Management Teams (first in the 15 pilot districts, and later in all districts) were required to prepare annual HIV/AIDS work programs in order to receive project funds. This gave the district teams valuable experience in planning and implementing HIV/STI programs. The actual funding received tended to be only a fraction of the total on the approved work program, however. (14) But government's agreement in 1999--following intensive discussions with the Bank--to establish local project accounts should facilitate future activities, and has proved to be a key step toward increased local control and accountability for the health sector and HIV/AIDS programs (see above).
35. In addition, the project helped establish District AIDS and STI Coordinators (DASCOs) and District Intersectoral AIDS Coordinating Committees (DIACs). The DASCOs have played an important role in organizing the district-level response to STIs/HIV. During field visits, OED was impressed by the training and motivation of the DASCOs, but the quality and motivation reportedly varied significantly among districts. Similarly, while DIACs in some districts were proactive in reviewing community funding proposals and coordinating local government activities, others were ineffective. The project sponsored training for DASCOs, District Medical Officers, and later district accountants in program planning and management. Less than a quarter of district health team members received training in management skills, however. In addition, the project provided computers, office equipment and vehicles to over 40 districts, which reportedly helped strengthen both HIV/STI and general health activities. The supervision and monitoring of district activities was weak, however. NASCOP had formal responsibility, but inadequate staff and capacity. A greater role for provinces might have strengthened monitoring and supervision of district HIV/STI activities. Provinces were mostly bypassed during project design, but following the midterm review, the project provided support to strengthen the role of Provincial Medical Officers in monitoring and providing technical support for district-level activities.
36. Negligible contributions to NGO capacity. The project provided limited direct financial support for local NGOs and none for national NGOs. The Memorandum of Understanding with the NGO community had limited impact on project support for NGOs (see above). Following the midterm review, the MOH agreed to initiate bidding for three "umbrella" contracts with national NGOs, to provide additional support and capacity building for local NGOs. These contacts were never awarded. Through support for district activities, the project also provided limited support for about three dozen local NGOs. (15) But the local support was small scale and ad hoc, which did not allow local NGOs/CBOs to develop specific programs or build capacity for HIV/AIDS prevention or mitigation activities. While there were some administrative obstacles to providing direct NGO support, it appears that the government gave low priority to financing these activities. (16) The Bank regularly raised concerns during project supervision regarding the slow progress of NGO support; the disappointing project experience resulted in a new approach to financing of NGO activities under current Bank HIV/AIDS projects.
37. Modest contributions to strengthening surveillance. The project financed US$1.9 million for HIV test kits and equipment to support the national HIV sentinel surveillance system. These inputs, together with technical assistance from other donors, expanded the coverage and consistency of the national HIV surveillance system. (17) Test kits were also used to help screen the blood supply. Some test kits were delivered close to expiration dates, but the MOH reports that expired test kits were routinely replaced by the supplier. Several other concerns merit noting. First, surveillance data are infrequently used at the district or province level to develop prevention strategies and target interventions. Second, district staff received inadequate training and support for maintenance for the more sophisticated equipment. Third, significant year-to-year variations in reported prevalence for some sites suggests the need for greater coverage and further strengthening of data quality (efforts are underway with donor financing). (18) Fourth, because the system relies on women seeking antenatal care, there is limited data regarding HIV or STI prevalence trends in specific high-risk groups. Finally, although sentinel surveillance collects data on syphilis, there are no reliable trend data on the prevalence of other STIs--which complicates the targeting and impact assessment of STI treatment programs.
38. Negligible contributions to research, monitoring, and evaluation. The lack of attention to the design and implementation of research, monitoring and evaluation was one of the project's greatest weaknesses. Most of the planned operational research, monitoring, and evaluation activities were not completed. Basic data on project outputs are not easily available at either the national or district level. Districts do not yet have effective health information systems, despite provision of computers and training. Staff recorded STI visits and sent the monthly returns to NASCOP, but these were simply stacked in file cabinets. Thus, despite the project's emphasis on improving treatment of STIs, neither districts nor NASCOP had trend data available on STI cases (NASCOP is in the process of entering and analyzing this backlog of data; preliminary results were shared with OED and incorporated into the discussion above). Reporting on local HIV/STI activities was weak, and no independent evaluations were conducted on any of the HIV/STI activities sponsored by districts. As a result, not only do we know little about project impact, the project missed an important opportunity to improve knowledge about what works in HIV/STI prevention and mitigation.
39. Outcome is rated moderately satisfactory. This rating is consistent with the preceding ratings of substantial relevance and efficacy, with modest efficiency. The project's major contributions may have been at the policy level--both promoting reforms in financing for district-level HIV/ATDS activities, and strengthening the national policy framework for and political commitment to HIV/ATDS. The project was one component of a larger program, and weaknesses in monitoring and evaluating make it difficult to assess whether the stated outcome targets were achieved. The project improved during its lifetime the quality and coverage of STI treatment, which may have reduced the prevalence of treatable STIs. The project may have made a modest contribution to reductions in risky behavior--either through direct support for TEC or its broader contributions to strengthening political commitment and public discourse regarding HIV/ATDS. The project's direct contribution to reducing HIV transmission cannot be documented (see Annex B).
40. Sustainability is rated as likely. Reinforcing sustainability, (i) the government's commitment to addressing HIV/ATDS appears to remain strong; (ii) most of the institutional reforms introduced by the project are being continued; and (iii) a number of STI Project activities are being continued through subsequent World Bank and donor projects. In addition, estimated $650 million in support has been mobilized to finance the Kenya National ADDS Strategic Plan for the period 2000-2006--half from government, the other half from donors and international partners. But despite efforts to plan for sustaining the STI drug supply following project closure, STI drug availability fell sharply in 2001, necessitating emergency procurements by donor partners and the Bank. The MOH has agreed to work with the Bank and other partners to develop a medium term expenditure and financing plan for essential drugs and supplies, and the MOH plans to integrate STI drugs into the essential drug supply system. These measures may reduce the risks to future disruptions in drug supply.
41. Bank performance is rated satisfactory. Quality at entry was satisfactory, despite inadequate attention to project implementation arrangements. Project supervision was unsatisfactory initially, but improved substantially prior to and following the midterm review. The Bank made effective use of both local and international staff in seeking to resolve implementation problems and in engaging government and stakeholders in dialogue regarding the importance of strengthening the national response to HIV/AIDS.
42. Borrower performance also is rated satisfactory. Borrower performance was satisfactory during project preparation, but unsatisfactory during the first 2-3 years of project implementation--on the part of the government overall as well as the implementing agencies. Financial accountability was weak, and only a fraction of originally intended funds were disbursed to districts or NGOs. Senior government officials and political leaders initially were reluctant to speak openly about HIV/ATDS. But beginning in the late-1990s, the government made substantial progress in establishing a policy and organizational framework for the national response, and local and national political leaders are speaking openly about HIV/AIDS. Project implementation performance also improved prior to and following the midterm review, as a number of long-standing project implementation bottlenecks were resolved.
FINDINGS AND LESSONS
43. Project experience suggests several findings and lessons.
Lesson 1: Monitoring and evaluation programs require clear and measurable indicators that are systematically collected over time. This includes tracking project outputs, as well as program outcomes. The project's weak performance in M&E means that it made only a limited contribution to understanding what interventions are likely to have the highest impact on HIV/STIs.
Lesson 2: The socio-political environment has a strong influence on the effectiveness of HIV/STI prevention activities--both in terms of the openness of public discourse, and in government support for implementation. The Bank initially overestimated political commitment. Subsequent dialogue and the convening of discussions among high-level political leaders and stakeholders contributed to improvements in the political context for HIV/AIDS, but was a long-term process.
Lesson 3: Decentralizing project activities to the district level can improve local implementation, but needs to be matched by giving districts sufficient authority to incur expenditures, improve financial flows to districts, strengthen accountability, and build capacity for planning and management. These efforts were ultimately successful, but required several years of intensive dialogue. In addition, project experience suggests that Provincial Medical Officers can play an important role in monitoring and providing technical support for district-level activities.
Lesson 4: Lack of disbursement to NGOs may reflect a lack of government commitment as much as administrative difficulties. Although NGOs may be well-positioned to reach high-risk groups and implement local ADDS care and mitigation programs, project and national programs should develop strategies to ensure that high-risk groups are targeted and that mechanisms are in place for monitoring, evaluation, and sharing lessons among NGOs.
44. The Bank recently approved two new operations to strengthen the national response to HIV/ATDS and the health system overall: the Decentralized AIDS and Reproductive Health (DARE) Project (US$50 million), and the Kenya AIDS Disaster Response Emergency (KADRE) Project (US$50 million). DARE supports the MOH in strengthening the delivery of health services, HIV/STI prevention and care at the district level. It builds on the progress of the STI project, and incorporates a number of lessons learned. For example, it is being implemented through MOH structures rather than a PMU. Credit effectiveness was delayed for over nine months pending implementation of several project conditions, including establishment of a new formula for budgetary distributions to districts.
45. In contrast, KADRE--the first of the Bank's Multi-country AIDS Program (MAP) projects--was designed and approved in less than a year. KADRE will provide support for HIV/AIDS prevention and care in non-health ministries--including establishment and strengthening of the new multisectoral coordinating structures. These include the National ADOS Control Council (NACC), Provincial ADOS Coordinating Committees and District AIDS Coordinating Committees (DACCs), and local Constitutuency AIDS Coordinating Committees. Having two projects has helped reduce tensions among MOH, NASCOP, and NACC, while allowing the health sector to continue its past activities without waiting while the new multisectoral structures are consolidated.
46. The implementation of the civil society component of KADRE has been delayed pending the selection and engagement of an independent financial management agency. But a number of informants reported that this delay--while generating questions from local constituents as to why funds are not flowing--has in fact been beneficial, in that it has given additional time to establish the institutional framework for NACC and the process for review and funding of civil society proposals. In particular, the Provincial, District, and Constituency AIDS committees are still very new, and most of their members have limited technical expertise or experience with planning and monitoring HIV/AIDs programs.
47. KADRE is to channel about a third of its support directly to communities and NGOs, while DARE supports all HIV/AIDS activities within the health sector. KADRE funds will be managed by an independent financial management agency, which will disperse funds directly to NGOs/CBOs once proposals are approved. There is lingering skepticism and mutual suspicion between government and NGOs, a problem that was partly exacerbated by the STI project, but NGOs are cautiously optimistic about this new arrangement. Another challenge is to reconcile a demand-driven approach to financing local NGOs/CBOs with the need for a clear strategy to finance activities that are likely to have the highest impact. The District and Constituency AIDS Committees will likely need significant support to develop such strategies.
48. The location of NACC in the Office of the President gives it a higher political profile, and greater leverage in coordinating other ministries than if it were located in the MOH. Yet because the Office of the President is clearly a political body, in a system where governance concerns persist, maintaining the transparency and credibility of NACC is critical. This applies both to promotion and appointment of personnel based on technical qualifications only, as well as transparent decisions regarding allocation of funds. Similarly, a challenge for the district and constituency coordinating committees is to ensure a high level of involvement by local political leaders without politicizing these bodies.
49. The introduction of the 'multisectoral approach' to HIV/ATDS, which is being supported by the KADRE project, has raised questions regarding the creation of overlapping or possibly competing structures to those established under the STI Project. For example, with the establishment of DACCs, the future role of District Intersectoral ADDS Committees is unclear. Moreover, most of the technical expertise regarding HIV/ATDS still resides within the MOH, and some of those in charge with coordinating the current multisectoral response--particularly at the district levels--have limited technical knowledge of HIV/STIs. Training is underway, but the skill and knowledge gaps remain formidable. The decision to designate District Development Officers to serve as the new District ADDS Coordinators under the National ADDS Control Council has led to some confusion regarding the future roles and responsibilities of the DASCOs. It might be worth opening up the District AIDS Coordinating position to other qualified staff, including the existing DASCOs.
50. In establishing strategic priorities at the national and local levels, several issues deserve highlighting. First, enhancing interventions targeted at young people--both in and out of school--to delay the onset of sexual activity and increase condom use will be critical to reducing HIV incidence among this group (Laga and others 2001). Second, although condom use has increased, risky sex remains common and emphasis needs to be given to ensuring consistent condom use. Third, expanding quality voluntary testing and counseling (VCT) services deserves attention, given its potential impact on personal behavior among those who seek it (Coates and others 1999; Painter 2001), and given that an estimated half of future incidence will result from discordant couples (where one spouse or regular partner has HIV, but not the other). Fourth, in light of the significant variations in HIV prevalence and sexual behavior patterns among regions within Kenya, it will be important to help develop locally-tailored prevention strategies, particularly to target groups and geographic areas with high rates of transmission. Fifth, the national STI treatment protocols may need to be revisited based on recent research, the prevalence of different STIs, and evidence of resistance to specific antibiotics--and provider training strengthened to improve behavior change counseling during STI treatment. Finally, establishing an effective M&E framework for the national program and projects is essential. Efforts are underway to establish indicators and baselines. But the government, Bank, and partners will need to give consistent attention and priority to ensure that plans for M&E are effectively implemented, in order to learn more about what works best and why.
Agha, Sohail. 2001. "The Impact of the Kenya Social Marketing Program on Personal Risk
Perception, Perceived Self-efficacy and on other Behavioral Predictors." PSI Research Division Working Paper N[degrees] 45. Washington, DC.
Arthur, Gilly and others. 2000. "The Changing Impact of HIV/AIDS on Kenyatta National Hospital, Nairobi from 1988/89 through 1992 to 1997. AIDS" 14:1625-1631.
Bernstein, Robert, David Sokal, Steven Seitz, and others. 1998. "Simulating the Control of a
Heterosexual HIV Epidemic in a Severely Affected East African City." Interfaces, 28: 3 May-June.
Buve, A. and others. 2001. "The Epidemiology of Gonorrhoea, Chlamydial Infection and Syphilis in Four African Cities." AIDS 15 (suppl 4):S79-S88.
Carael, Michel, and King Holmes. 2001. "Dynamic of HIV Epidemics in Sub-Saharan Africa: Introduction." (The Multicentre Study of Factors Determining the Different Prevalences of Hiv in Sub-Saharan Africa). AIDS, Vol. 15, Supplement No. 4
Coates, T. and others. 1998. "Voluntary HIV Counseling and Testing (VCT) Reduces Risk Behavior in Developing Countries: Results from the Voluntary Counseling and Testing Study." Paper presented at the International Conference on AIDS, Geneva, 28 June-3 July.
Dallabetta, Gina, Antonio Gerbase, and King Holmes. 1998. "Problems, Solutions, and Challenges in Syndromic Management of Sexually Transmitted Infections." Sexually Transmitted Infections. Vol. 74, Supplement 1: S1-S11.
Elderkin, Sarah, ed. 1998. "HIV/AIDS and Development in Nyanza Province: Leadership Symposium." Nyanza Professional Caucus, Kisumu.
Feldblum, Paul J. and others. 2001. "Female Condom Introduction and Sexually Transmitted Infection Prevalence: Results of a Community Intervention Trial in Kenya." AIDS 15:1037-1044.
Fleming, D.T., and J.N. Wasserheit. 1999. "From Epidemiological Synergy to Public Health Policy and Practice: the Contribution of Other Sexually Transmitted Diseases to Sexual Transmission of HIV Infection." Sexually Transmitted Infections. 75: 3-17.
Glynn, J. R. and others. 2001. "Why Do Young Women Have a Much Higher Prevalence of HIV than Young Men? A study in Kisumu, Kenya and Ndola, Zambia." AIDS 15 (suppl. 4): S51-S60.
Grosskurth, Heiner, Frank Mosha, James Todd, and others. 1995. "Impact of Improved Treatment of Sexually Transmitted Diseases on HIV Infection in Rural Tanzania: Randomized Controlled Trial." The Lancet. Vol. 346. August 26.
Horizons. 2001. "Results from an Exploratory Study in Nairobi, Kenya, and Kampala and Masaka, Uganda. HIV Voluntary Counseling and Testing Among Youth." October. Horizons Program: International Center for Research on Women and Population Council.
Jackson, Denis J. and others. 1999. "Stable Antenatal HIV-1 Seroprevalence with High Population Mobility and Marked Seroprevalence Variation among Sentinel Sites within Nairobi, Kenya." AIDS 13:583-589.
Kenya AIDS NGO Consortium (KANCO). 1998. "Lessons Learned: The Role of Networking in Policy Development and Advocacy Process." Nairobi, Kenya.
Kenya Demographic and Health Survey (KDHS). 1998. Kenya Demographic and Health Survey, 1997. National Council for Family Planning and Development and Macro International. Nairobi, Kenya.
Kenya Demographic and Health Survey (KDHS). 1994. Kenya Demographic and Health Survey, 1993. National Council for Family Planning and Development and Macro International. Nairobi, Kenya.
Korenromp, Ellen. 2001. Treatment of Sexually Transmitted Diseases As an HIV Prevention Strategy? Cofactor Magnitudes, Syndromic Management and a Reappraisal of the Mwanza and Rakai Trials. Doctoral Dissertation, Erasmus University, Rotterdam, The Netherlands.
Laga, Marie, Bernhard Schwartlander, Elizabeth Pisani and others. 2001. "To Stem HIV in Africa, Prevent Transmission to Young Women." AIDS 15: 931-94.
MacDonald, Kelly S. and others. 2001. "Vitamin A and Risk of HIV-1 Seroconversion among Kenyan Men with Genital Ulcers." AIDS 15:635-639.
Ministry of Health. 1997. AIDS in Kenya. Sessional Paper No 4 of 1997. Nairobi, Kenya.
--. 1998. "National Health Communication Strategy: 1999-2010." Division of Health Education. Nairobi, Kenya.
--. 2001. AIDS in Kenya: Background, Projections, Impact, Interventions and Policy. Sixth edition. MOH in Collaboration with the National AIDS Control Council. Nairobi, Kenya.
Moses, Stephen and others. 2001. "Response of a Sexually Transmitted Infection Epidemic to a Treatment and Prevention Programme in Nairobi, Kenya." Sexually Transmitted Infections. 78:0-6.
National AIDS Control Council (NACC). 2002. "Measurement of HIV Seroprevalence in Kenya through Sentinal Surveillance 2001: Challenges in Rural Measurement and Description of Trends." Report to the National AIDS Control Council, Technical Working Group. January 30. (PowerPoint presentation). Nairobi, Kenya.
--. 2000. The Kenya National HIV/AIDS Strategic Plan 2000-2005. Popular Version. October. Nairobi, Kenya.
Nganda, Benjamin, Erastus Njeru, M'Imunya Machoki and others. 2001. "Baseline Study for the Decentralised HIV/AIDS and Reproductive Health (DARE) Project," Clinical Epidemiology Unit, College of Health Sciences, Faculty of Medicine, University of Nairobi, Kenya.
Nicoll, Angus and others. 1999. "Correspondence--Preventing HIV-1: Lessons from Mwanza and Rakai," The Lancet, Vol. 252, May 1.
Odhiambo, J.A. and others. 1999. "Tuberculosis and the HIV Epidemic: Increasing Annual Risk of Tuberculous Infection in Kenya, 1986-1996." American Journal of Public Health. July. Vol. 89. N[degrees] 7. pp. 1078-82.
O'Hara, H.B. and others. 2001. "Quality of Health Education During STD Case Management in Nairobi, Kenya." Original Article. International Journal of STD & AIDS. 12:315-323.
Painter, Thomas. 2001. "Voluntary Counseling and Testing for Couples: A High-Leverage Intervention for HIV/AIDS Prevention in Sub-Saharan Africa," Social Science and Medicine, Vol. 53, pp. 1397-1411.
Population Services International (PSI). 2001. "Kenya Knowledge, Attitudes, and Practices (KAP) Survey 2000." Nairobi, Kenya.
Rakwar, Joel and others. 1999. "Cofactors for the Acquisition of HIV-1 among Heterosexual Men: Prospective Cohort Study of Trucking Company Workers in Kenya." AIDS 13:607-614.
Rottingen, John-Arne, D. William Cameron, and Geoffrey Garnett. 2001. "A Systematic Review of the Epidemiologic Interactions between Classic Sexually Transmitted Diseases and HIV: How Much Is Really Known?" Sexually Transmitted Diseases, Vol. 28, No. 10.
Steadman Research Associates (SRA). 1999. "Behavioral Surveillance Survey in Mombasa." Steadman Research Associates and Family Health International. Draft report. Nairobi, Kenya.
Stover, John. 'The Effects of Behavior Change on Trends in HIV Incidence in Uganda and Kenya," The Futures Group International, March 2002 (PowerPoint presentation). Washington, DC.
Stover, John and Alan Johnston. 1999. "The Art of Policy Formulation: Experiences from Africa in Developing National fflV/ATDS Policies." The POLICY Project. August. Washington, D.C.: The Futures Group International.
Waithaka, Margaret and Ruth Bessinger. 2001. Sexual Behavior and Condom Use in the Context of HIV Prevention in Kenya. Population Services International, Nairobi, Kenya, and Measure Evaluation, Chapel Hill, North Carolina, USA.
Wawer, Maria, Nelson Sewankambo, David Serwadda, and others. 1999. "Control of Sexually Transmitted Diseases for ADDS Prevention in Uganda: A Randomized Community Trial," The Lancet, Vol. 353, February 13.
Weiss, H. A. and others. 2001. "The Epidemiology of HSV-2 Infection and its Association with HIV Infection in Four Urban African Populations. AIDS 15 (suppl 4): S97-S108.
World Bank. 1999. Confronting AIDS: Public Priorities in a Global Epidemic. Oxford University Press for the World Bank, Washington, DC.
World Health Organization (WHO). 2001. Global Tuberculosis Control. Geneva.
Annex A. Basic Data
KENYA--sexually transmitted infections project (credit-2686)
Key Project Data Appraisal Actual or Estimate current estimate Total project costs (US$) 65.51 55.53 Loan amount (US$) 40 36.7 * Cancellation (US$) 1.2 Date physical components completed: December 31, 2000 Actual as % of appraisal estimate Total project costs (US$) 85 Loan amount (US$) Cancellation (US$) Date physical components completed: December 31, 2000 * The difference between the appraisal estimate of the IDA credit amount and the actual disbursements can be accounted for by foreign exchange variation. Project Dates Steps in project cycle Original Actual Approval 03/14/1995 Signing/Agreement 05/22/1995 Effectiveness 08/07/1995 08/07/1995 Closing 06730/2001 06/30/2001 Staff Inputs (staff weeks) Stage of project cycle Actual/Latest Estimate Weeks US$ Identification/Preparation 36.0 108.0 Appraisal/Negotiations 15.5 74.3 Supervision 210.48 620.4 ICR 28.02 72.0 Total 290.0 874.7 Mission Data Stage of Date Specializations represented project cycle (month/year) Identification/ 1993 Unavailable Preparation 1994 Unavailable Appraisal/ 06/ 20-28/94 1 medical doctor/public health Negotiation spec., 1 operations officer, 1 financial management specialist, 1 division chief Nov 9-21, 1994 1 division chief, 2 operations officer, 1 medical doctor/public health spec, 1 operations officer, 1 population spec, 1 HIV/AIDS spec 1 resident representative, 2 consultants. Supervisions 09/25-10/6/95 3 operations officers, 1 WHO consultant 02/12-23/96 3 operations officers, 2 technical consultants (WHO/DfiD), 1 financial management specialist 09/23-10//1096 4 operations officers, 1 financial management specialist, 1 social development specialist, 1 statistician, 2 procurement specialists, 1 architectural consultant, 1 WHO consultant 05/29-06/12/97 3 operations officers, 1 medical doctor/public health specialist, 1 economist, 1 financial analyst, 1 NGO specialist, 2 consultants 02/2-19/98 1 medical doctor/public health specialist, 1 operations analyst, 1 operations officer, 1 financial management specialist, 1 economist, 1 NGO specialists, 1 gender specialist 07/27-08/7/98 1 senior population health (MTR) specialist, 2 operations officers, 1 medical doctor/public health specialist, 1 operations analyst, 1 operations officer, 1 economist, 1 financial management specialist, 1 social development specialist, 1 NGO specialist 02/2-12/99 1 medical doctor/senior public health specialist, 1 medical doctor/public health specialist, 1 economist, 1 operations officer, 1 financial management specialist 03/03-xx/OO 1 medical doctor/senior public health specialist, 1 operations analyst, 1 operations officer, 1 financial management specialist, 1 procurement auditor 10/07-10/13/00 1 operations analyst, 1 operations officer, 1 procurement specialist, 1 financial management specialist, 1 public health specialist ICR 10/16-27/00 1 public health specialist, 1 operations officer Mission Data Performance Ratings Stage of Date Implementation Development project cycle (month/year) Status Objectives Identification/ 1993 Preparation 1994 Appraisal/ 06/ 20-28/94 Negotiation Nov 9-21, 1994 Supervisions 09/25-10/6/95 S S 02/12-23/96 S S 09/23-10//1096 05/29-06/12/97 02/2-19/98 07/27-08/7/98 (MTR) 02/2-12/99 S S 03/03-xx/00 S S 10/07-10/13/00 ICR 10/16-27/00
Annex B: Evidence of program impact
1. This section reviews in greater detail the available evidence regarding the national program's impact on sexual behavior, STI prevalence, and HIV incidence. While the STI Project supported this larger program--together with over $20 million in grant financing from bilateral donors--it is not possible to separate out the project's specific contributions to changes in national indicators. In addition, program outcome indicators--including HIV and STI incidence, knowledge of HIV, and behavior change--are influenced by a wide range of factors, some independent of the project or national program. Finally, the lack of a strong monitoring and evaluation framework makes it difficult to assess the impact and even the output of some project activities. This section will therefore first assess national trend data regarding key outcomes, then assess the likelihood that program and project activities could have influenced these indicators, either directly or indirectly.
Modest Reductions in Risky Behavior
2. Surveys suggest improved knowledge of HIV/AIDS and a modest reduction in high-risk behaviors since the mid-1990s (see Table 1). Awareness of ADDS is now nearly universal; knowledge about how to prevent HIV transmission also has improved, but is not yet universal. In the 1998 DHS, for example, most respondents knew about condoms, but only half knew they prevented HIV or STI transmission (compared to a third in 1993). A subsequent survey found that over 90 percent of respondents could name two ways to protect themselves from HIV (Nganda 2001). Significant misperceptions remain, however, regarding HIV transmission and perceptions of individual risk--particularly among youth. For example, in Mombasa, fear of pregnancy far outweighs fear of STIs among women under age 19, and girls' initial sexual encounters are usually unprotected (Nganda and others 2001; SRA 1999).
3. Increased awareness was slow to translate into behavior change, but surveys suggest a reduction of risky behavior since the mid-1990s. Condom use remains low among married couples (6 percent), but the percentage of sexually active men who have ever used a condom increased steadily from one-third in 1993 to half in 2000: Moreover, condom use has increased among unmarried couples, casual partners, and commercial sex workers. The percentage of men using condoms with non-regular partners increased from 43 percent to about two-thirds in 2000 (Table 1). This is an improvement, but is still too low a level of condom use for high-risk sex. In Mombasa three-quarters commercial sex workers and their customers reported use of condoms "every time or most times," but with higher rates among one-time clients compared to regular clients or boyfriends (SRA 1999).19 The number of men reporting more than one sexual partner declined from 27 percent in 1997 to 19 percent in 2001 (Table 2). Although the percent of women reporting more than one sexual partner remained low and unchanged, surveys show a slight decline among single women and sexually active girls. But recent surveys in Mombasa and Kisumu suggest that over half of girls become sexually active before age 18, and condom use remains rare among sexually active teenagers (SRA 1999; Glynn and others 2001). There are no reliable trend data regarding the percentage of men and women engaged in commercial sex: the 1998 DHS reported that 13 percent of men and 7 percent of women reported exchanging money for sex--including 15 percent of women aged 15-19 (KDHS 1998).
4. Sources of information on AIDS. The 1997 KDHS found that men and women are most likely to hear about AIDS from the radio and friends and relatives (Table 3). A quarter or fewer receive information from posters or pamphlets or health workers, although the latter are an important source for women. Thus, it is likely that to the extent that the project was able to raise awareness or convey messages, the multimedia messages (implemented after the DHS) would have reached the most people. The results of the next DHS will reveal the extent to which the sources of information have changed, for which some of the project's components might have contributed. It is possible that community meetings have become a more significant source following the "National Disaster" declaration in 1999.
5. Influences on behavior change: Attributing changes in behavior to program interventions is difficult, even with a rigorous evaluation design. While the increased death rate from AIDS was likely an important factor in translating awareness of HIV into actual behavior change, national and district-level IEC activities probably contributed as well. A statistical analysis of a recent knowledge, attitudes, and practices survey found that after controlling for other variables: (i) individuals exposed to "generic" AIDS prevention messages in the mass media were more likely to use condoms; (ii) the likelihood of condom use increased with repeated exposure to prevention messages (Agha 2001). Yet, an analysis of 1998 KDHS results found that there was no association between receiving information from a health worker and condom use for men or women (Waithika and Bessinger 2001). These results, together with the findings in Section 1 above suggest that (i) DEC activities could have contributed to awareness and behavior change, but the impact was probably reduced by inadequacies in targeting, pre-testing, and dissemination; (ii) despite training, health providers are not yet providing adequate HIV prevention education and counseling. The increased discussion of HIV/ADDS by national and local political leaders in the late-1990s probably also contributed to increased awareness, but there is no way to assess the impact on behavior.
Impact on STI Prevalence?
6. The lack of national population-level or facility data on trends in STI prevalence make it difficult to assess the impact of STI treatment programs. Available evidence, together with anecdotal reports from clinicians, suggests that prevalence of bacterial STIs may have declined over the course of the project. In Nairobi, prevalence of syphilis among women attending antenatal clinics declined from 7 percent in 1995 to 5 percent in 2000. Improved STI treatment and/or behavior change conceivably could have played a role. From 1996 to 2000, the share of reported cases of vaginal and urethral discharge declined compared to other STI syndromes (NASCOP 2002). These are susceptible to syndromic management (see Box 3), and the decline corresponds to strengthened training and syndromic management of STIs sponsored by the project. Moreover, the percentage of these infections increased in 2001 when drug availability fell following the end of the STI project. The national data do not suggest any clear trends with respect to the share of STIs that are genital ulcerative disease cases, however.
7. Recent research studies suggest low and/or declining levels of most bacterial STIs, but continued high prevalence of vaginal trichomoniasis and very high levels of genital herpes. In Nairobi, the intensive interventions over the past decade to strengthen STI services (and target commercial sex workers) contributed to declines in gonorrhea, syphilis, and chlamydial infections from 1992-1999 among female antenatal clinic attendees (Moses and others 2002). (20) The STI project sought to scale up syndromic treatment based on this model--although the quality of treatment and supervision was probably lower, compared to the intensive support provided to pilot clinics in Nairobi, and national coverage was not achieved until 1999.
8. A one-time population-based survey of STI prevalence among residents of Kisumu in 1998 found that relatively low prevalence for most bacterial STIs. (21) But prevalence of genital herpes (HSV-2) was strikingly high: 68 percent of all women tested positive for HSV-2, as well as 39 percent of women aged 15-19 (Weiss and others 2001). (22) Herpes is transmitted by the same types of sexual behavior as HIV, but does not respond to antibiotics. While HSV-2 prevalence could be significantly higher in Nyanza than elsewhere in Kenya, these data plus research findings elsewhere (Weiss and others 2001) suggest that--perhaps partly as a consequence of improved treatment of bacterial STIs--the relative or even absolute prevalence of genital herpes has increased during the life of the STI Project. If ulcerative STIs are primarily responsible for enhancing the transmissibility of HIV, then the high prevalence of genital herpes suggests that it may play a significant role in new HIV infections. In addition, the prevalence of trichomoniasis was high among women in Kisumu (29 percent of all women, and a third of women aged 15-19). Trichomoniasis was found to be significantly associated with both HIV and HSV-2 infection in women--but the direction of causation is not yet well understood (Buve and others 2001). It can be cured by antibiotics, but is frequently asymptomatic in women and therefore left untreated.
Impact on HIV incidence?
9. In the course of the STI project, adult (aged 15-49) HIV prevalence stabilized at about 15 percent, and estimates based on surveillance data from 2001 suggest a possible decline in prevalence to 13 percent. But HIV prevalence is cumulative, and can be affected both by death rates the number of new infections, so it is not a good indicator of program impact. The key question is whether the program, and STI project, might have had an impact on HIV incidence--the number of new infections. Unfortunately, HIV incidence is difficult to measure, and there are no national trend data for HIV incidence.
10. HIV prevalence among youth (aged 15-19) represent a reasonable proxy for incidence, since AIDS mortality in that age group is low. Data from Nairobi suggest that prevalence among female antenatal clinic attendees under age 20 declined from a peak of 21 percent in 1994 to 14 percent in 1999. In Kisumu, in contrast, 27 percent of women aged 15-19 were HIV positive (compared to 4.6 percent of men) in 1998. Although infection rates may have fallen since then, this suggests very high HIV incidence in Kisumu.
11. The level and determinants of HIV incidence can also be estimated using a statistical model. One such model estimates that HIV incidence in Kenya has stabilized at around 2 percent annually, based on 1998 DHS data and HIV prevalence data up to 2000 (Stover 2001). The model further estimates that incidence would have been about one percent higher (slightly above 3 percent) with no behavior change or STI treatment. The changes contributing to this reduction were estimated to be (in order of importance) reductions in casual sex, increased condom use, reductions in the number of sexually active individuals, and STI treatment (Stover 2001). These findings are consistent with other research and statistical modeling of HIV epidemics, which suggest that STL treatment may help reduce HIV transmission, but has less of an impact than behavior change--including increased abstinence, fewer partners, and increased condom use (Bernstein and others 1998; World Bank 1999).
12. Why is HIV prevalence significantly higher in Kisumu and Nyanza Province compared with other regions of Kenya? Research in Kisumu and several other African cities suggests three factors that are responsible for high prevalence rates overall and among girls in particular. First, the early age of sexual debut among girls (16 years on average), both inside and outside of marriage. Girls are highly vulnerable biologically to HIV infection, which partly explains higher prevalence rates among girls, and tend to marry older men and/or have older, more experienced sexual partners). Second, the high rates of HSV-2, which was strongly correlated with HIV infection. Third, the lack of circumcism among most men in Kisumu (circumcism is more widely practiced elsewhere in Kenya) (Glynn and others 2001). Women with genital herpes were five times more likely to have contracted HIV, even after controlling for behavior and other factors. Positive syphilis serology and trichomoniasis vaginalis were also found to be significantly correlated with HIV infection, but the prevalence rates were lower, so their contribution to HIV transmission was probably less (Weiss and others 2001).
13. In summary, it is plausible that the Kenya STI treatment program contributed to a modest reduction in HIV transmission. But the impact might have been reduced by variations in the quality and coverage of treatment among districts; mixed success in partner tracing; uneven coverage of the highest risk groups; and the high prevalence of genital herpes (see Box 4).
Box 4. Links Between STIs and HIV Transmission Research completed over the past decade has established a clear biological link between HIV transmission and the presence of STIs, but the extent to which STI treatment reduces HIV transmission among target populations is less certain, and appears to depend both on the program and the context. At least three types of studies have been used to examine the links between STIs and HIV transmission. These include studies on the biological links between STIs and HIV; cohort studies of the risks of HIV transmission to HIV negative individuals; and two randomized control trials in Africa, which compared the impact of STI treatment on HIV and STI prevalence in intervention and control communities. Biomedical and cohort studies have found a consistent association between the presence of STIs and HIV transmission, with the presence of STIs increasing risk of HIV transmission by a factor of three to five on average, even when controlling for behavioral factors (Flemming and Wasserheit 1999). (At the time of STI Project design, preliminary research had suggested that STIs increased the risk of HIV transmission by ten times or more.) But the risk may vary depending on the STI--for example, genital ulcers (e.g., chancroid and genital herpes) may increase risk of HIV transmission more than other STIs. Moreover, the impact of STI treatment programs on HIV incidence in a given population is influenced by the effectiveness and coverage of treatment, the extent of targeting to high-risk individuals, the types of STIs and possible the stage of the HIV epidemic (Flemming and Wasserheit 1999). The randomized controlled trials in Mwanza, Tanzania, and Rakai, Uganda produced seemingly contradictory findings. In the Mwanza study, researchers implemented syndromic treatment of STIs in rural communities, and found that it contributed to a 40 percent reduction in HIV prevalence (Grosskurth and others 1995). Yet in Rakai, mass treatment of the population with antibiotics reduced STIs but not HIV incidence (Wawer and others 1999). Researchers are currently debating the interpretation of these findings. The Mwanza study implemented syndromic treatment of STIs--similar to that supported by the STI project, but the Rakai population had a higher prevalence of HSV2 than Mwanza, similar to parts of Kenya (feottingen and other 2001; Fleming and Wasserheit 1999).
Annex C. Drug Procurement and Distribution
An important challenge faced by the STI Project was to efficiently procure and distribute STI drugs to health facilities around the country. Efforts to procure and distribute contraceptives through the government's existing system in previous Bank-supported population projects had encountered a myriad of problems with delays, misprocurements, and inadequate logistics and stock management. The Medical Supplies Coordinating Unit in the MOH was formally responsible for procurement and distribution, but capacity was weak. Plans were underway to convert it to a semi-autonomous drug agency, but progress was slow (the Kenya Essential Drugs Supply and Management Agency (KEMSA) has only recently been established). The project thus sought to bypass these existing arrangements to ensure drug availability at the clinic level, but it did not seek to strengthen the existing system.
Procurement agent. Despite delays in hiring the procurement agent, the process worked smoothly thereafter, and resulted in substantial cost savings. The Bank has argued that procurement is not an essential function of government, and should be contracted out. The MOH has adopted the use of a procurement agent for its own procurements.
Essential drug "kit" system. The kit system had the advantage of ensuring that a complete set of STI drugs specified by the treatment protocols, but resulted in the oversupply of some drugs, and shortages of others. The districts were eventually given permission to redistribute drugs from the kits. The STI drugs were stored separately from the rest of the drugs at the facility to ensure continued supply, but clinicians were given permission to use the drugs for other ailments when expiration dates approached. Still, the distribution system was a "push" rather than "pull" system, which did not allow for adjustments for local use and availability of specific drugs.
Distribution logistics. For the initial "emergency" procurement of drug kits, DflD contracted with the Family Planning Logistics Management (FPLM) program--which already had an extensive distribution network in place for delivery of contraceptives--to deliver STI drug kits directly to each facility. Rather than contract with FPLM, however, the Project hired a local commercial storage facility, and contracted with a commercial shipping company to deliver the drug kits to each districts. Each kit was designated for a specific hospital or clinic, but the districts were responsible for delivery to the clinic level. Many districts did not have adequate storage prior to distribution, resulting in a few widely reported cases of theft, and lack of transport in some districts led to delays in getting the drugs to clinics. In response to these difficulties, DAD financed an additional emergency contract with FPLM to assist with delivery to clinics in districts where the HPAC project was active. The drug kits designated for the mission sector arrived in a single shipment to the essential drugs distributor for mission hospitals (MEDS), which was responsible for distributing the drug kits to all mission hospitals. Since the drugs were required to be distributed free of charge, the mission sector did not have a mechanism to cover its costs.
Delivery of drugs close to expiration dates. During interviews and site visits, a number of health providers and official expressed concern to OED that some of the STI drugs financed by the project arrived close to their expiration dates. In response to these concerns, the Bank's Kenya country team commissioned a procurement specialist (in July 2002) to undertake an independent review of the procurement of drug kits under this project. The resulting report concluded that the borrower implementing agency and procurement agent had executed their responsibilities diligently, professionally and in full accordance with Bank procurement guidelines. Problems arose, however, because the terms stated in the original contract were not fully consistent with those stated in the Letter of Credit used to purchase the drugs. According to the original contract, drugs were to have been delivered as fully assembled kits in a series shipments over a period of two years--to simplify storage and distribution and to reduce the risk of expiration. But these terms were not clearly spelled out in the Letter of Credit. The international pharmaceutical company that won the contract thus chose to follow the terms of the Letter of Credit rather than the original contract--shipping all the drugs within a nine-month period and assembling the kits locally (rather than shipping kits pre-packaged). The shelf life stipulations were also not adequately spelled out in the Letter of Credit. The supplier complied with the required shelf life at the time of shipping, rather than the time of delivery to the warehouse. Because of delays in customs clearance at the port of entry in Kenya (due to late transmission of documents by the supplier and delays in procuring exemptions by the government), the first consignments were not cleared from port until January 1999, about 5 months after arrival.
When the risk of expiration became apparent, the MOH and the procurement agent took steps to avoid loss of drugs. First, the MOH issued a circular allowing drugs to be used to treat other infections. Second, they negotiated an agreement with the supplier to replace any expired drugs. According to MOH procedures, health facilities are required to return all expired drugs to the central warehouse, rather than destroy them on site. The MOH reports that such replacement was common practice for other items procured under the project (e.g., HIV test kits, which have short shelf lives). But according to MOH records, no expired STI drugs were returned for replacement. There is thus no direct evidence of the loss of drugs through expiration. (But based on interviews with providers, it appears that the impending expiration dates led to less than optimal use of some of the pharmaceuticals.)
Annex D. Comments from the Borrower
These comments were written in response to an earlier draft of this evaluation. The text was revised in response to the comments.
MINISTRY OF HEALTH
Telegrams: "MINHEALTH," Nairobi
Telephone: Nairobi 717077
When replying please quote
Ref. No. MED/46/A/1/Vol. I/2
OFFICE OF THE
P.O. Box 30016
5TH. July 2002 2a....._
Mr. Alain Barbu, Manager
Sector and Thematic Sector Group
Operations Evaluation Department
The World Bank
Washington DC USA
Dear Mr. Barbu
RE: PROJECT PERFORMANCE ASSESSMENT REPORT: KENYA SEXUALLY TRANSMITTED INFECTIONS PROJECT CREDIT 2686-KE
Refers to the above report which you requested the Ministry's comments. We are in agreement with your overall impression that this project made a substantial contribution to institutional development, that sustainability is likely and that the outcome and Bank, Borrower performance is satisfactory.
However, there are some specific observations which are not accurate and need to be revised to reflect the correct position:
1. The HIV/AIDS Working Group was active throughout the period of implementation of the project. It assisted the Ministry to better coordinate donors supporting HIV/AIDS programmes in the country. Through this Working Group, an estimated US$ 653.6 million was mobilized from Government, World Bank, DFED, USAED, CDC, UN agencies, JICA, SEDA and BMZ to finance the National HIV/AIDS Strategic Plan for the period 2000-6 by the time the project was closing. If this is considered within the context of the project having assisted the Government to lay the policy, strategy and legal framework for the long war against HIV/AIDS, then the outcome ratings could be higher than Satisfactory.
2. The Ministry is not aware of any World Bank financed STI drugs having expired and had to be destroyed as reflected in page 7 paragraph 19 and last paragraph of annex C page 29. The STI drug kits were procured through an international tender and were delivered in several shipments from the United Kingdom, United States of America, Germany, Italy, Switzerland, Spain, India and South Africa. All the drugs were manufactured new for the Project and inscribed with a Government of Kenya Logo. Pre-shipment inspections were done and payment made through Irrevocable Letter of Credit on shipment. I confirm that all STI drugs were distributed to health facilities and no drugs were destroyed after expiry. The Ministry was informed through a copy of a letter to GTZ Procurement Management Unit by the supplier that "the issue of remaining shelf life has been discussed and undertakes to indemnify MOH and replace, at no cost, any expiries arising thereof." This was caused by problems related to contract management, especially delays in the clearing of goods at the port of entry and local packaging of drugs into kits before delivery to the Ministry's Warehouse. Replacement of expired perishable goods e.g. HIV test kits by the suppliers at no cost to MOH was a common practice. Unlike the KfW and DFED financed STI drugs which were strictly for STD treatment, the EDA financed STI drug kits supplemented the essential drug supply and were also used for treatment of other diseases. Circulars were issued to support this strategy. The Ministry has detailed reports of where the drugs were distributed. We will investigate if there could be any expired STI drugs and if any is found then mopped up and returned for replacement at no cost to MOH.
3. The German Government through KFW procured STI drugs at an estimated cost of US$ 4 million in co-financing arrangements.
4. The project financed the development, dissemination of a national Information, Education and Communication strategy. The achievements in policy, advocacy, institutional development and STD care had strong and effective EEC components.
5. Although the proceeds of the project were used to target interventions at the military personnel and STD patients, other high-risk groups were equally targeted using resources from other partners e.g. AMREF, CIDA, WHO and UNICEF.
6. Co-financing of district annual workplans was done.
I look forward to receiving the final report and use the lessons learnt to improve the implementation ongoing and future EDA-financed projects.
PROFESOR S. MEME, EBS, FAAP
OFFICE OF THE PRESIDENT
NATIONAL AIDS CONTROL COUNCIL
Telephone: Nairobi 715109/715144
Fax No. 711231/711072
When replying please quote
Ref. No. NACC/P/4/9
The Chancery Building
6W Floor, Valley Road
P.O. Box 61307
Date: 27 June 2002
Manager, Sector and Thematic Evaluation Group
Operations Evaluation Department
1818 H Street, NW
Washington, DC 20433 USA
Fax: 202 522 3123
Dear Alain Barbu
RE: KENYA-SEXUALLY TRANSMITTED INFECTIONS PROJECT (CREDIT 2686): DRAFT PROJECT PERFORMANCE ASSESSMENT REPORT
I acknowledge receipt of your letter of June 11 requesting our comments on the above project. Our comments are as follows:
The Report points out that, the Project under review had the following weaknesses:
* No baselines were established or mechanisms put in place to collect the data necessary to track progress towards stated objectives
* Little attention was given to implementation arrangements or the capacity of implementing agencies to carry out proposed activities
* No specific mechanisms were established to target high-risk groups
* The initial two years were dominated by non-performance due to disbursement problems and delay in the appointment of judgment agent for all major procurements.
However, it should be noted that despite the issues raised above the project overlooks the positive side of the project.
Some of the positive effects of the project include:
* The nationwide coverage of STI treatment through the STI Drug Kit. This aspect alone has brought down the STI prevalence in the country. Given this it will be difficult for the Ministry of Health to allow the situation to go back to what it was before.
* The project created the necessary environment for the establishment of National AIDS Control Council, when the Sessional Paper No. 4 of 1997 was prepared, and used to launch series of regional awareness workshops.
* The Project facilitated a number of Development Partners to respond and support HIV/AIDS Programmes such as the DARE Projects.
* Although the Report has rated Financial Management as not satisfactory, this justifies the reasons for contracting one such service as financial management as it has been done the KHADRE Project.
Overall, we found the report to contain in-depth analysis, which will be useful to the National AIDS Control Council as it implements the Kenya HIV/AIDS Disaster Response Project. We, therefore, take this opportunity to thank the World Bank for its continued support of NACC: activities and look forward to receiving the final copy of the report.
Dr. Margaret Gachara
(1.) The total fertility rate dropped from 6.7 in the late 1980s to 4.7 children per women in the late 1990s, and the contraceptive prevalence rate (CPR) among married women increased from 7 to 39 percent (KDHS 1994; KDHS 1998).
(2.) Following the Bank's 1998 Country Assistance Strategy, the Bank subsequently sharply reduced lending for several years, except for social sector and emergency operations.
(3.) Prevalence is the percentage of the population (typically adults 15-49) infected at a given time. Because those infected with HIV remain infected until they die, but may remain alive for a decade or more, prevalence reflects the cumulative number of HIV infections, minus the total deaths. Although incidence--the number of newly infected cases per year--is a better measure of the spread of the HIV epidemic and the possible impact of prevention programs, it is difficult to measure.
(4.) These included Population III (1988-1996, US$12.9 million), Population IV (1990-98, $35 million), and the Health Sector Rehabilitation Project (1993-98). Population III and IV were both rated as unsatisfactory by OED, while the Health Sector Rehabilitation Project received a rating of moderately unsatisfactory (based on satisfactory contributions to improving services at Kenyatta Central Hospital, but little progress in improving primary health care in Nairobi).
(5.) The disbursement lag resulted in part from the use of the World Bank-financed Population III and IV Projects to finance STI-related activities, including 66 million condoms and $5 million in STI drugs.
(6.) In 1999, for example, while 10 districts received 75 percent or more of their approved funding, nine districts received less than a quarter, and two had received none. While the STI project provided most funds for district HIV/STI programs, it was not the only source. Other sources of funding included proceeds from user fees collected at the district level, as well as support from NGOs and other partners. There are no comprehensive data on the level or trends of overall district funding for HIV/AIDS/STIs, however.
(7.) In response to an intensive dialogue with the Bank and partners, government agreed to establish a multi-agency task force (which included staff from the Bank's country office as well as Treasury officials) to study the problem and make recommendations. This dialogue and the task force's report led to the issuance of a Treasury circular in 1999 that for the first time permitted the establishment of separate project accounts at the district level. This gave greater resources and flexibility to district health teams--progress which the current Decentralized AIDS and Reproductive Health (DARE) project seeks to build upon--but it came late in the STI Project's life.
(8.) For example, the Amoxycillin vials and clavunic acid tablets were due to expire in August of 2000, nine months after arrival in December 1999. The original contract specified that all drugs were to be delivered with 5/6 of their shelf life
(10.) Staff in Nyeri and Embu (both pilot districts) reported receiving substantial shipments of printed materials, which reportedly were widely distributed. Some posters and pamphlets remain, but supplies are now mostly exhausted. In Kirinyaga (a non-pilot district), however, few project-financed posters or other materials were visible at the hospital--other than the charts for syndromic management of STIs--nor in the community. OED saw few HIV/AIDS billboards during district visits (in urban or rural areas); those installed by the project were reported to have deteriorated.
(11.) But staff Nyeri district reported that one local billboard financed by the project used a local obscenity in its message. This billboard was defaced by the local community, and later replaced with a message translated in consultation with local staff.
(12.) The 30 second to 1 minute radio and TV spots had a clear message emphasizing the "ABC's" of HIV/STI prevention (Abstain, Be faithful, use a Condom). But this "ABC" message appeared inconsistently in the printed materials. One of the most widely distributed and watched videos produced under the multi-media contract, "The Silent Epidemic," showed graphic pictures of sexually transmitted infections. Focus group results report a strong "shock value" from this video (which was shown at the 1999 Parliamentary session in Mombasa), but the main message was that "free drugs are now available" in government clinics, and those with STIs should seek treatment. Behavior change was not emphasized. The one-hour docu-drama produced by ACE Communication, which was broadcast on Kenyan television and through mobile videos, effectively highlighted the personal toll of AIDS, and dramatized issues related to sexual behavior, including unfaithful spouses. Of the overall multimedia contract, about US$500,000 was spent on research and message design, US$650,000 on production, and US$700,000 for dissemination.
(13.) Partly to reduce these tensions, the PMU director also assumed the directorship of NASCOP in the final year of project implementation.
(14.) The ICR suggests that districts were able to implement the majority of their workprograms despite the funding shortfalls. OED reviewed STI/HIV workprograms in the three districts visited, but found limited documentation regarding activities actually completed, as opposed to those planned.
(15.) Project accounts included no separate line item for NGO activities, so there are no official records regarding actual funding to NGOs. In a 1998 report, the Kenya AIDS NGOs consortium identified 65 NGOs and CBOs in 10 districts that had received a total of Ksh 7.1 million (about US$120,000), most of it in grants of less than $100.
(16.) This seems to partly stem from the perception that donors are already providing substantial support to NGOs--in a context where the MOH faced a variety of financial constraints. But while bilateral donors provide substantial support for international NGOs and local NGOs in a few provinces, support for NGOs/CBOs in other provinces/districts was limited.
(17.) Sentinel Surveillance has covered 13 urban sites since 1990, and 9 rural/peri-urban sites were added in 1994-95. Rural rates were extrapolated based on urban prevalence data. Twelve new sites were recently added to improve rural- urban and regional balance. The quality and coverage of the surveillance system is currently being further enhanced with support from the U.S. Center For Disease Control.
(18.) Some of the variation in urban antenatal surveillance sites may be due to changes in migration patterns, however (Jackson and others 1999).
(19.) The progress in condom use among CSWs is more likely attributable to the targeted interventions of other partners than to the STI Project, however. An intensive donor-financed intervention with commercial sex workers in Nairobi resulted in an increase in condom use to over 90 percent (Moses and others 2001). Counseling and behavior change interventions also resulted in increased condom use and fewer sexual partners among a group of Kenya truck drivers (Rakwar and others 1999). While these successful interventions with high-risk groups are internationally recognized, they have not yet been scaled up systematically at the national level.
(20.) Between 1992 and 1999, among antenatal clinic attendees screened in Nairobi, gonorrhea declined from 5.3 percent to 1.7 percent; chlamydial infection declined by half (from 33 percent to 17 percent), as did syphilis serology. (5.4 percent to 2.5 percent). Moreover, the cases of genital ulcers, urethral and vaginal discharge declined by more than half--both in absolute numbers and as a percentage of total clinic visits (Moses and others 2001). Although this project was donor financed, the STI Project provided STI drugs in the latter years.
(21.) Prevalence for gonorrhea was the lowest, at less than 1 percent (average among men and women); followed by positive syphilis seriology (4 percent); and chlamydial infection 4.5 percent. Syphilis seriology was lower among young adults, but chlamydial infection was significantly higher (9 percent among women 15-19) (Buve" and others 2001).
(22.) These data are based on a random sample of women living in Kisumu, as opposed to women attending antenatal clinics.
Table 1. Condom Use is Increasing--But Still Not Universal with Non-Regular Partners 1993 DHS 1998 DHS 2000 PSI "Ever used" a condom? Male 34 40 49 Female 7 9 25 Used condom at last sex with... Non-regular partner Male 43 63 Female 15 ** Spouse/regular partner *** Male 16 14 Female 5 9 Sources: KDHS 1994 and 1998; 2000 Knowledge, Attitudes, and Practices Survey (PSI 2001) Note: All data are reported as a percentage of sexually active respondents ** Insufficient responses. *** The KDHS survey reports separately condom use with spouse and regular partners, while these categories are combined in the PSI survey. The figures therefore may not be fully comparable (e.g., it should not be inferred that condom use has declined among males with their regular partners). Table 2. Fewer Men-and Single Women-Report Having More Than One Sexual Partner 1998 (DHS) 2000 (PSI) Male (more than one sexual partner) 27 19 Married 16 13 Never married/Single 43 26 Youth (15-19) 43 18 Female 3.6 3.8 Married 1.6 2.4 Never married/Single 10 4 Youth (15-19) 8 6 Note: All data are reported as a percentage of sexually active respondents. Table 3. Percent of Women and Men Who Have Heard of Aids Through Different Channels Women Men % N % N Heard about AIDS through: Radio 73 5716 87 2973 Television 20 1562 32 1097 Newspaper 23 1780 42 1415 Posters/pamphlets 18 1385 26 869 Health Workers 26 2029 16 557 Church/mosque 12 950 7 251 School 17 1355 14 467 Community meetings 13 983 12 415 Friends/relative 57 4422 34 1149 Workplace 3 218 4 145 Drama performance 4 318 3 100 Other 3 204 4 121 Ever heard of AIDS 99 7880 99 3386 Source: PSI 2001 based on KDHS 1998.