2. Key factors affecting implementation and outcomes.
Project Preparation and Design: Clearly, the project was not adequately prepared and appraised before it went to the Board. Project implementation plan (PIP) and implementation arrangements were not discussed and firmed up during project preparation. Key activities were deferred to the first year of implementation, without any plan in place for completing them; for example, the project's Strategic Implementation Plan (SIP) was to be completed within the first year of implementation, but had not been done two years into the project period. Since this was the first HIV/AIDS project in Bangladesh, it was crucial that the project was clear and well-designed, with management and coordination arrangements (particularly between the project and NGOs) laid out in detail.
Quality at Entry: Quality at Entry is rated Unsatisfactory. Project design was based on three unsupported assumptions: (i) That given the experience in the Bangladesh Integrated Nutrition Project and National Nutrition Program, contracting of NGOs could be undertaken by NASP (PAD, Annex 2, pg. 39). NGO contracting under BINP was relatively small, and when it was scaled up under the NNP, proved to be extremely problematic. Despite identifying delays in operationalizing GOB/NGO collaboration as a risk in the Project Appraisal Document (PAD), adequate safeguards were not put in place; (ii) That NASP had the capacity to complete the activities assigned to Phase I and undertake the complex set of activities required to complete preparation of subsequent phases of the project. Inadequate leadership, institutional and management capacity were identified in the PAD as being a high risk; the mitigation measure for this was that MOHFW would post regular staff to NASP and guarantee their continuity for the duration of the project. However, within the government system, appointments and transfers of staff are made at the highest level and extremely politicized; the PAD recognized this, yet the project did not put in place any systemic solution to the problem; (iii) That there was a need to rush the project through because of the "rapidly closing window of opportunity to prevent an HIV/AIDS epidemic" (PAD, pg. 5). In fact prevalence among the high risk groups was low; and some TIs among these groups were already in place, funded by DfID and USAID. While there was definitely a need for a well designed national HIV/AIDS program to scale up effective interventions, putting in place instead a poorly prepared project due to an unnecessary sense of urgency actually was self defeating.
Lessons Learned and Assessment of Risk: The design of the project did take into account best practice interventions established through global experience: (i) scaling up TIs among high risk groups through partnerships with NGOs; (ii) expansion of advocacy and awareness efforts among the general population; (iii) measures to reduce stigma and discrimination against PLWHAs; and (iv) capacity building within government and NGO partners. Although a risk assessment was undertaken as is required in the PAD, the mitigation measures put in place were not well articulated (see above).
Participatory Processes: The project was developed in close consultation with different groups of beneficiaries, including FSWs and IDUs at the national and local levels. Key donors, including UNAIDS, UNDP, the HPSP Donor Consortium, NGOs, and researchers also provided feedback on the preparation and design of the project during individual and group meetings and stakeholder workshops.
2.2. Implementation Stage (including any project changes/restructuring, mid-term review, Project at Risk status, and actions taken, as applicable):
Several factors affected project implementation progress:
Delays in Project Start-Up: The project faced serious start-up problems in the first two years of implementation. While poor preparation was identified as an important reason for this (see above), the Problem Solving Mission prior to MTR also identified two other issues: (i) Unclear implementation arrangements: There was confusion within the MOHFW on whether HAPP was part of HPSP or not. Since HAPP was being financed by a separate IDA Credit, even though an Annual Operation Plan (AOP) was approved under HPSP, each action was being scrutinized again for clearance; (ii) Capacity constraints: NASP lacked management, technical and procurement capacity. Regular staff was to be appointed on a long-term basis, gradually developing greater experience and knowledge of HIV-related issues, as well as consultants in key technical areas. However, regular staff was never appointed to NASP; it continued to be managed by MOHFW officials on deputation for short periods of time; there was frequent turnover of senior staff, with the Line Director holding multiple charges; and GOB was reluctant to hire consultants. Only one of the nine consultants agreed in the AOP was hired; this was the Procurement Consultant, who subsequently resigned due to delays in fee payments.
Mid-Term Review and Reprogramming: The project was twice designated as a "Problem Project" prior to the MTR. In preparation for the MTR scheduled for mid-2003, several additional reviews were undertaken: (i) a "Problem Solving Mission" was fielded in January 2003 to not only undertake a retrospective review of project implementation, but also to get the project back on track (A-M of Problem Solving Mission, February 2003); (ii) a Quality Enhancement Review (QER) was held in March 2003; and (iii) a Preparatory Technical Mission for the MTR was fielded in March 2003.
The main issues identified by these reviews and missions were: (i) inadequate preparation of the project prior to approval, and non-completion of preparatory activities even two years into the project period; (ii) unclear implementation arrangements, due to the merging of this project into the HPSP mid-stream into that program; and (iii) lack of capacity and full-time personnel within NASP to manage the project. The QER Panel concluded that restructuring the project appeared to be the appropriate option (Panel Report, Bangladesh HAPP QER; March 2003). The Panel recommended that the restructuring should: (i) simplify and focus the components; (ii) simplify implementation arrangements; (iii) ensure adequate project leadership; and (iv) develop a post-project integration proposal.
The MTR was, therefore, used as an opportunity to undertake a major re-structuring of the project. While it was decided that the Project Development Objectives (PDOs) remained valid, design aspects were re-structured as follows (A-M, MTR; June 2003):
* The overall size of the project was reduced by half, and a partial cancellation of both the IDA Credit and DfID grant was agreed.
* A new logical framework was proposed with a new set of indicators to measure progress towards achievement of PDOs.
* The scope of TIs was changed, with only 5 of the planned 15 packages of TIs being retained.
* A new HIV/AIDS Interventions Fund (HAIF) was established to procure the services of NGOs through a simpler mechanism, also to be managed by UNICEF.
* It was agreed that UN agencies would be contracted to support the implementation of each of the components: UNICEF for procurement of NGO services, Communications and Advocacy, WHO for Blood Safety, and UNFPA for Project Support and Institutional Strengthening.
* Pharmaceuticals were to be procured through UNICEF.
* To address the issue of non-disbursal of DfID funds, funds were now to be disbursed in an 80-20 distribution between the Bank and DfID, managed by the World Bank, based on a common withdrawal application.
* The DCA was amended to change the disbursement percentage for incremental salaries and operating costs for the remaining two years of the project to 80%.
Short-term extensions to the Closing Date: Due to delays in formalizing implementation arrangements with UNICEF and subsequently contracting NGOs, it was August 2004 by the time NGO activities with high risk groups got underway (A-M, October 2004). In June 2005, a one year extension to the Closing Date was requested and granted; however, there was a three month delay by the time the necessary clearances to maintain UNICEF as the implementing agency were obtained. UNICEF had no facility for retroactive financing, and during this interim period, NGOs could not be reimbursed for any activities, and hence TIs were suspended. A second extension to the contract was granted in June 2006, but due to various procedural delays, the service delivery agreements with the NGOs were signed finally only on December 31, 2006. This time UNICEF did pay for some part of the operating costs of the NGOs, but did not cover program costs, resulting in another interruption in services.
Another consequence of the piece-meal extensions was the inability of both NGOs and implementing agencies to retain good staff. Without the assurance of longer term employment, staff was quick to take up opportunities which were more likely to offer secure employment (Personal communication, UNICEF Program Officer).
2.3. Monitoring and Evaluation (M&E) Design, Implementation and Utilization:
Results Evaluation Framework
The outcome indicators to measure progress toward the development objective were changed several times over the course of the project (see details in Annex 9). While baseline values were available for some of the indicators, no targets were set and many of them were very difficult to collect data for in a consistent manner over time. At the MTR, a new logical framework was agreed, which retained the same objective, with two PDO indicators: 1) % of HIV infected people among CSWs, MSM and IDUs; 2) Prevalence of syphilis among CSWs and IDUs. Of these, the first indicator has been tracked till the end of the project, albeit with a change in targets while the second one was dropped in May 2005. No explanation was offered in the ISR or relevant Aide-Memoire for either of these changes.
A large number of intermediate and output indicators were also included in the Logical framework of the PAD and were subsequently dropped in the MTR. At the same time several new indicators were included during the MTR and dropped in May 2005. The initial Logical Framework being poorly conceptualized follows from overall poor project preparation; however, poor tracking of indicators has been a problem throughout the project period.
Surveillance: The national HIV surveillance was established in 1998 by GOB. Under the project, five rounds of sentinel serological surveillance were undertaken by the ICDDR, B. In addition three rounds of behavioural surveillance surveys (BSS) were completed with financing from USAID. Surveillance has been a strong component of the HAPP (A-M, November 2007) and has provided critical information to track the evolution of the epidemic and for overall programming purposes. By identifying groups at most risk it has been instrumental in sharper targeting of these sub-populations. There is some concern that the serological surveillance may underestimate prevalence since the sample was limited to intervention areas and accessed through NGOs. Respondent driven sampling (RDS), a technique to obtain more representative samples of hard to reach populations, has been successfully tested and is likely to be expanded to other groups in future rounds.
Dissemination of the results of the surveillance has been weak mainly due to the lengthy process of clearance required within MOHFW for sharing surveillance findings, leading to substantial delays in release of information. The data is thus not currently being routinely discussed with implementing NGOs to better plan and focus HIV prevention services, and triangulate information with the data on NGO interventions being routinely collected by UNICEF.
Monitoring of NGO interventions: NASP did not have the capacity to implement the TI monitoring plan, and the participation of NASP in monitoring visits was irregular. UNICEF monitoring was also initially weak, but picked up after the first phase. During the last phase, a structured system has evolved of regular meetings with NGOs to share findings of monitoring reports and to agree on actions to remedy shortcomings identified in the field. The NGOs' quarterly reporting covers a list of input and activity indicators: number of awareness raising sessions, counseling sessions, condoms and syringes distributed etc.
Monitoring of the Blood Transfusion Component: The indicators for blood transfusion that were agreed in 2005 related to percent of blood screened in the centres, and other input related ones such as number of centres fully equipped and staff trained in screening and rational blood use. These were revisited in August 2007 and another set of indicators recommended. Neither set of indicators were tracked regularly nor was there any focus on monitoring the blood transfusion centres to improve effectiveness of services.
NASP has finalized the National AIDS Monitoring and Evaluation Framework and Operational Plan for 2006-2010, in line with the three "ones" with the assistance of UNAIDS.
2.4. Safeguard and Fiduciary Compliance (focusing on issues and their resolution, as applicable):
Tribal Development Plan: The PAD identified the tribal populations of the Chittagong Hill Tract (CHT) as being possible risk populations for HIV, as identified by a separate social assessment. Estimates put the population of CHT at between 1 and 1.5 million: only about 1% of total population. The MTR noted that, while it was too early to assess compliance with safeguard policy, there was an urgent need to take actions to take into account the findings of the social assessment conducted for the project. The MTR made several recommendations, and by April 2005 (A-M, May 2005), some progress had been made with regard to these recommendations: (i) tribal groups were consulted when preparing IEC materials; (ii) the Surveillance Advisory Committee (SAC) had included tribal populations in the sample of the 7th Round; (iii) three NGO projects had been undertaken in tribal areas for raising awareness under the HAIF. A Technical Assistance Project Proposal entitled Raising Awareness on HIV/AIDS among the Indigenous Populations in Bangladesh to make tribal people aware of the spread and prevention of HIV/AIDS had been prepared and was to be implemented by NASP with SIDA funding. Subsequent IDA missions did not specifically record progress on this component in aide-memoires. However, based on an overall assessment of progress with planned activities, the Safeguards rating was changed from "U" to "MS" in May 2005 (PSR#12), and this rating was maintained for the remaining project period. It appears that with no clear Tribal Strategy in place, and with more pressing technical and management issues to address in the project as a whole, this component received scant attention.
Environmental Issues: Environment Category 'C' (environmental assessment and plan not required). The management and disposal of used needles and syringes by IDUs had been identified during project preparation as an issue. It was expected that since HAPP was included under the overall umbrella of HPSP, the waste management system under HPSP would include this component. In addition, NGOs would be required to adopt safe segregation practices; and final disposal was to be through transportation of waste in dedicated vehicles to the closest Line Directorate Hospital. In actual practice, NGOs did collect used needles from their clients; but final disposal was undertaken on site by burning used needles and syringes (personal communication with NGO field workers).
Procurement: Procurement under HAPP has been Unsatisfactory. Procurement of goods and services had been delegated to UNICEF for NGO services as well as condoms and medicines. A Procurement Specialist was brought on board at NASP shortly after the MTR, but he resigned within a year. Subsequently there were disruptions in supplies of goods, particularly condoms. The gap of 3.5 million condoms in the period July 2005-February 2006 was tided over by an ad hoc arrangement with FHI. Another Procurement Specialist was recruited in late 2006; and it is expected that a Procurement Focal Point will be in place after the transition into HNPSP. Under HNPSP, procurement of services for HIV/AIDS activities will be undertaken by the Management Support Agency (MSA); and the CMSD will undertake procurement of goods, along with other goods procured under the HNPSP.
Financial Management: Initially, the project suffered due to a lack of clarity on how funds would flow for its implementation: although it was financed under a separate IDA Credit, it was treated as part of HPSP for implementation purposes. Additionally, there was no provision for the pooled co-financing of DfID, as a result of which there was complete non-disbursement of DfID funds prior to the MTR. All these issues were clarified during the MTR (June 2003); yet in October 2004, the Aide-Memoire recorded that financial management and reporting were complicated by the lack of a unified financial management system for HAPP. Over the duration of the project, financial management capacity remained weak and several audit issues that were pending since 2003 remained unresolved. Most of the audit objections related to financial reporting requirements of UN contracts and could have been avoided with more effective coordination with the UN Agencies. It is expected that with the transition to HNPSP, with more standardized reporting formats and systems, and a dedicated financial management team in place, financial reporting will improve.
2.5. Post-completion Operation/Next Phase (including transition arrangement to post-completion operation of investments financed by present operation, Operation & Maintenance arrangements, sustaining reforms and institutional capacity, and next phase/follow-up operation, if applicable):
HAPP is now being mainstreamed into the sector-wide HNPSP. Reasonable progress has been made towards this transition (ISR #17; 11/29/2007). Funding for HNPSP is already in place, and allocations have been made within the program for HIV/AIDS activities.
Institutionally, the NASP will continue to function as the nodal point for HIV/AIDS activities. Key positions at NASP have been recently filled, and it needs to be seen whether they have the technical competence to manage the program. NASP has prepared Procurement Plans for the 3year period remaining under HNPSP, for which No Objection has been received from the Bank.
MOHFW has agreed to contract UNICEF to manage the NGO contracts for 12 more months until December 31, 2008, after which it is expected that the Management Support Agency (MSA) under the HNPSP will take over this activity. Given the volume of contracting to be undertaken by the MSA, and the long drawn out procedure for doing so, involving clearances from various parties, there is some apprehension that the transition from UNICEF to the MSA will not be a smooth one. The financial management arrangements of HAPP will be transitioned into the arrangements set up under HNPSP, where there is a dedicated Financial Management cell, and a system is already in place for accounting and reporting that is acceptable to the Bank.
Technically, the globally accepted approach of focusing on TIs among high risk groups is well accepted within the program and will continue under HNPSP. Both BSS and sero-surveillance have been undertaken over several rounds; it is important that the MSA ensure that experienced and high quality institutions are brought on board at the earliest for the remaining period of HNPSP to undertake this key activity without interruption. The monitoring of the various contracts with NGOs and other institutions remains a matter of concern, as long as NASP lacks the capacity to do so effectively.
At the political level, the National Policy on HIV/AIDS and STD related issues was ratified in 1997, based on which various policies and/or guidelines have been developed for a range of issues. More recently, the National Harm Reduction Strategy for Drug Use and HIV, 2004-2010 and the National Strategic Plan (NSP) for HIV and AIDS, 2004-2010 have been endorsed. The Operational Plan of NSP and Resource Requirement for Prevention, Treatment, Care and Support form the basis for the rollout of the NSP.
|Printer friendly Cite/link Email Feedback|
|Publication:||Bangladesh - HIV/AIDS Prevention Project|
|Date:||Jun 10, 2008|
|Previous Article:||1. Project context, development objectives and design (this section is descriptive, taken from other documents, e.g., PAD/ISR, not evaluative).|
|Next Article:||3. Assessment of outcomes.|