5. Assessment of bank and borrower performance (relating to design, implementation and outcome issues).
(a) Bank Performance in Ensuring Quality at Entry (i.e., performance through lending phase):
Quality at Entry is rated Unsatisfactory. It is the general consensus of Country Management and the independent reviews that the project was poorly prepared. The PSR dated 12/22/2002 carries the following comment from the Manager: "The project's implementation performance demonstrates once again the impact of project quality at entry. However, the project was not adequately prepared and appraised before it went to the Board. Project implementation plan (PIP) and implementation arrangements, including project management arrangements were not discussed and firmed up during project preparation. The fact that all these critical project preparation activities were deferred to the first year of project implementation proved to be a major failure and will continue to impact on the project performance for the remaining project period...". This view was fully supported by the Problem Solving Mission and the QER Panel, which squarely held poor project preparation responsible for a large part of the problems faced by the project during the initial phase.
(b) Quality of Supervision (including of fiduciary and safeguards policies):
Rating: Moderately Satisfactory
Quality of supervision is rated Moderately Satisfactory. The initial delays in project start-up were largely due to poor preparation; but could also be attributed to the Bank's wanting to mainstream the project into the sector-wide program, which caused the loss of focus and ownership of HAPP both within GOB and the Bank. However, when it became clear that the project was seriously faltering, the Bank did take proactive steps to address the issue: a Problem Solving Mission, followed by a QER were undertaken shortly prior to the MTR to add value and provide guidance to the MTR process; and based on the recommendation of these missions, restructuring was undertaken in a proactive manner, and an attempt was made to put the project back on track. The MTR was used as an opportunity to set realistic goals for the project and to scale down project activities to fit within the reduced timeframe. Subsequently, the project did benefit from advice and technical assistance from the team, and was able to improve implementation performance as a result.
Joint Review Missions were undertaken along with DfID and UNAIDS, conducting a comprehensive review of project implementation, as well as analysis of available data. Issues faced during implementation were documented in the Aide Memoires and PSR/ISRs. There was a period prior to the MTR when documentation was perfunctory (PSR #04), and the Manager's Comments section reflected managements dissatisfaction; however, this was not repeated. Project ratings were fair, and reflected the assessment of the mission; they were responsive to changes in project status.
* There were 7 changes in Task Team Leaders (TTLs) over the course of the project. This lack of continuity caused disruptions in dialogue with the government and in some cases led to differing advice. For instance the package for male sex workers, a group widely recognized to be most at risk, was dropped during the MTR as being "too sensitive" and included again in the last phase of the project.
* Although the MTR attempted to get TIs off the ground, this was problematic: due to the procedural lags involved in contracting UNICEF and then the implementing NGOs less than a year was left in the project period by the time the TIs were initiated. Given that HIV prevention interventions deal with sensitive socio-economic, gender and behavior change issues, the Bank should have explored options early on to ensure continuity and longer-term engagement, particularly for TIs.
* The subsequent short term extensions further compounded the uncertainties; here again, the timing of the extensions was determined by the constraints of the HNPSP, and not the best interests of the activities initiated under HAPP.
* The design and follow-up on the Results Framework has not been systematic. Some indicators included in the PAD were never reported on; many indicators had no targets set; and many were re-worded without explanation. In addition some of the targets were overly ambitious and not adjusted in line of the reduced time frame of interventions such as 100 percent coverage and 95 percent of IDUs not sharing needles by end of project.
(c) Justification of Rating for Overall Bank Performance:
Rating: Moderately Unsatisfactory
Since preparation was Unsatisfactory and supervision only Moderately Satisfactory; and Development Outcome is rated Moderately Unsatisfactory; overall rating is Moderately Unsatisfactory.
NOTE: When the government and implementing agency are indistinguishable, provide rating and justification only for Overall Borrower Performance.
 Click here if the Government and the Implementation Agency is the same or indistinguishable
(a) Government Performance:
Rating: Moderately Unsatisfactory
The project was managed on behalf of GOB by the National AIDS and STD Program (NASP). Government performance is rated Moderately Unsatisfactory:
* There was no movement on any of the components for the first two years of project life, largely due to lack of leadership and ownership of the project within the NASP.
* Capacity at NASP for managing the project was an on-going concern. There were frequent changes in leadership at the level of both Line Directors and Director General Health Services, leading to lack of knowledgeable oversight. Several activities suffered as a result, including extensions of the UNICEF contract, and as a result, NGO contracts.
* Professional posts, that ought to have been filled with specialists, were filled with a mix of MOHFW out-posted staff, contract staff and consultants, all with different loyalties and different pay structures. As a result there was no cohesive and functional team.
* The work itself was carried out by UNDP-contracted consultants who were to assist the NASP team. Instead, they did NASP's work and NASP staff missed an opportunity to strengthen their skills by working together with the consultants.
* The capacities of NASP were further over-stretched by having to not only manage the program but also try and coordinate among donor partners over current and future programs. As a result, both activities suffered.
* Both technical and managerial supervision were weak. PMU staff rarely went to the field; and in any case, many were not technically competent to provide useful guidance.
Despite this, NASP was able to push through important policy initiatives, as well as a large number of guidelines and standards for service delivery. They have also been able successful in repeatedly mobilizing a large amount of funding and support from international agencies such as the GFATM. The Program also deserves some credit for being open to the innovative solution of using a UN Agency for managing the contracting and monitoring of NGOs. NASP has also managed to complete many of the preparatory activities for mainstreaming HAPP into HNPSP, such as the Operational Plan and Procurement Plan.
(b) Implementing Agency or Agencies Performance:
Rating: Moderately Unsatisfactory
The UN Agencies and NGOs were involved in the implementation of various components. The assessment of the UN agencies' performance should be seen in the context of the fact that they agreed to take on a role that was outside their mandate and area of expertise (particularly in case of UNICEF and UNFPA) in a last ditch effort to salvage the project.
UNICEF (NGO contracting and IEC): UNICEF performance is rated Moderately Satisfactory. 37 NGOs, grouped in 12 consortia, implemented the TIs among various high risk groups identified under HAPP. During the second extension phase, the consortia covered a target population of 109,229, including IDUs and heroin smokers (31%), SBSWs (5%), BBSWs (4%), HRBSWs (8%), MSM (6%), and CLSWs (46%). From 2003-2006, UNICEF also managed the HAIF, which provided small grants to NGOs and Community Based Organizations. This was discontinued in 2006 due to high management costs, limited evidence of impact and poor targeting. There were gaps in service delivery due to the delays in government clearance of extensions and to UNICEF procedures that did not allow for retroactive financing. Despite managing the contracting of NGOs over a period of 3 years, UNICEF has not been able to develop reliable costs for each package; and given the current variation in costs, it is evident that there is also a variation in the content and quality of services being offered by different NGOs. Capacity building of implementing NGOs was undertaken in an ad hoc manner. However, TIs for high risk groups, implemented by NGOs contracted by UNICEF and others, have been the central focus and backbone of HIV prevention efforts in the country (A-M, November 2007).
In terms of communications, the main output of UNICEF was the National HIV/AIDS Communications Strategy 2005-2010, produced by the Johns Hopkins University. The strategy and implementation plan have mixed advocacy and awareness-raising without prioritization or clear goals. Stakeholders were not adequately involved in the process of developing the strategy, as a result of which there is little ownership and interest in implementing the recommendation.
UNFPA (Capacity Building): The performance of UNFPA is rated Unsatisfactory. The specific objectives of the UNFPA contract were to: "achieve a strengthened institutional response within the MOHFW and the NASP to lead and coordinate national efforts.; effective management and administrative support.; intensified and scale up interventions; improved institutional capacity within other key ministries; broader base of technical expertise available in Bangladesh."
A final assessment of the UNFPA "Capacity Building" contract under HAPP was undertaken in April 2007. The assessment observes that not much progress was made in developing the organizational and management structure that should have served as the basis of capacity building and concludes that very little was accomplished by way of long-term institutional strengthening of NASP. There was a complete mismatch between the expectations of the Bank/DfID and the understanding within UNFPA of its role vis-a-vis capacity building under HAPP; UNFPA was of the view that management/organizational issues were not part of the contract while the funding agencies expected UNFPA to take the lead in analysis of organizational tasks, proposing options and actively advocating for solutions. The ambitious objectives of the TA were beyond the technical capacity of UNFPA and the situation was further compounded by the fact that MOHFW did not allocate sufficient or adequately qualified counterparts at NASP reflecting a lack of government commitment and ownership. A part of the failure to perform could also be ascribed to lack of a support from UNICEF, UNAIDS and WHO (A-M, Joint Review Mission, April, August 2006) in the effort to get NASP to be more proactive in coordinating the activities of various agencies.
As elaborated in Section 3.5 (b) the TA did contribute to the program through several pieces of technical work some of which were of high quality including the STI guidelines and the harm reduction strategy. It was less successful in developing implementation plans for the communication and advocacy strategy.
WHO (Blood Safety): The performance of WHO is rated Moderately Unsatisfactory. The specific objectives of the WHO contract were to strengthen capacity of 19 blood transfusion centres through ensuring mandatory blood screening including quality assurance, rational use of blood and increasing voluntary blood donations. WHO was also to provide supplies to continue screen in 79 other centres.
An expert assessment of the safe blood transfusion component was conducted at the end of project in November 2007. The assessment found that although the centres were well equipped, good quality supplies were used for testing and managers and medical staff had been trained, little attention had been paid to make effective use of these inputs to improve blood safety management and quality assurance in any of the centres visited. The equipment was installed in unsuitable premises because the planned refurbishment of the centres could not be completed over the three year period and there was no planning for service maintenance of the equipment. Improvement in voluntary blood donations was limited. Despite extensive training of a large number of staff there was little change in practice due to the lack of a system of mentoring and supportive supervision of the trainees. Part of the problem was that the Government did not assign a counterpart from the Ministy. International TA provided by WHO comprised of brief visits by four different consultants who often provided differing advice.
The limited progress was attributed to the lack of focus on monitoring and improving effectiveness of inputs, little oversight and sustained guidance of the project by an international expert as well as to the short term repeated contracts that were conducive to an input based approach. The assessment concluded that "despite some improvements in provision of numerous inputs under the SBTP program in the previous 3 years, progress on improving standards of blood safety had not been satisfactory... "
NGOs (Implementation of TIs): The performance of implementing NGOs is rated Moderately Satisfactory. 130 Drop-In Centers (DICs) formed the core of the intervention, and were set up at all intervention sites. Peer Educators were used to communicate with the target groups. Awareness raising activities were undertaken with both high risk groups and the general community, as also STD treatment and other referral services, condom distribution and needle exchange. However, many intermediate indicators of service delivery show variable results.
(c) Justification of Rating for Overall Borrower Performance:
Rating: Moderately Unsatisfactory
Although government commitment to HIV/AIDS prevention efforts has been growing, this has not been reflected in more human and other resources being allocated to the prevention program. Capacity within the program to manage key activities such as NGO contracting, financial management and procurement of goods continues to be weak. However, there is a strong partnership with capable grassroots organizations across the country, as well as with other donor partners. This could serve as a good basis for scaling up the program in future. It is commendable that Government took a strong public health approach and dealt with normally marginalized groups in society in a professional manner in order to combat HIV.
|Printer friendly Cite/link Email Feedback|
|Publication:||Bangladesh - HIV/AIDS Prevention Project|
|Date:||Jun 10, 2008|
|Previous Article:||4. Rationale for rating of risk to development outcome.|
|Next Article:||6. Lessons learned (both project-specific and of wide general application).|