Audit The World Bank.
"[R]ecent data from the Bank's independent Operations Evaluations Department [OED] show substantial improvement in project outcomes, especially for adjustment lending. For operations evaluated in FY98-99, 81 percent of funds lent and 93 percent of funds lent for adjustment were in operations rated satisfactory or better by OED. Even by the very stringent test of likely sustainability, the respective scores were 66 percent and 84 percent ..." Bank Vice President Salop writes (emphasis mine).
When the Bank's statement of overwhelming success is examined, word by word and number by number, the translation of the above quote into the language of the real world should actually read: "For projects where the disbursement of loan monies to borrowers was completed in 1998-99, but years before physical or policy practical effect can be measured, Bank staff estimate that less than half (47 percent) are likely to maintain satisfactory results in the future and to generate benefits that exceed debt service payments, operation, and maintenance costs."
"Independent," as the OED refers to itself, normally denotes freedom from control or influence. Yet the OED evaluation unit is predominantly staffed by Bank employees on a rotating tour of duty, who, save the Director General, look forward to returning to standard line jobs in the Bank. But a temporary change of desk and a new nameplate do not alter the signature on the paycheck. The Executive Board, to which the group reports, is passive at best. And because results are public, there is strong pressure to display performance gains by what Ms. Salop holds up as the "New Bank."
"Outcomes" are usually equated with results and are measured after the fact. At the Bank, however, judgments are determined at the time of final loan disbursement, often years before physical plants are up and running or policy changes are executed and can impact the economy. "Estimates" or "projections" are the terms employed in the real world at the point of disbursement, not "outcomes."
"Adjustment" programs, which represent almost 40 percent of the collective total and receive the most elevated marks, are also the most difficult to quantify. Is there a staff soothsayer who predicts ten-year results, even before reforms are implemented?
And what is "satisfactory"? For the Bank, it runs the wide and undifferentiated gamut from marginally satisfactory to highly satisfactory and is divorced from the sine qua non of development, that of sustainability. In the Bank's 1997 Annual Review of Development Effectiveness, "sustainability" was defined as "the likelihood that the project will maintain its results in the future.... The evaluator must determine whether, given the risks, future benefits are likely to exceed debt service payments, operation, and maintenance costs." Is this not a necessary condition of a "satisfactory" outcome? As Bank President James Wolfensohn testified before the Meltzer Commission: the Bank can have "a very satisfactory program of building schools ... [but] [i]f we have not simultaneously dealt with roads to get the kids to school [and] teacher training ... you won't have a successful long term project."
Numbers are only as good as the data, and the Bank has chosen to ignore the record in its most widely-used table of evaluation which focuses on numbers of projects and has based its argument instead on dollar amounts of lending. This leads to overstatement of success rates by imparting greater weight to the large, prosperous countries that borrow in the form of multi-billion dollar structural adjustment loans. As would be expected, success for this group is uniformly higher -- an implausible 98 percent satisfactory and 99 percent predicted sustainability in 1998-99. For the most needy nations, ratings are 69 percent for satisfactory outcomes and 45 percent for sustainable results.
A look at the Bank's primary data classified by number of projects reveals a different picture in which success figures are sharply lower. For the 1998-99 time span, 72 percent of all projects were rated as satisfactory, while likely sustainability falls to 49 percent. When the OED was asked to combine the criteria of satisfactory outcome and likely sustainability of benefits, only 47 percent of projects qualified in the 1998-99 period. For two previous periods, 1990-93 and 1994-97, the figures were 41 percent and 44 percent. All were below 50 percent success.
The current trend is down not up. Rather than the "substantial improvement in project outcomes" claimed by the Bank, the data show a recent deterioration: satisfactory ratings have fallen over the past two years from 74 percent in 1997 back to 1996 levels of 69-70 percent.
Semantics aside, there is the substantive issue of whether any of the numbers are accurate gauges of effectiveness. During testimony before the Meltzer Commission, Mr. Wolfensohn was asked why the Bank measures its success at the time of the last loan disbursement and why performance of programs is not scrutinized routinely, four or five years later, after an operating history is available? He could only respond: "I've asked the same questions.... Why do we measure the data from the date of the last (loan disbursement) payment?" After five years of the "New Bank" even a simple change in timing has not been forthcoming.
Since both the president of the Bank and the Commission concur that the data are of limited usefulness and that it is urgent to ascertain the true effectiveness of Bank projects, why not establish a program of truly impartial audits by private sector firms to examine on site the lasting viability of projects after a four to six year operating history. Individual program audits and complete evaluations of performance would be published and the exercise repeated every three years to provide a continuing benchmark for Bank efforts. Auditors would report directly to the legislative and executive branches of the Group of Seven.
It is all too easy for the Bank to hide behind the technicalities of client confidentiality and sovereignty rights of nations that wish to evade scrutiny. For those on the receiving end of subsidies mounting to billions of dollars each year that flow from industrialized nation taxpayers through the channel of World Bank financing, there should be a corresponding obligation. Truly independent and publicly available performance audits must be a condition of all World Bank loans.
Never before have those current bywords of accountability and transparency been more clearly indicated. Both the responsible use of donor country taxpayer monies and the increasingly desperate condition of the global poor demand it. Why should the Bank not set the example for the standards it requires of its borrowers?
Adam Lerrick is Senior Advisor to the Chairman of the International Financial Institution Advisory Commision.
|Printer friendly Cite/link Email Feedback|
|Publication:||The International Economy|
|Date:||Jul 1, 2000|
|Previous Article:||Benign Neglect.|
|Next Article:||Today's Global Capital War.|