Producing knowledge for development: research at the World Bank.
Research findings are an essential input to Bank operations and lending. They are used in the Bank's advisory services, which include policy dialogue with governments, analytical reports (known in Bank jargon as economic and sector work), and technical assistance. In addition to research, development practitioners and policymakers need access to good-quality country-level and comparative data. This must be combined with high-quality global monitoring and consistent projections to enable policymakers to adjust development strategies to a rapidly changing external environment.
World Bank lending provides opportunities for learning about development. Key to such learning is the evaluation of programs in a rigorous way that isolates their effects from those of other factors. Ex post, research strengthens the lending-learning-knowledge cycle by drawing lessons from lending operations. Ex ante, it explores the likely outcomes of different policy options by applying historically derived knowledge to current development challenges.
Externally, the Bank influences the global environment in which development efforts are undertaken by producing and disseminating research and data that contribute to global development knowledge. Bank staff is given the mandate and resources to do so by its governing body, which represents governments. Bank publications address the specific needs of clients, as well as issues and themes crucial to understanding the development process itself. These publications boost developing countries' access to information and analytical capacity and strengthen their ability to engage on the global stage.
This article first describes the organization, staffing, resources, and research agenda at the World Bank. It then presents an assessment of the quality and relevance of this research, drawing on the findings of a recent independent evaluation.
Organization of World Bank Research
Research is done throughout the World Bank under the general direction of the senior vice-president and chief economist. Research is carried out by Bank staff, often in collaboration with external consultants. The bulk of research is conducted by the research group in the Development Economics Vice Presidency (DEC). Research is also produced by other staff from six regional vice presidencies (Africa, South Asia, etc.) and five networks: Environmentally and Socially Sustainable Development, Infrastructure, Finance and Private Sector Development, Human Development, and Poverty Reduction and Economic Management.
Bank research is financed by the administrative budget (including a substantial amount financed by the Research Committee) and by external trust funds. The Research Committee, created in 1971 and chaired by the chief economist, has its own budget and allocates research grants competitively, following a formal review process. The Research Committee also supports capacity-building initiatives in developing countries, such as the African Economic Research Consortium (AERC), and provides financial support to economics education in underserved areas of the world.
DEC is divided into five teams: Data Group, Prospects Group, Research Group, Development Policy, and World Development Report. The Research Group produces empirical and methodological research on development policy. The Data Group is the focal point for statistical data work. (1) Data quality, availability, and timeliness have become increasingly important given the international community's focus on results. The Prospects Group, the focal point for analysis of trends and global projections, produces two annual reports: Global Development Finance and Global Economic Prospects.
Research Projects and Outputs
The World Bank produces, with limited resources, a wide range of research outputs, including books, journal articles, and a series of Policy Research Working Papers. It also publishes two acclaimed journals, holds conferences and workshops, and produces databases used by policy analysts, educators, and researchers in the development community all over the world (see http://econ.worldbank.org).
Between 1998 and 2005, the Bank undertook 705 research projects and published 3,635 research publications in English. (2) Table 1 shows the number of current research projects and research publications produced by the Bank in FY 2002/03 and FY 2004/05. (3) Both increased by more than 20 percent between the two periods.
The figures in Table 1 refer to research in a narrow sense and not to what the World Bank calls economic and sector work (ESW). The distinction between them is not easy in practice. The Bank's analytical work ranges from narrow application of existing knowledge to address questions specific to a country project, to research exploring broad patterns and relationships important for development processes. Along this spectrum, ESW includes findings that have a broad (often regional) importance, whereas some research sometimes focuses on a single case or institution within a country. What separates ESW from research lies in both intention and design. Research encompasses analytical work designed to produce results with wide applicability across countries or sectors. ESW and policy studies take the product of research and adapt it to particular projects or country settings.
Research Expenditures and Staffing
Spending on research represents only 1.1 percent of the total Bank administrative budget. After an increase between FY 2001/02 and FY 2002/03, spending on research has declined to reach its lowest share in twenty-five years and half its average level during the 1990s (Table 2). (4) At constant prices, research expenditures decreased by 3 percent per year, in FY 2003/04 and FY 2004/05.
DEC accounted for about 80 percent of all Bank expenditures on research over the period, with about two-thirds of that spending concentrated in its Development Economics Research Group (DECRG). The regions and networks accounted each for about 5 percent of research expenditures.
DEC has a total staff of 230, including regular and term professionals and support staff, supervised by five directors. The Research Group accounts for about half of the DEC staff and has seventy-seven regular and term professional researchers (down from eighty-five professional grade researchers three years ago).
Overview of the Research Agenda
The research agenda is determined by relevance. The main criterion for undertaking work is whether it is likely to produce convincing solutions to development questions that, in turn, are likely to influence policy and outcomes. Not all research concerns the immediate levers of policy, but there is an overriding awareness that policy is the channel through which research contributes to development. Consequently, most of the Bank's research is empirical, not just theoretical or methodological. Broadly speaking, the research agenda is structured around the Bank's two-pillar development strategy: building the climate for investment, job creation, increased productivity, and sustainable growth; and empowering and investing in poor people to enable them to participate in development. (5)
Research managers, in assessing relevance, consider the needs and priorities of both Bank operations and the development community. They may, for example, judge that insufficient evidence is available on a given topic--for instance, on the likely impact of WTO reforms, or of epidemics like the avian flu--and decide to invest in those topics. Identifying those needs and the research necessary to meet them relies partly on the judgment of researchers and managers--all of whom are experts in their field--and partly on many other people. Operational staff suggest topics, so research often grows out of joint work between DEC and operations. Every researcher is expected to spend 30 percent of his or her time working with operations, and operational staff is often involved directly or indirectly with research projects. In addition, DEC managers frequently meet their counterparts in operations, for example, to discuss specific projects, or in formal network and sector board meetings. Interactions with policymakers and researchers outside the Bank are also a constant source of stimulus. Finally, of course, the senior vice-president (SVP) and chief economist are heavily involved in guiding research priorities.
One of the challenges of research management is to anticipate topics sufficiently early to have research ready to address them when it is required. Although it is difficult and risky, research management also needs to create initiative for researchers to explore "blue skies" issues before any demand for them has been expressed.
In terms of substance, the emphasis of World Bank research has shifted in several ways in recent years. First, in country-level work, there has been a move away from macroeconomic studies and toward more emphasis on the country-specific microfoundations of growth and development. The aim is to better understand poorly functioning markets and market failures, how institutions arise at the microlevel to cope with such failures, and the implications for policy. Research programs on the investment climate and on mechanisms of delivering public services are core components of this work. There has also been a growing interest in the role of institutions in shaping governance. Research continues on aid effectiveness and on policies to achieve the Millennium Development Goals (MDGs). This work focuses on the consistency between public expenditure programs and development goals, on scaling up assistance to meet the MDGs, and on the political economy of development assistance.
Second, the Bank has long been undertaking research on global issues, including trade policy reform, the impact of World Trade Organization (WTO) and regional trade arrangements, agricultural trade, the abolition of textile quotas, and the Doha Round. In addition, a major research program on the role of international migration and remittances in development started in 2004. Research on important global externalities, such as climate change and conflicts, has also been initiated. The latter focuses on fragile low-income states and also examines new approaches to aid in postconflict countries.
Finally, a major program of impact evaluation has been launched, responding to the increasing demand to improve development effectiveness and assess results on the ground. New evaluation tools, often based on randomized trials, are becoming available for evaluating the development impact of projects in education, health, or infrastructure, and in delivery mechanisms and cash transfers. There is a concerted effort throughout the Bank--the Development Impact Evaluation (DIME) initiative--to support and coordinate such evaluations of Bank-financed programs for different types of development interventions. DIME monitors the performance of projects and identifies what works and when.
Quality and Relevance of World Bank Research
In 2006, an evaluation exercise was carried out to address the quality and relevance of Bank research. A panel of external experts was asked to what extent World Bank research has achieved two objectives: generating new knowledge on development; and contributing to a broader understanding of development policy. Only two similar exercises have taken place in the history of the World Bank--in 1978 (chaired by Sir Arthur Lewis) and in 1983 (chaired by Assar Lindbeck). Partial evaluations of some DEC research by external reviewers were also carried out, the last one dating from 1998.
The independent panel of experts, chaired by Angus Deaton, was appointed by the Bank's chief economist. (6) The panel, in turn, selected thematic evaluators who are world-class researchers in their field (7) and asked them to review a random sample of 186 research projects. Using this sample, they evaluated publications by focusing on reliability, rigor, completeness, and relevance. They also provided an overall assessment on the strengths and weaknesses of Bank research in their areas of expertise and their general knowledge of Bank research.
The Quality of Research
The evaluation concluded that much of the Bank's research is of leading quality, that "Bank researchers have delivered value to the institution that is much larger than could be reasonably expected from the very small share of the budget that they command," and that they have done so in a period when new hiring has been severely limited and where the salaries of Bank economists have fallen rapidly relative to those in academia.
The panel and evaluators found some outstanding work in the Bank's portfolio, especially publications on the measurement of poverty and inequality; pioneering work on the delivery of educational and health services; work on environmental monitoring; surveys such as the Living Standards Measurement Surveys and the Investment Climate Surveys; and the Bank's leadership in the International Comparison Project. According to the panel, the Bank has maintained its position as the intellectual leader among development agencies, thanks to innovative and valuable research. The evaluators considered 61 percent of the studies assessed to be above average quality and 28 percent average.
The panel was critical of several Bank research outputs, particularly the work on growth, aid, and poverty, (8) and on pension reform. They criticized the fact that certain techniques (in particular cross-country regressions) have been overused and that this research has been used to underpin Bank advocacy without ensuring the robustness of the empirical results. (9)
The Balance Between Quality and Relevance
The evaluators also found that a fraction of Bank research was "undistinguished," in the sense that it had neither great relevance to policy nor claim to academic distinction. These are subjective judgments, but the evaluators are distinguished development economists, and they all shared similar views. Their judgments did not refer to the lack of good papers in good journals, many of which were innovative and important by any standard. Nor were any of them counting citations. The evaluators did not provide objective evidence that the Bank's research "tail" is longer than that of other leading institutions. The concern was that a large fraction of papers did not seem to be useful from the perspective of an academic or a policymaker. The criticism stresses that these papers were driven more by the concerns of journal referees rather than by the policy needs of the Bank.
There is an unwritten rule that each researcher in DEC is expected to publish two papers per year in leading academic journals. The panel considers that the norm of "two publications a year" is a reasonable mechanism. But the panel also recognizes that the publication rule may lead to a substantial body of work that is successful neither academically nor in policy relevance.
The Balance Between Research and Advocacy
The panel mentioned that a great deal of the research on globalization, growth, and poverty, as well as pension reform, was useful but that there was a failure of the checks and balances that should separate advocacy and research. The panel endorses the right of the Bank to strongly defend and advocate its own policies but states that when the Bank leadership selectively appeals to relatively new and untested research as hard evidence that its preferred policies work, it lends unwarranted confidence to the Bank's prescriptions. In other words, the panel criticized the way the research was used to proselytize on behalf of Bank policy, often without taking a balanced view of the evidence and without expressing appropriate skepticism.
The Bank's Data Function
The panel sees the Bank's data work as central to its mission of learning from development. Yet it claims that data activities are organized haphazardly, whether in collection, archiving, or dissemination. The Bank could strengthen its central role in development data--which is a public good for the international community. But three observations should be made here. First, development data are not collected only for research purposes. They are also used to monitor activity in developing countries and to provide information to Bank clients. Second, the Bank has been encouraging open access to data for many years, but many client countries do not allow access to the data except for limited official purposes. Third, strengthening the Bank's data function would require senior management to endorse the Bank's central data role and to channel substantial financial and human resources into the area over the long term, as well as to implement some changes in the Bank's business model.
Outreach and Relationships with Academia
Bank researchers have close links with academics around the world, whom they often hire as consultants, and have regular interchanges with them through conferences and other means. The panel recommended specific steps to improve outreach and relationships with academia, including revitalizing the visitors program, strengthening the research function of the Annual Bank Conference on Development Economics (ABCDE), and improving the accessibility of the DEC website.
Much of the World Bank's engagement is intellectual. The Bank is equipped to make an assessment of the state of development. The research it carries out is policy oriented, primarily economic, and almost entirely applied. Researchers have a considerable degree of freedom from the Bank's shareholders to choose relevant topics they want to work on. The research they have selected includes topics that have managed to irritate some powerful shareholders--for example, research on agricultural trade barriers. The World Bank is not an apolitical institution--it views itself as an advocate for developing countries in the international arena--but its views on development are evidence based: this is what gives it legitimacy. Its thinking on development has evolved in line with that of the development community and to that extent, it follows trends--but it also leads trends. Throughout its history, the World Bank has pioneered a number of tools of analysis. The Bank has also been good at "mainstreaming" topics that are important for development, such as gender, and at identifying certain topics before most of the world took notice, which have turned out subsequently to matter, such as HIV/AIDS and conditional cash transfers. World Bank research aims to be policy relevant and of high quality. According to the recent evaluation, it has to a large extent succeeded and has produced findings that have great value for the international community. The Research Department attempts to be as competent as top academic departments specialized in development, such as MIT, Harvard's Kennedy School, the London School of Economics, and DELTA (now the Paris School of Economics) in Paris. But it also requires its researchers to spend a third of their time getting involved in and providing support to operational staff. Research expertise must evolve slowly to become top quality. It takes time to build up a critical mass. It cannot afford to be subjected to fashions and fads. Researchers need to be engaged but also to be measured. There is a time gap between the research stage and what comes out. All the hypotheses need to be carefully tested, and it is essential to obtain accurate evidence before any policy recommendation is made. Research is inherently a risky business and failures are to be expected; but, judging by the citations and evaluators' comments, failures were not a disproportionate share. Thanks to the quality and relevance of its research, the World Bank has been able so far to maintain its role as the world's leading development agency.
Jean-Jacques Dethier is a research manager at the World Bank. The findings, interpretations, and conclusions expressed in this article are entirely those of the author. They do not necessarily represent the views of the World Bank, its executive directors, or the countries they represent.
1. The Data Group produces World Development Indicators and manages the Development Data Platform (DDP), a Web-based repository of a large number of surveys that is linked to software for analyzing and disseminating time-series information (through Web-based tables, charts, and maps).
2. The complete list of Bank research publications in 1998-2005 is available, with the evaluation report, at http://econ.worldbank.org/research (see note 9).
3. Of the 617 ongoing projects, there were 170 new projects initiated in FY 2004/05.
4. The budget figures reported here refer to activities charged as research (RF code) in the Time Reporting System and do not include spending on ESW that can be assimilated to research. They include direct charges, such as staff time and travel, but omit management and clerical staff time and other overhead expenses. The figures do not include data from the International Finance Corporation (IFC) and the Multilateral Investment Guarantee Agency (MIGA).
5. For a more complete discussion, see Nicholas Stern, Jean-Jacques Dethier, and Halsey Rogers, Growth and Empowerment: Making Development Happen (Cambridge: MIT Press, 2005).
6. The panel includes Angus Deaton (Princeton University), Abhijit Banerjee (Massachusetts Institute of Technology, MIT), Nora Lustig (United Nations Development Programme), and Kenneth Rogoff (Harvard University), former chief economist at the IMF.
7. The evaluators are Daron Acemolu (MIT), Francesco Caselli (London School of Economics), Tim Besley (London School of Economics), Sebastian Edwards (UCLA), Gordon Hanson (UC San Diego), Nina Pavcnik (Dartmouth College), Esther Duflo (MIT), Murray Leibbrandt (University of Cape Town), Nancy Birdsall (Center for Global Development), Josh Angrist (MIT), Sebastian Galiani (Universidad de San Andres, Argentina), Jonathan Morduch (NYU), Marianne Bertrand (University of Chicago), Justin Lin (Beijing University), Chris Udry (Yale), Marcel Fafchamps (Oxford), Edward Glaeser (Harvard), Michael Kremer (Harvard), Andrew Foster (Brown), Geoffrey Heal (Columbia), Peter Diamond (MIT), Antoinette Schoar (MIT), and Jan Sjvenar (University of Michigan).
8. The panel mentions "Aid, Policies, and Growth," by Craig Burnside and David Dollar, in the American Economic Review, 2000 (which has 743 citations in Google Scholar); and "Growth Is Good for the Poor," by David Dollar and Aart Kraay, in the Economic Journal, 2004 (893 citations in Google Scholar), as well as the 2001 World Bank Policy Research Report, Globalization, Growth, and Poverty.
9. The complete evaluation report and the response of the Bank's chief economist are available on the World Bank external website: www.worldbank.org (type "research evaluation" in the search box) and/or at http://econ.worldbank.org/research.
Table 1 World Bank Research: Comparative Indicators, Fiscal Years 2002/03 and 2004/05 Number of Research Publications Number of Research Projects 2002/03 2004/05 2002/03 2004/05 Poverty 90 178 54 75 Growth and investment 148 168 83 121 Human development and 202 237 87 109 public services Finance 110 120 43 47 Infrastructure and 95 117 81 94 environment Rural development 82 98 74 70 Trade and international 249 346 65 101 integration Total 976 1,264 487 617 Source: Development Economics Vice Presidency (DEC), World Bank. Table 2 World Bank Expenditures on Research and Other Services, Fiscal Years 2002/03-2005/06 2002/03 2003/04 2004/05 2005/06 Research Current millions of $ 19.4 21.9 22 20.8 Percentage of administrative 1.20 1.30 1.20 1.10 budget Economic and sector work Current millions of $ 90.3 112.7 120 122.3 Percentage of admininstrative 5.70 6.60 6.40 6.30 budget Technical assistance Current millions of $ 37 36.6 33 35.4 Percentage of admininstrative 2.30 2.10 1.70 1.80 budget Total administrative budget (millions of $) 1,589.70 1,704.20 1,889.40 1,939.70 Source: Business Warehouse, World Bank. Note: Expenditures exclude trust funds.
|Printer friendly Cite/link Email Feedback|
|Title Annotation:||GLOBAL INSIGHTS|
|Date:||Oct 1, 2007|
|Previous Article:||The role of the World Bank in poverty alleviation and human development in the twenty-first century: an introduction.|
|Next Article:||The disciplinary monopoly in development research at the World Bank.|