Annex: study methodology.
* Update of the database, created for the 1994 study, of investment projects approved since FY67 and found to include some type of gender-related action. The original database included 615 projects approved from FY67-93. The database now includes the years FY94-96, with a total of 802 projects. Of these 802,254 are completed and have been rated for project achievements (in an implementation completion report [ICR] if not through an OED review).
* Analysis of implementation and results for the 58 projects with gender-related action, approved in FY87 or since, completed by December 1995, and for which a project completion report (PCR) or ICR could be obtained by August 15,1996 (at least in draft). Project documents and files were reviewed, together with midterm evaluations and completion report.
* Review of sector work focused specifically on gender issues (women in development [WID] assessments had been reviewed for the 1994 study), as well as poverty assessments and country assistance strategy papers for the 12 countries with the greatest gap between men and women on key health and education indicators.
* Focus groups with task managers who included particularly good attention to gender issues in their projects and sector work, and with staff responsible for promoting attention to gender in the regions and central vice-presidencies. A total of 43 staff participated.
Sources of data
The 1994 database of 615 projects with some type of gender-related action had been compiled from the Bank's management information system, notes from the first WID advisor, a database created by the first WID division with projects approved starting in FY88, and reviews of annual reports, world development reports, and internal reports. For projects approved from FY94-96, a list of projects with gender-related action was obtained from the Gender Analysis and Policy team (GAP). In all cases, OED re viewed the project objectives to confirm the presence of some gender-related action. The percentages of projects with gender-related action in this report differ slightly from those given in earlier Women in Development Division, Population and Human Resources Department (PHRWD), or GAP progress reports, for two reasons. First, since practically all projects with gender-related action are investment projects, OED calculated percentages against all investment projects only, while PHRWD gave percentages of projects with gender-related action against all lending operations. Second, PHRWD made a few corrections in its database after some of its early reports were issued. After further checks by OED, a few more errors (project duplication) were corrected, slightly lowering the total number of projects with gender-related action for FY88-93. For projects approved in FY94-96, the OED review confirmed all rating given by GAP.
Review and analysis of completed projects
The 1994 study evaluated the implementation experience and gender achievements for two clusters of projects with gender-related action: (a) an "old cluster" of the 36 projects cited in World Bank 1979; and (2) a "new cluster" of 24 projects approved in FY87-88 and expected to be completed by June 1994. Surprisingly, 8 of these 24 projects were still not completed in June 1995, although they had been scheduled for closing before or on June 1994 at the time of the study.
For the update report, the characteristics and achievements of completed projects are discussed on the basis of a larger cluster of 58 projects with gender-related action: all such projects approved in FY87 or later, closed by December 1995, and for which a PCR or ICR could be obtained, even in draft form, by August 15,1996. This cluster combines 16 of the projects already discussed in the new cluster of the 1994 study, albeit with more evidence of results, together with an additional 42 projects. In each case, the most recent rating of outcomes, sustainability, and institutional development impact was used, that is, audit or OED rating on the basis of an ICR review, or the ICR itself. In the update, the analysis is limited to projects that are actually completed, rather than within six months of expected completion, partly because of the delayed closings observed for the 1994 new cluster, and partly so evaluation ratings would be available for all these completed projects.
The review focused on the implementation experience and actual achievements of the gender-related actions objectives, the quality of their supervision, and generally their incorporation in the overall project implementation and achievements. All documents from staff appraisal reports (SAR) to ICRs or audits were reviewed, together with the project files and supervision reports, using a standard questionnaire revised in light of the 1994 findings. Achievements of the gender-related actions were rated against their stated objectives, and quality of supervision was assessed through evidence of attention to gender issues in supervision reports or aide-memoires. A matrix was built around the key factors that could affect implementation.
A logit analysis was done to assess whether various variables made achievement of gender objectives in these 58 projects more likely. Univariate analyses show that gender achievement is lower in the Africa regional office (but a weak relationship only--see the end of this section for further explanation) and higher in the human resources sector. There was no relationship with the source of funding (International Development Association [IDA]/ International Bank for Reconstruction and Development) nor with the countries' income.
All projects that achieved their gender objectives were rated satisfactory. Positive correlations were also found between sustainability and gender achievement, and between institutional development and gender achievement. A relationship noted between time period and gender achievement (recent projects being more likely to achieve their gender objectives) could be due to the higher concentration of human resources projects in the more recent projects or to the known fact that delayed completion often signals implementation difficulties.
In order to determine which factor(s) seemed correlated to achievement of gender objectives, an unconditional logistic regression analysis was done. The outcome variables were defined as gender achievement score of "2" or higher versus "0" or "1." The explanatory variables were as follows: fiscal year of approval of the project, amount of loan in nearest million dollars, income level of country, any IDA component to the loan, region, agricultural, human resource or other sector, sustainability, institutional development, and outcome scores as based upon the OED rating schemes, gender analysis, supervision of the gender component, integration of the gender objective into the main project objectives, and supervision of the gender component.
Variables were analyzed in a logistic regression model. The EGRET software model was used, and tested for significance to a two-sided p value of 5 percent. Each explanatory variable was examined in the model singularly, and then a baseline model consisting of year, amount, income level and IDA lending was established. To this baseline model, individual variables which were significant in univariate analyses were examined individually. These were Africa or other region; human resource or other sector; outcome, sustainability, and institutional development scores; and integration of the gender objectives into the main objective. Finally the last model included all variables from the baseline model and the significant explanatory variables. The latter may be unreliable due to the small sample size and co-linearity of some of the explanatory variables. For each analysis, we produced two sided p values, beta-coefficients, and an overall goodness of fit, as measured by deviance on the appropriate degrees of freedom. Any missing values were omitted from the analyses.
Review of design quality in 120 projects approved in FY94-95
The SAR and president's report for all projects with gender-related action approved in those two years were reviewed, together with any supervision documentation already available. The 1994 questionnaire was used (with some revisions). The reviewer assigned ratings for the presence and quality of gender analysis, presence and level of participatory approaches during design or implementation, and integration of gender objectives into project objectives. Staff (and in a few cases consultants) involved were frequently interviewed for further information and to understand the context in which some design decisions were made.
Focus group discussions with successful task managers and gender coordinators
The OED reviews of selected sector work and of design quality in projects approved in FY94 and FY95, together with information provided by the regional coordinators, led to the identification of 85 operations (mostly projects, with a few economic and sector work [ESW]) with good attention to gender aspects in their design. The task managers for these projects were invited to two focus group discussions. The enthusiasm with which most of the invitees reacted, even though many were or would be on mission at the proposed dates, shows a need for recognition that their work is worthwhile and a strong desire to help change Bank practices. Twenty-three task managers participated in two groupware sessions; another eight contributed comments and additions to the draft summary of the two focus group discussions.
The task managers were asked to list individually factors that facilitated or hampered the integration of gender issues in the ESW or project for which they were selected. In the next step, the groups organized their lists into logical categories. Finally, the participants were asked to draw an action plan through which the Bank could act on the factors under its control, or could promote attention to gender in its dialogue with the borrowers.
Because task managers raised a number of institutional issues, another round of focus groups was organized with staff who have a formal assignment (full or part time) to promote attention to gender in their region or sector, as well as all members of the central GAP. Twenty-seven staff were invited, nine participated, and another four provided their views individually. The participants were asked to identify what evidence would demonstrate that gender was being mainstreamed in line with gender policy. They were then asked to assess the Bank's strengths and weaknesses against the standard thus established and to identify actions that would accelerate the pace of mainstreaming.
|Printer friendly Cite/link Email Feedback|
|Title Annotation:||Mainstreaming Gender in World Bank Lending: An Update|
|Publication:||Mainstreaming Gender in World Bank Lending|
|Date:||Jun 1, 1997|
|Previous Article:||5. Gender in a changing Bank.|