Printer Friendly

Impact evaluation of the smallholder dairy commercialization programme in Kenya.

Contents

                                                                   Page

Acknowledgements                                                      i
Acronyms                                                             ii
Plain Language Summary                                              iii
Introduction                                                          1
Intervention, Theory of Change, and Research Hypotheses               3
  Development of Intervention                                         3
  Theory of Change                                                    5
  Primary Outcomes and Impacts of Interest                            9
Context                                                              10
Timeline                                                             16
Evaluation: Design, Methods, and Implementation                      16
  IRB                                                                16
  Quantitative Design                                                16
  Qualitative Design                                                 20
  Limitations of the Study                                           23
Programme or Policy: Design, Methods, and Implementation             23
Impact Analysis and Results of Key Evaluation Questions              24
  Descriptive Statistics and Quality of Counterfactual               24
  Mixed Methods Analysis (Quantitative and Qualitative)              28
  Intermediate Outcomes                                              29
  Main Outcomes                                                      36
  Gender Differential Effects                                        39
  Cost-Analysis                                                      41
Discussion and Conclusions                                           42
  Internal and External Validity                                     46
References                                                           48
Appendix A  Field notes and other information from formative work     1
Appendix C  Survey instruments (qualitative and quantitative)         1
Appendix E  Sample Size and Power Calculations                        1
Appendix J  Cost Data for the Programme                               1
Appendix K   do files                                                 1

Tables

                                                                   Page

Table 1: SDCP Divisions and Evaluation Study Areas                   14
Table 2: Qualitative Sampling                                        21
Table 3: Pre-Programme, Division-Level Characteristics used for
Matching (N=183)                                                     25
Table 4: Determinants of treatment probability                       26
Table 5: Standardized differences in observable characteristics
with and without weights                   Error! Bookmark not defined.
Table 6: Availability of Other Agricultural and Livestock
Support Programs                                                     28
Table 7: Impacts on feeding practices                                30
Table 8: Impacts on access and use of livestock technology           31
Table 9: Impacts on animal health services                           32
Table 10: Impacts on availability on cattle extension services       32
Table 11: Impacts on probability of receiving information from
trainings                                                            33
Table 12: Impacts on number of animals                               36
Table 13: Impacts on milk production and total value                 37
Table 14: Impacts on food categories consumed in last 7 days         38
Table 15: Impacts on women decision making outcomes                  40
Table 16: Impacts on milk production and total value (female
household heads only)                             Error! Bookmark not d
Appendix Table J1  SDCP Yearly Expenditure by Category
(in thousands of KSH)
Appendix Table J2: Dairy Groups Inventory (Revised as of 2011-2012)

Exhibits

                                                                   Page

Exhibit 1: Recreation of Original Programme Targeting                12
Exhibit 2: Evaluation Treatment and Comparison Divisions and
Non-Evaluation Treatment Divisions                                   15
Exhibit 3: Balance Plot                                              27


Plain Language Summary

Programme overview

The dairy sector in Kenya is one of the largest and most developed in Sub-Saharan Africa, accounting for four percent of its gross domestic product. Despite high production volumes, the sector is still dominated by smallholder farmers who rely on livestock for income and food security. Dairy farmers face a number of barriers to increased profitability, including animal diseases, lack of access to artificial insemination and other veterinary services, high costs of improved technologies, such as silage equipment, inadequate access to markets, poor rural infrastructure and unsteady supply of quality animal fodder.

The Smallholder Dairy Commercialisation Programme (SDCP) was funded by the International Fund for Agricultural Development (IFAD) and implemented by the Government of Kenya from 2005-2015. It was designed to reach 600 dairy groups (24,000 smallholder dairy farmers) in nine counties. SDCP provided training to dairy farmers to build their enterprise, managerial and organisational skills. Aside from training, the programme also aimed to enhance dairy farming productivity and reduce production costs through demonstration, field days and grants. To strengthen market linkages, SDCP invested in improving road infrastructure and conducted additional training on milk-handling practices and value-added opportunities.

The programme identified three main areas where barriers to improving dairy income potentially operate: dairy group activities, household production and market intermediaries. Programme designers hypothesised that increasing net dairy income for smallholder farmers can occur through four primary contextual factors (1) increasing milk production; (2) increasing milk prices; (3) decreasing the costs of producing milk; and (4) decreasing the transaction costs of participation in input and output markets. They assumed that increased net income will lead to improved food security and increased participation by women and marginalised communities.

Impact evaluation overview

A key objective of agricultural extension is to increase farmers' knowledge about agricultural practices, which in turn could have an effect on productivity. Evaluating the impact of the SDCP can provide useful insights for the design of other agricultural extension programmes for smallholder farmers in developing countries. 3ie funded this grant under 3ie's Agricultural Innovation Thematic Window. The main evaluation questions were the following:

1. Does SDCP improve the well-being of dairy farmers through improved animal management, improved efficiency, increased production and productivity, and increase in farmer incomes and food security?

2. Are there differences in the participation in programme activities between female and male dairy farmers? Does the programme have sex-differentiated effects?

3. How does the process by which SDCP is implemented influence the effectiveness of the programme? Given this process, how do contextual factors affect programme success?

Methodology and identification strategy

The ex-post evaluation used a quasi-experimental design using propensity score matching. The sample of 2,500 famers was split equally between 1,250 SDCP beneficiaries and 1,250 non-programme participants. Key information was collected through a survey (conducted by Research Solutions Africa) and the programme causal chain, including the underlying assumptions, was mapped out. To capture key time-invariant characteristics, as well as retrospective information on dairy farming, the study included a short filter questionnaire at the beginning of the household survey for accurately matching households in the treatment group with households in the comparison group.

The study used qualitative data to explain the quantitative findings and understand the implementation of the programme. It compared accounts of local government officials with that of dairy farmers, by using semi-structured interviews and focus group discussions.

Main Findings

The findings suggest that the SDCP was successful in increasing milk production but had limited impacts on increasing either the quantity of milk sold or prices received by famers. Regarding intra-household bargaining power, SDCP farmer households were more likely to have women managing cash from the sale of milk relative to non-SDCP households. This was also found to be true for decisions relating to use of services such as artificial insemination, anthelmintic drugs, tick control, vaccination and curative treatments.

Qualitative research highlighted the challenges in programme implementation, such as those related to linking dairy groups to various service providers and limited knowledge among farmers to negotiate terms favourable to them.

We calculate that it would take approximately 4.74 years for the program to "break even," that is for the benefits to equal the costs, assuming the benefits of increased milk production remain constant across years. This estimate of the number of years to break even seems reasonable for although there likely were no benefits at the beginning of the programme (when upfront costs were associated with setting-up the SDCP), farmers likely began benefiting from the programme prior to 2016. Furthermore, to the extent farmers continue to employ the best practices advocated by the programme, the benefits may extend into future years.

Introduction

The International Initiative for Impact Evaluation (3ie) on behalf of the International Fund for Agricultural Development (IFAD) contracted the American Institutes for Research (AIR), in partnership with Lead Analytics and Research Solutions Africa to evaluate the Smallholder Dairy Commercialization Programme (SDCP) in Kenya.

The Smallholder Dairy Commercialisation Programme (SDCP), implemented by the Government of Kenya (GoK) from 2005 to 2015, was designed to reach Dairy Groups engaged in milk production in nine milk-producing counties. The overall goal of SDCP was to increase the income of poor rural households that depend substantially on production and trade of dairy products for their livelihoods. The SDCP was implemented through various interrelated components. First, SDCP provided training on organisational, managerial, and enterprise skills (e.g., bookkeeping, accounting, financial planning) to farmers. Second, the programme targeted household production by aiming to enhance dairy farming productivity and reduce production costs through grants, trainings, field days, and demonstrations. Lastly, SDCP aimed to strengthen market linkages for small-scale milk producers by improving road infrastructure and conducting additional trainings on milk handling practices and value-addition opportunities.

Many researchers, nongovernmental organizations, governments, and donors have long held the position that smallholder dairy can be a particularly effective mechanism for alleviating poverty and increasing food security in regions well-suited for dairy production, such as those located in western Kenya (Staal, et al., 1997; Thorpe, Muriuki, Omore, Owango, & Staal, 2000; Burke et al., 2007). While there had been some successes prior to 2006 in promoting the production and marketing of milk in these types of regions in Kenya, the SDCP was designed to address a number of barriers to increased profitability of smallholder dairy farming in the country, including relatively high transactions costs for production and marketing (Staal, et al., 1997 (World Development)), and under performance of dairy groups and cooperatives in reducing those transactions costs (Holloway et al., 2000 (Agricultural Economics, 23(3): 279-288); Atieno and Kanyinga, 2008). However, empirical evidence on the impacts of reducing transactions costs--either through analyses of household surveys or through rigorous impact assessments of specific projects--remains relatively limited. A few studies report results that would suggest that reducing transactions costs will improve farm productivity and incomes, primarily by increasing access to improved production methods and marketing information and by being able to link to private sector actors in the value chain. For instance, Burke et al. (2015) find that smallholders closer to electricity sources and with access to private sector value chain actors are more likely to be dairy producers and net sellers, and that "active" dairy cooperatives also induce greater participation by smallholders in dairy markets. Gelan and Muriithi (2015), and references cited therein, find that practices such as zero grazing and adoption of improved cows, led to increased milk production efficiency, while Nafula (2013) provides evidence of the importance of enhanced nutritional and feed practices on the health of dairy calves using a randomized control trial method. The SDCP project promotes all of these activities and also seeks to reduce marketing-related transactions costs.

Understanding the impacts of the SDCP is particularly valuable given the importance of the dairy sector in Kenya. The dairy sector in Kenya is one of the largest and most developed in Sub-Saharan Africa, accounting for 4% of the country's gross domestic product (GDP). Despite high production volumes, the dairy industry is dominated by smallholder farmers who rely on livestock for income and food security. However, smallholder dairy farmers face a number of barriers to increased profitability, including steady supplies of quality animal fodder and feed throughout the year; animal diseases; inadequate access to markets; artificial insemination (AI) services and other veterinary services; poor rural infrastructure; high costs of improved technologies, such as silage equipment; limited skills with which to bargain with input suppliers and output purchasers; limited knowledge to run their dairy activities as a commercially oriented enterprise, including maintaining animal health and providing high-quality milk; and dependency on traders and limited ability to maintain milk quality, putting downward pressure on farmgate milk prices.

While the project undertook a study of the programme in the form of household surveys at the end of the project in 2015, no formal impact assessment had previously been conducted. Specifically, the 2015 survey did not include households from "control" areas. Instead, the questionnaire included retrospective questions, going back many years. Thus, it is possible that findings from the 2015 survey might suffer from recall bias and from lack of data from households on which to construct a counterfactual. The study AIR and Lead Analytics have conducted is the first rigorous impact assessment designed to examine the effects of the SDCP. AIR and Lead Analytics designed the evaluation to address the knowledge gaps that related to project's impacts on smallholder dairy producers, including measures of production efficiency and profitability, as well as impacts of project activities aimed at ensuring that women and the resource-poor farmers benefitted from the project.

The study investigated programme impacts and implementation of the SDCP. Our main research question was whether SDCP improves the wellbeing of dairy farmers through improved animal management, increased production and productivity, improved efficiency (e.g., input and transaction cost reductions), and a rise in farmer incomes and food security. We also investigated whether there are differences in programme participation and effects by the gender of the farmer. Lastly, we looked at the process by which SDCP is implemented and how contextual factors may affect programme success.

We addressed these research questions through a mixed methods design. Quantitatively, we used a quasi-experimental approach using a matching ex-post design. Since programme implementation is complete, an ex-post analysis was appropriate to inform future scale up of similar efforts. Our design involved two steps of matching at the division and then household level. At the division level, we relied heavily on a two-step targeting approach. First, we used existing administrative information from the original programme targeting to identify similar nonprogramme areas that were not affected by the SDCP due to capacity constraints (observation-based targeting). Second, we discuss the filtered control communities with local experts and stakeholders to determine which of the preselected control areas were more similar to treatment communities at project inception (criteria-based targeting). One of the major benefits of the criteria-based targeting exercise is that, based upon the insight of the experts, we were able to exclude divisions where other policies or actions may have had differential effects over time, which is especially beneficial since this is an ex-post evaluation for a long-running programme. Furthermore, we matched treatment and comparison divisions within the same counties, which helps ensure that both study areas received the same level of support from their county over time, especially after devolution of power to the county, which occurred in Kenya in 2013 At the household level, to estimate programme impacts, we further used matching through our use of the doubly robust estimator, which combines regression and propensity score methods, to achieve some robustness to mis specification of the parametric models (Imbens & Wooldridge, 2009). Combining regression and weighting (with weights derived from the probability of being part of the program) can lead to additional robustness by removing the correlation between omitted covariates and by reducing the correlation between omitted and included variables. We complemented these analyses with qualitative research in the form of focus group discussions and key informant interviews.

Our findings indicate that the programme had statistically significant positive impacts on improved animal management, including grazing and feeding practices, keeping practices, and animal health services. These results suggest that farmers were receptive to programme education regarding the nutritional content and feeding practices, which may have a positive effect on milk production. The SDCP also led to an improvement in breeding services. Specifically, the SDCP increased the propensity of households monitoring their cattle on a regular basis and the use of Artificial Insemination (AI) services. The positive quantitative findings related to access to and use of AI are especially promising considering that many farmers discussed ongoing challenges with AI. We found that SDCP treatment farmers have improved access to extension visits, field days, and demonstrations, services which the programme provided. We also found an increase in the probability of receiving any information on specific aspects of the production process. The largest increase in probabilities was for receiving information on general livestock practices, milk processing and quality control, and fodder establishment and fresh milk marketing, although positive impacts are observed for almost all type of cattle-related topics, as well as topics related to enterprise skill. The results also suggest that the SDCP was successful in increasing milk production, but with more limited --though positive--impacts on increasing milk marketing and increasing milk prices received by smallholders.

The report is structured as follows. First, we describe the intervention and research hypotheses. Then we describe the context and the timeline for the project and evaluation. We describe the evaluation's and the policy's design, methods, and implementation. We then present the impact analysis followed by a discussion and recommendations for policy and practice.

Intervention, Theory of Change, and Research Hypotheses

Development of Intervention

During the 1990s, the monopolistic dairy cooperative structure in Kenya collapsed. While plagued by inefficiencies, the structure provided some support to dairy farmers. The collapse led to great distrust of new collectives despite the potential gains from collective action in the sector and due to economies of scale in linking smallholders to input suppliers and milk purchases (e.g., through bulk purchasing of fodder and bulk milk sales). At the outset of the project in 2006, the GoK was restructuring at least some of its legislation related to smallholder participation in dairy markets, with the aim of reducing costs that smallholders face in joining formal markets. One of the project's goals was to work with the government and relevant ministries to further enhance the policy and legislative environments for smallholders, primarily with respect to issues dealing with animal breeding.

The project was built on empirical evidence on smallholder dairy systems produced by the International Livestock Research Institute (ILRI), with support from Department for International Development (DFID). Given evidence on the level of dairy commercialization potential, smallholder dairy producers, poverty rates, and indicators of rural infrastructure, the project selected to work in nine districts in central and central-western Kenya. Smallholders dominate the production of milk in these districts, which also exhibit high-poverty rates. At the same time, smallholder dairy farmers face a number of barriers to increased profitability, including steady supplies of quality animal fodder and feed throughout the year; animal diseases; inadequate access to markets; AI services and other veterinary services; poor rural infrastructure; high costs of improved technologies, such as silage equipment; limited skills with which to bargain with input suppliers and output purchasers; limited knowledge to run their dairy activities as a commercially oriented enterprise, including maintaining animal health and providing high-quality milk; and depending on traders with limited ability to maintain milk quality, putting downward pressure on farmgate milk prices. Additionally, women play a key role in the smallholder dairy sector. Research showed that many female-headed households in these districts had dairy cows, and even in male-headed households, females controlled over 60% of the income from dairy activities. Women dairy farmers have been traditionally even further disadvantaged in terms of receiving extension advice and playing leadership roles in dairy groups to ensure their specific needs are addressed.

The SDCP--funded by IFAD, the GoK, and the local community, and implemented by the GoK from 2005 to 2015--was designed to address some of the constraints faced by smallholder dairy farmers. The overall goal of SDCP was to increase the income of poor rural households that depend substantially on production and trade of dairy products for their livelihoods. The SDCP was implemented through various interrelated components. First, the programme provided dairy group activities by training beneficiaries on organisational, managerial, and enterprise skills (e.g., bookkeeping, accounting, financial planning) to fully benefit from market-driven milk commercialisation. Further, capacity building of Dairy Groups was complemented by competitive access to investment grants for improving forage production and feed milling; milk bulking, chilling, and processing; and management and market information systems. Second, the programme targeted household production by aiming to enhance dairy farming productivity and to reduce production costs through trainings, field days, and demonstrations. Lastly, SDCP aimed to strengthen relationships with market intermediaries by enhancing market linkages for small-scale milk producers by improving road infrastructure and conducting additional trainings to beneficiaries on milk handling practices and value-addition opportunities.

The primary beneficiaries of the project were resource-poor smallholder dairy farmers, with an emphasis on ensuring women's participation in all project activities. The project intended to reach 24,000 smallholder dairy farming households with members participating in 600 dairy groups across nine milk-producing counties. IFAD and GoK determined the target of 600 dairy groups as a number large enough for an impact but still within capacity constraints. Based on dairy group inventories as of 2011 and 2012, the project ended up working with 15,535 smallholder dairy farming households with members from 505 dairy groups across the nine counties. In addition to resource-poor smallholders (85% of targeted beneficiaries), the project also targeted smallholders already engaging in more intensive production. Finally, the project also targeted 300 milk traders and 90 milk bar/milk processors to improve milk quality and contractual arrangements with buyers and sellers.

Theory of Change

The project first reviewed the substantial evidence on factors associated with low milk production, productivity per cow, and relatively low participation in milk markets with marked seasonal fluctuations. The project identified three main areas where impediments to improving dairy incomes operated: dairy group activities, household production, and market intermediaries. These three areas conform to three of the five project components, with support to policy and institutions and to project management comprising the fourth and fifth components.

Increasing dairy incomes for smallholders can occur through three primary channels: increasing milk production, increasing prices received for milk sold, and decreasing costs of producing and marketing milk. Here we focus on the first three components, as these components address all of the channels to varying degrees.

Component 4 (support to policy and institutions) concerns activities to shape national-level policies and regulations, primarily concerned with regulating AI, registering (improved) breed births, and milk-related phytosanitary regulations. To the extent that these have been adopted and implemented, they would affect all dairy cattle owners in Kenya, that is dairy cattle owners in both the treatment and comparison groups in this study. Since both groups would benefit from these policy and regulatory changes, it will be impossible to evaluate these impacts using the treatment and comparison households. Component 5 (support to project management) affects only treated households, so it is more of an IFAD-specific procedure than a separate activity. In fact, most development projects do not put project management as a separate component of activities.

In other words, all beneficiaries receive all five components, whereas comparison farmers would receive benefits from Component 4. Since beneficiaries receive all five components, isolating the effects is not feasible with the quantitative evaluation. However, since the main difference between treatment and comparison groups is found in Components 1-3, we focus on these components. These components are linked in that farmer level interventions (Component 2) would interact with how groups of farmers performed (Component 1), in order to be able to take advantage of greater market opportunities (Component 3).

Component 1: Organization and Enterprise Skills of Dairy Groups

Main barriers:

a. Disorganized groups with limited business/commercialization skills and knowledge

b. Limited or weak links with input suppliers and output purchasers, leading to missed opportunities to secure lower input prices or higher and more reliable output prices

c. Limited ability to disseminate relevant production advice to farmers, particularly disadvantaged groups, including women and resource-poor farmers

Main inputs to address these three barriers:

I1. Extensive training on group organization and management, business skills, development of enterprise plans, and preparation of business proposals that are then eligible for dairy enterprise grants.

I2. Training on establishing and maintaining links with input and service providers and output purchasers

I3. Linking groups to advisory and extension systems

I4. Reaching out to women and resource-poor farmers

Main intended outcomes:

O1. Dairy groups with financially viable and sustainable business plans, and the ability to develop and successfully obtain external grant funds (due mainly to I1 input). The main assumption here is that dairy group members were able to successfully understand the training materials and translate that knowledge into business plans and proposal writing. Another key assumption is that there were real business opportunities that relatively small and resource-poor dairy groups could take advantage of.

O2. Transactions costs and input costs reduced, and output prices potentially increased (due mainly to I2 and to some extent I1). The main assumption is that access to milk markets and market players can be increased by knowledge gained in training. It should be noted the project was undertaken in areas determined to be high-potential commercialization areas, which was intended to limit the impact of other barriers such as geographic isolation and very high transportation costs.

O3. Dairy group members, including women and resource-poor farmers, increase knowledge on dairy production and markets, leading to higher and more stable milk production and to lower transactions costs of participating in markets (due mainly to I3 and to some extent I1).

Component 2: Technical Support to Dairy Producers

Main barriers:

d. Poor breeds of dairy cows

e. Lack of adoption of improved management practices, such as zero-grazing and keeping milk records, and producing hay and silage

f. Poor animal nutrition, and pronounced seasonality in fodder and feed use

g. Animal diseases

Main inputs to address these four barriers:

I1. Trainings and demonstrations to disseminate information about benefits to improving breeds through AI and benefits from animal registration, improved husbandry and dairy enterprise management practices, improved fodder production and management and supplemental feed use, and animal diseases and disease management

I2. Establishing community AI schemes

I3. Establishing a revolving community-based animal health fund

I4. Training community resource persons to aide in disseminating information and linking farmers to relevant resources

Main intended outcomes:

O1. Better-bred dairy cows, leading to higher milk per cow and total output (I1 and I2). Assumes that quality AI seed is available, and that farmers see the value in improving breeds, which provides delayed benefits.

O2. Greater production and better management of fodder and feed, leading to lower costs of milk production and greater stability in milk output throughout the year (I1 and I4). Assumes limited opportunity costs associated with fodder being put to other uses, and that options for extending forage availability throughout the year (e.g., storage) are profitable.

O3. Healthier cows producing more milk (I1, I3 and I4). Assumes that information on disease management and access to revolving funds are sufficient to address substantial issues with tick-borne disease control.

O4. Better overall management practices, leading to greater production and potentially lower costs of production (I1 and I4). Assumes that training materials contain relevant and understandable information that farmers can apply in practice. Also assumes relatively low opportunity costs, investment costs, and input costs, or alternatively, that switching to new systems is actually profitable.

O5. More milk to meet household needs and to participate year-round in the milk market (I1, enterprise management and fodder production and management; and to some extent I4). Assumes that market linkages are established (e.g., through the dairy groups) and transactions costs are sufficiently lowered.

Component 3: Development of Milk Marketing Chains

Main barriers:

h. Disorganized markets

i. Market traders with limited skills to maintain high-quality milk (e.g., hygiene standards)

j. Market traders with limited enterprise management skills

k. Limited generation and dissemination of milk market information

l. Limited market infrastructure (e.g., bulk milk facilities)

m. Limited production and marketing of dairy goat milk

n. Inefficient contractual arrangements between dairy groups, milk collectors, cooling centres, and processors

o. Limited access by smallholders to rural finance

Main inputs to address these eight barriers:

I1. Developing a low-cost market information system and strengthening the Dairy Information Centre.

I2. Linking activities between smallholders and rural finance operators

I3. Capacity building for milk marketing groups

I4. Pilot testing school milk programmes as an opportunity to expand milk marketing.

I5. Performing a study on milk marketing opportunities

I6. Trainings on hygienic milk handling

I7. Establishing milk bulking facilities and other infrastructure (e.g., cooling facilities)

I8. Training and demonstrations on dairy goat production and marketing, as well as procurement and distribution of dairy goats to resource-poor smallholders

I9. Developing new and improved contractual arrangements

Main intended outcomes:

O1. Reduced transactions costs of participating in the market (I1, I2, and I7). Assumes that farmers and other market players can access new information sources, and that information is relevant and understandable.

O2. Increased size of the market (I3, I4, and I5). Assumes there is scope for expanding the milk market. External evidence suggests there is such scope.

O3. Increased quality of milk in the market, increasing value added and potentially leading to higher prices for smallholders (I6 and I7). Assumes that knowledge is disseminated in a practical and useful way, and more importantly, that sufficient access to technologies and infrastructure exist throughout the entire value chain.

O4. More effective contractual arrangements increases the quantity of milk in the market throughout the year. Assumes that contract terms are currently inefficient and there is scope to make improvements.

O5. Increased participation in the dairy goat milk market by women and resource-poor smallholders (I8).

From Outcomes to Impacts: As noted above, the primary impact is expected to be higher net milk incomes, though increased production and productivity per animal, reduced input costs, reduced transactions costs, and potentially higher farmgate milk prices. The outcomes clearly map to this impact. The second expected impact is greater participation by women and resource-poor farmers in milk markets and as leaders in dairy groups, which follow from: Component 1, Outcome 3 and from Component 3, Outcome 2. While we can analyse this impact through heterogeneous effects analysis, the greater participation by women and resource-poor farmers is an impact in and of itself, since the project aims to ensure that women and the most vulnerable were indeed included. The third expected impact is increased food security. In part, this is related to higher net dairy incomes, and thus related to two of the four pillars of food security, access, and availability. Additionally, greater stability of milk production and sales throughout the year is related to the stability pillar. Finally, trainings on hygiene increases milk safety for consumers, which is related to the utilisation pillar.

Primary Outcomes and Impacts of Interest

The programme's key intended outcomes included more knowledgeable dairy farmers, healthier and better-bred milk cows, reduced seasonality in milk production, increased participation and labour opportunities in milk markets (generating the impact of higher net dairy incomes for smallholder dairy farmers); well-developed business plans and project proposals, and establishment of reliable trade relations with input suppliers and output purchasers for dairy groups (generating the impact of commercially viable and sustainable dairy organizations); and increased networks and higher quality milk for milk traders (generating the impact of greater value addition).

In this study we investigate programme impacts and implementation of the SDCP. Our main research question is the following:

1. Does SDCP improve the wellbeing of dairy farmers through improved animal management (e.g., better husbandry practices), increased production and productivity (e.g., litres/cow/day), improved efficiency (e.g., input and transaction cost reductions), and increased incomes for farmers (e.g., gross margins and higher milk prices)? Does this lead to increased income from dairy farming and ultimately improved food security?

To better understand our primary research question, we consider the following secondary questions:

1. Are there differences in programme participation by the sex of the farmers? Does the programme have differential effects by the sex of the farmers?

2. How does the process by which SDCP is implemented influence the effectiveness of the programme? Given this process, how do contextual factors affect programme success?

The first secondary question is critical since women play a key role in dairy production in Kenya. However, they face a number of constraints that may alter the extent to which they benefit from the programme, such as owning smaller farms, which affects their access to credit using land as collateral, or being less educated, which limits their access to technical information for enhancing production. The second secondary question is especially important because the impacts of a programme are ultimately a function of the manner in which it is implemented. Understanding the implementation of this programme is a critical aspect of this evaluation because it largely determines the type of dairy farmer the programme reaches. There are clear implications if the programme does not reach certain segments of the population (e.g., women, poorer farmers, certain locations, etc.). The process can also lead to variation in programme effectiveness in that it may vary by contextual factors, such as the population density of a region or the type of farming.

Context

Agriculture is one of the leading sectors in Kenya, employing 70% of the rural population and accounting for 25% of the GDP. More specifically, the dairy sector in Kenya is one of the largest and most developed in Sub-Saharan Africa, accounting for 4% of the country's GDP. Dairy farms are concentrated in the highlands and former provinces of the Rift Valley in the central and eastern parts of Kenya. There are more than 3.5 million heads of purebred Friesian--Holstein, Ayrshire, Guernsey, and Jersey cattle and their crosses, with a total yearly production of about 2 billion litres of milk (Muriaki, 2003). Despite these high production volumes, the dairy industry is dominated by smallholder farmers. Approximately 80% of people engaged in agricultural activities are smallholder farmers, producing three quarters of the agricultural output in farms that commonly have an area of less than 3 ha. Smallholders produce over 50% of the predominant crops in the country--including maize, coffee, and tea--and 80% of milk and beef products. There are over 1 million smallholder farmers who depend on dairy farming for their livelihood. Less than 15% of marketed milk flows through milk processors (Thorpe et al. 2000), with the rest being sold as raw milk through direct sales to consumers by farm households, dairy cooperative societies, and individual traders.

As with many other countries in the sub-Saharan region, smallholder dairy farmers in Kenya rely on livestock for income and food security. However, milk production is regularly threatened by inadequate access to markets, animal diseases, poor quality and unstable supplies of animal food and feeds with corresponding decreases in market milk supply, poor rural infrastructure, and inadequate access to AI services and other veterinary services. More importantly, small dairy farmers face a series of constraints that prevent them from effectively commercializing their milk, including large seasonal fluctuations in milk output and prices, poor rural infrastructure (roads and electricity), limited skills with which to bargain with input suppliers and output purchasers, lack of management and business skills, and inefficiencies in the post-harvest segment of the dairy value chain. Furthermore, the dependency on traders who have limited ability to maintain milk quality puts downward pressure on farmgate milk prices. The inability of farmers to avoid and respond to these risks and constraints associated with milk production is partly explained by the lack of access to productive inputs and improved technologies, such as silage equipment, and also to a lack of knowledge on how to maintain animal health and increase milk productivity through better production practices.

Programme Targeting

In 2005, IFAD and the GoK commissioned a study by the ILRI to lead the initial targeting of the programme. Three key indicators were initially considered to determine the programme area: milk production and production potential, incidence of rural poverty, and market access. Milk production was measured as litres of milk produced per square kilometre per year, an indicator that reflects well the productivity by individual cow; number of animals in a given area; and percentage of lactating animals. Poverty incidence was drawn from data constructed by the Central Bureau of Statistics in 2003. Finally, market access was defined as distance from farms to main milk cooling centres, as well as distance from farms to main urban centres. Upon realization that the three key indicators were too restrictive in terms of potential programme locations, the programme targeting instead focused on the two key indicators of milk production and production potential and incidence of rural poverty. For these indicators, thresholds were defined to target the initial participating districts/counties. First, the programme selected districts with high (>90,000 litres/km2/year) and medium 60,000 (<90,000 litres/km2/year) milk densities. Second, they chose districts with a poverty rate of at least 46%.

To select beneficiaries, the project relied on "Targeting Pro-Poor Investment in the Dairy Sub-Sector," a report produced by ILRI in March 2005, which contained rich analysis on the smallholder dairy subsector. Using additional information from dairy farmer surveys, the project first selected nine districts out of the 25 milk-producing districts in Kenya. Then, because not all divisions/subcounties (the next administrative unit after districts/counties) within the nine selected districts met the targeting criteria, SDCP focused operations on only 27 divisions (out of a total of 53 divisions in the nine districts). These divisions corresponded to Dairy Commercialization Areas (DCA), and the GoK selected locations and sublocations within the 27 DCAs, with the aim of identifying 600 farmer groups. The project attempted to work with already existing farmer groups that had at least some activities focused on dairy farming. Each DCA had 500-800 dairy farmers, which was considered a manageable number of farmers to be handled by divisional government staff.

Once the geographical units for the programme were selected, SDCP operationalized programme activities in the field through dairy groups. Dairy groups are formally registered as self-help groups, cooperative societies, or common interest groups that have a common interest in dairy farming. On average, each SDCP dairy group has 30 dairy farmers. At the end of 2015, SDCP had worked with 527 dairy groups, with more than 16,000 members, of which 60% were women. The selection of dairy groups within programme areas was done using a participatory selection process that gave priority to resource-poor dairy farmers, including women and youth. More specifically, SDCP chose individual dairy groups based on the following observable characteristics: (i) farmers had a maximum of two cows; (b) farms had an average of 2.5 acres (0.91 hectares (ha)) of productive land; and (c) the group was not marketing more than 30% of its milk before joining the programme. The programme targeted the dairy group; thus, all participating farmers in the dairy group were eligible. Lastly, the identification of existing dairy groups in each district was facilitated by the fact that in Kenya, all common interest groups need to be registered with the national and local governments. While IFAD and GoK targeted 600 dairy groups, numerous other dairy groups existed in the districts who were not treated due to limited implementation.

Study Site Selection

To select the sites for the study, we replicated the targeting process that ILRI conducted in 2005. This replication process involved the use of observation-based and criteria-based targeting, as proposed by Ouma and colleagues (2007). Observation-based targeting involved determining where the SDCP was adopted, plotting those sites on a map, and identifying the common characteristics the sites shared. To conduct the observation-based targeting, the AIR and Lead Analytics team worked with contacts at ILRI to examine the data that was originally used for the targeting exercise. We engaged with Ms. Pamela Ochungo, a geographic information systems analyst from Kenya who conducted the original targeting exercise on behalf of ILRI, to recreate the maps used for the original targeting of the programme. Exhibit 1 shows the areas that met the threshold for high and medium milk production and poverty levels.

We used this information to determine a group of potential nonprogramme areas that had characteristics similar to SDCP areas before the programme started. Because IFAD and GoK determined the target of 600 dairy groups in part due to capacity constraints, there were additional nontreated areas similar to the treated areas that could serve as a comparison group. After recreating the original targeting map, we performed propensity score matching at the level of the division by using the original data to match treatment divisions to potential comparison divisions within each county. We calculated from the original ILRI data, a propensity score for the division based on the 2005 values of the variables of milk density, proportion of poor households, proportion of Grade cows in dairy households, proportion of dairy households, and travel distances to urban and cooling centres. We matched similar comparison divisions to treatment divisions within the same county. That is, we simultaneously selected treatment and comparison divisions for the study areas on the basis of their being the most comparable pairs according to observed 2005 data. Restricting to the same county was important because over the programme implementation years, there was variability in county-level support for dairy farming after devolution. Thus, to mitigate specific influences at the county level, we needed to compare treatment divisions with comparison divisions in the same county.

In addition to the results of the matching exercise, we used criteria-based targeting to refine the selection of the comparison sites. Criteria-based targeting is based on the opinion of experts, who determine to what extent nontargeted areas could have been chosen for the programme. Experts relied on historic variables that are likely to be associated with the uptake of the intervention in 2005, such as climate, market access, and other agro-ecological conditions. Mr. Luke Kessei, the Ministry Desk Officer for the SDCP from 2005 to 2015, played a key role in designing and implementing the programme and is now an independent consultant. He, along with the SDCP technical team, served as our experts for the criteria-based targeting. In October 2016, we met with IFAD and SDCP officers in Nairobi and Nakuru. Through the meetings with the SDCP technical team, we discussed at a high level which of the potential comparison areas might serve as the best counterfactual in terms of similarities to SDCP treatment divisions.

We presented to the team a list of the nine programme counties along with all the potential divisions (all the beneficiary divisions which could serve as study treatment areas and the non-beneficiary divisions which could serve as comparison areas). With the help of our experts, we narrowed down the list of SDCP treatment and comparison divisions to include in the evaluation. In these discussions, the experts considered eight criteria: (1) whether the evaluation included at least one county from each of the three geographic programme clusters (1); (2) the variation in county-level support after devolution, where areas with high levels of support were excluded; (3) the variation in support provided by other dairy-focused programmes, where areas with high levels of support were excluded; (4) the violence that happened in certain areas during the 2007 election, which resulted in factories being destroyed, livestock being killed, and some areas receiving government support in response to the violence. Areas that had very high levels of violence were excluded; (5) the similarities in farmer composition, since some potential comparison divisions had large-scale farmers which would not serve as a valid counterfactual to the smallholder farmers participating in the SDCP. Areas where the farmers tended to be larger-scale were excluded; (6) the geographic proximity of the potential comparison areas to the SDCP treatment areas, since comparison areas that neighboured the SDCP areas would be at higher risk of contamination from spillover effects. Comparison areas that closely neighboured SDCP treatment areas were excluded; (7) the focus of the farmers in the area, since some farmers were focused more on tea farming than on dairy farming. Areas where the farmers tended to be mostly tea farmers were excluded; and (8) the feasibility of data collection requirements in very remote divisions, which were excluded. One of the major benefits of the criteria-based targeting exercise is that, based upon the insight of the experts, we were able to exclude divisions where other policies or actions may have had differential effects over time, which is especially beneficial since this is an ex-post evaluation for a long-running programme. Specifically, we excluded divisions where county level policies provided a high level of support, where other dairy programmes provided a high level of support, and where 2007 election violence was extreme.

We cross-referenced the sites recommended by the experts from the criteria-based targeting exercise to the results of the division-level propensity score matching from the observation-based targeting exercise to arrive at the final sample. The results of the two-part targeting exercise are detailed in Table 1 and further visualized in Exhibit 2.

Since the evaluation sampled from all three geographic clusters of programme implementation, the results should be generally representative of the SDCP population. That is, the results should apply to the average dairy farmer who is located in areas with medium to high levels of milk production and poverty. However, because we excluded certain divisions that would have posed difficulties in forming our study group, our sample lacks representativeness on certain dimensions. Specifically, the study did not include areas where there was a high level of support provided by other dairy-focused programmes, where the aftermath of the 2007 election was severe, where large-scale farmers were more common than smallholders, where farmers were less focused on dairy farming, and where farmers were located far from each other. Nevertheless, the observation-based and criteria-based targeting exercises ensured that we had comparison areas that are similar to the treated areas. However, there are some limitations to the external validity of the study sites. Specifically, as indicated by the upper left corner of Exhibit 1, implementation of the SDCP covered districts/counties in the western region of Kenya. Thus, the results do not necessarily extend to other regions of Kenya where dairy farming is a less central focus of smallholder farmers. However, lessons may well be drawn for other East African countries with similar ecological and socioeconomic characteristics as those found in western Kenya, including those in the central highlands of Kenya, as well as the highlands of Uganda, Rwanda, Tanzania, and to a lesser extent, Ethiopia (Herrero et al. 2014, Global Environmental Change, and references cited therein).

Timeline

The following timeline depicts the stages of implementation and evaluation. As an ex-post evaluation, all evaluation activities occurred after implementation of the program.

Evaluation: Design, Methods, and Implementation

IRB

AIR is registered with Office of Human Research Protection as a research institution (IORG0000260) and conducts research under its own Federal wide Assurance (FWA00003952). AIR's Institutional Review Board (IRB) (IRB00000436) reviewed our procedures for minimizing the risks to participants, along with the instruments and protocols. To ensure ethical research, RSA read a consent statement that explained the purposes of the research and the expected duration of the subject's participation, and that described the procedures to be followed.

Quantitative Design

It was necessary to establish a clear counterfactual to conduct a valid assessment of the impact of SDCP on smallholder farmers. To address the question of what would have happened to programme participants had they not received the intervention, we used a rigorous quasi-experimental methodology. An experimental design was not possible because of the ex-post nature of the evaluation. However, estimating programme impacts by comparing a treatment group with a nonexperimental comparison group may be biased because participants self-select into the programme or implementing partners specifically target those beneficiaries that are more likely to experience the largest programme impacts. Within the context section above, we describe how the SDCP targeted programme beneficiaries and how we used that information along with observation-based and criteria-based targeting to select the study sites. That is, we first, used existing administrative information from the original programme targeting to identify (through propensity score matching at the division level) similar nonprogramme areas that were not affected by the SDCP due to capacity constraints (observation-based targeting). Second, we discussed the filtered control communities with local experts and stakeholders to determine which of the preselected control areas were more similar to treatment communities at project inception (criteria-based targeting).

After matching comparison areas to treatment areas for the study sites, RSA, with oversight from Lead Analytics, conducted a comprehensive dairy survey to a sample of 2,562 dairy farmers that covered key information needed to map out the causal chain among inputs, activities, outputs, outcomes, and impacts as well as the underlying assumptions of the SDCP. The sample of 2,562 was split between 1,297 SDCP beneficiaries (from 95 dairy groups) and 1,265 matched comparison farmers (from 89 dairy groups).

To capture key time-invariant characteristics and retrospective information on dairy farming, we included a short filter questionnaire at the beginning of the household survey to improve the matching process between each one of households in the treatment group to a similar farm household in a comparison area. That is, only farm households that were eligible as potential comparisons based on predetermined variables were part of the sample. The idea behind the filter questionnaire is to mimic the selection mechanism that SDCP staff used when defining the dairy groups that later joined the programme.

The identification strategy proposed to estimate the causal effects of the programme relies on the doubly robust estimator developed by Robins and Rotnitzky (1995); Robins, Rotnitzky, and Lue Ping Zhao (1995); and van der Laan and Robins (2003). At the household level, the proposed approach combines regression and propensity score matching methods in a three-step approach to estimate treatment effects. In the first step, a treatment model is defined that explains the probability of programme participation. From this step, inverse-probability weights are derived from the estimated propensity score. Second, using the estimated inverse-probability weights, weighted regression models are fit for the outcome equation for each treatment level and obtained the treatment-specific predicted outcomes for each subject. Lastly, means of the treatment-specific predicted outcomes are computed and the difference of these averages provides the estimate of the average treatment effect of the programme. Intuitively, weighting can be interpreted as removing the correlation between the treatment condition and other covariates that may be correlated with treatment, and regression as removing the direct effect of such variables on the outcomes of interest (Imbens & Wooldridge, 2009).

This approach assumes that programme participation is exogenous to potential outcomes conditional on observable characteristics--that is, that there is no selection bias due to unobserved characteristics and that the observable characteristics we capture determine programme participation. Due to the unobservable nature of these potential additional characteristics, this assumption is untestable. Nevertheless, we employ a series of strategies to reduce the potential threat of the impact estimates being driven by unobserved characteristics of programme participants. Specifically, in addition to replicating the division selection process that was conducted to determine programme placement, we use a filter questionnaire to replicate the selection of potentially eligible dairy groups, and to collect numerous covariates as controls that are good predictors of programme participation. Several authors have argued that social programmes can be evaluated using matching methods, as long as there is access to a rich set of variables that determine programme participation, and that the nonexperimental comparison group is drawn from the same local region as participants (Heckman, Ichimura, & Todd, 1997; Heckman, Ichimura, Smith, & Todd, 1998). We are confident that our proposed empirical strategy will allow us to estimate the causal effect of the SDCP on smallholder dairy farmers.

Sample Size Determination

We determined that a sample of 2,500 farmers, split evenly between the treatment and comparison groups, would be sufficiently large to detect meaningful programme effects. To calculate the sample size, we conducted a number of power analyses based on existing farmlevel data for SDCP beneficiaries that were collected as part of a 2014 survey commissioned by SDCP to Capital Guardian Consulting. These calculations are described in more detail in Appendix E. The calculations account for the fact that the dairy group serves as the clustering variable. We proposed to collect information for 154 dairy groups (77 treated and 77 comparison). For each dairy group, we proposed to randomly select 15 households. In practice, we conducted a full survey instrument to the sample of 2,562 observations (1,297 treatment and 1,265 comparison dairy farm households), coming from 183 dairy groups (95 treatment and 89 control). In the survey, we collected key information to map out the causal chain among inputs, activities, outputs, outcomes, and impacts, as well as the underlying assumptions.

Sampling Design

To increase comparability beyond the efforts we took for the study site selection, we included a short filter questionnaire at the beginning of the household survey to capture key time-invariant characteristics and retrospective information on dairy farming to improve the matching process between each one of households in the treatment group to a similar farm household in a comparison area. We included only dairy farmers who met characteristics that should have increased comparability between the treatment and comparison groups. We have included the Filter questionnaire in Appendix C. The filter questionnaire ensured that only those potential comparison group farmers who were most similar to the treated farmers remained in the final sample. Furthermore, this procedure represented an efficient use of project funds and helped to reduce the time burden placed on the farmers. By including a filter questionnaire that eliminates dissimilar farmers, we ensured that project resources spent on the full household survey were allocated to those farmers most relevant for inclusion in the study. Additionally, these procedures reduce the time burden associated with the survey, since we administered the survey to only relevant farmers.

Data Collection

AIR, in conjunction with LEAD Analytics, designed the quantitative questionnaire, adapting questionnaires used by ILRI in Tanzania and the World Bank Living Standards Measurement Study-Integrated Surveys on Agriculture in Malawi. The questionnaire was designed to collect detailed data about milk production, cost, and sales to generate information on net milk income and milk sales, which are two primary outcomes of interest to assess project impact. The project also aimed to reduce seasonality of milk production, so that net incomes would be higher and less variable throughout the year. Thus, the questionnaire also collected data on practices, such as second-season fodder grass production, associated with less pronounced seasonality in milk production. Finally, the hoped-for impacts include increased food security. In part, lower seasonality should contribute to more smooth consumption patterns throughout the year, reducing or eliminating the lean season. Second, while a full-scale consumption module was not included, we did include a module to capture dietary diversity, based on recent recall data.

Building on evidence from the literature, the questionnaire also included sections to recover information on the most important control variables at the household level, in order to improve precision of estimating project impact. These included basic household demographics and wealth variables; landholdings; and access to extension and other sources of information, density of social networks, etc.

Importantly, we also designed a dairy group questionnaire. The functioning of dairy groups (i.e., structure, conduct, and performance) is likely to have a strong impact on the ability of households to benefit from project activities, many of which were carried out through the dairy group leadership. Indicators of dairy group performance can serve as controls and can also provide valuable additional insights to feed into future project designs. The dairy group questionnaire also included a module on the history of presence of other development projects in addition to SDCP, which could prove to be useful control information, as well as basic information on community characteristics. In control villages, where no dairy groups are currently functioning, the community-level questions were addressed to village leaders. Appendix C presents the household and dairy group questionnaires.

The paper questionnaire was translated to tablet, primarily by our local survey partners, RSA, with inputs from LEAD Analytics during the training and piloting phases. The questionnaires were conducted in the field by RSA between November 15 and December 20, 2016. This time frame corresponds to the secondary rainy season in Kenya, which happens for a few weeks in November and December and is followed by a dry season of hot weather through March. The heaviest rainy season in Kenya occurs in late-April, May and early June. The survey time period closely corresponds to the end of harvest period for the main rainy season. (3) A Lead Analytics' field manager oversaw the training and piloting of data collection and, after coordinating with the other AIR and Lead Analytics team members, provided daily feedback to the enumerators and survey programmers throughout both processes.

Strategies to Avoid Biases

Training for both our quantitative and qualitative data collection was key to ensuring that interviewers understood the study, the interview protocols, interviewing techniques, and the importance of understanding questions exactly as written and recording responses exactly as stated--all of which improve data quality and reduce bias. In addition, the training covered administering and obtaining consent from every participant, to ensure compliance with AIR's IRB. The training covered the following topics:

* Overview of SDCP

* Overview of AIR's evaluation of SDCP

* Ethics, consent, and confidentiality

* Basics of qualitative research (interview protocols and observations)

* Professional conduct

* Data security

* In-depth practice of all protocols, including translations and role playing

* Team debriefs and protocol revisions

* Field plans and logistics

We further reduced biases by choosing comparison areas, as opposed to comparison households. The use of comparison areas helped overcome concerns about not capturing the true programme impacts due to spillover effects. If farmers in the comparison areas are outside the SDCP's catchment area, the benefits would be less likely to flow to the comparison areas. We investigated the validity of this requirement through the farm-level surveys by asking farmers in comparison areas about how much they know about IFAD's SDCP and other agricultural development programmes. Since numerous development and other agricultural programmes have been implemented in Kenya, it is important that the comparison and treatment groups have the same level of exposure to these programmes over time, with the exception of SDCP. Many of these other development and agricultural programmes are implemented at a county level. By choosing comparison areas that are within the same counties as the treated DCAs, we can account for exposure to other dairy programmes. Although we chose comparison areas within the same counties, we made sure not to choose comparison divisions that directly neighboured a treatment division. Ensuring that treated and comparison areas are sufficiently spaced apart helps to minimize the risk of spillovers.

An obvious concern when evaluating the programme is that due to a sense of reciprocity (i.e., Hawthorne Effect), respondents in the treatment group may offer answers that they believe the research team seeks. We structured the questionnaire in such a way that the questions related to the programme intervention are nonconspicuous and occur after questions related to the measurement of important outcomes, so as to delay activation of potential Hawthorne effects. In addition to that, we do not believe this evaluation exhibited John Henry Effects where nontreated farmers react to overcome the disadvantage of being in the control group, because we do not expect comparison farmers to know much about the SDCP, since they will be drawn from different geographical areas.

Qualitative Design

Addressing the research questions required a combination of qualitative and quantitative methods. In qualitative research, questions--and the responses they elicit--tend to be discursive and descriptive, while the analysis privileges explanation and interpretation over quantification. In general, qualitative approaches allow researchers to explore and understand the experiences, opinions, and perspectives of their informants in greater depth than that offered by quantitative approaches. In turn, the use of qualitative approaches entails sacrifices in terms of generalizability and comparability--areas in which quantitative methods excel because of their use of large and probabilistic samples. Samples chosen for qualitative studies are often nonrandomized--or "purposively" selected--and always smaller: "There is growing evidence that 10-20 knowledgeable people are enough to uncover and understand the core categories in any well-defined cultural domain or study of lived experience" (Bernard, 2011, p. 154).

We drew from qualitative methods to augment the quantitative surveys by capturing interaction among complex and changing contextual factors that could influence the impact of the SDCP and evaluated programme fidelity. The process component also aimed to assess gaps in implementation that may have affected impacts and how dairy farmers may have changed their practices based on what they learned with regard to Components 1-3. The contextual information we obtained through the qualitative information helped to clarify how the programme impacted individuals, thus contributing to the transferability of study findings to other settings (i.e., external validity).

Design

We conducted semistructured interviews with key informants directly involved in the programme at the national, county, and sub-county levels. These discussions aimed to clarify uptake of the programme and the coordination of county departments of agriculture with the farmers. Second, we conducted key informant interviews with private service providers who are unrelated to the programme to ascertain how the dairy sector functions outside of the programme and public sector.

Qualitative Sample

We conducted 12 focus groups using semistructured protocols with 6-8 farmers each from 12 dairy groups across two counties. We included two counties with three SDCP administrative areas each in the qualitative sample, just in case there were any notable regional or within-county differences by administrative area. We included dairy groups with varying proportions of male and female farmers. The focus group sample was half male and half female, and we separated the groups by male and female farmers to capture a better understanding of experiences by gender. Finally, we included groups who were considered to be high performing and low performing based on consultations with stakeholders familiar with the dairy groups.

We structured the questions to understand whether all of the groups engaged similarly with the various components, or whether and how some of the components were stronger for some groups than others. To compare these data with how groups function in the absence of the programme, we also conducted four focus group discussions (two male, two female) with farmers in comparison areas. We aimed to cover all components with some piece of the qualitative work (Table 2).

Methods

To ensure reliability of the findings, the research team employed several analytic methods to systematize the data review and coding: (a) content coding, (b) comparison of findings among researchers, and (c) grounded theory to guide analysis using the qualitative software programme (NVivo qualitative data analysis software, QSR International Pty Ltd., Version 10, 2012).

The first step in analysing qualitative data is to develop a coding structure that helps to systematically categorize information. Researchers "open code" data to identify primary categories of interest. These categories form the basis for the coding structure that the team uses to categorize raw data from interviews and focus group discussions into the primary findings. While categorizing the data, researchers utilized grounded theory (Glaser & Strauss, 2009) to deduct new themes from the findings, rather than testing an existing theory. Combining the use of grounded theory with a rigorous impact evaluation design enabled us to triangulate the research findings. The quantitative research served to test predefined hypotheses, and the qualitative research enabled us to ground new ideas on why the programme positively influenced some but not all outcomes of interest among respondents' real-life experiences.

After coding, the research team attempted to quantify the data, where applicable. This method helped to characterize the prevalence of responses to deduce which themes were common and which were outliers. It is important to note, however, qualitative data are not best analysed using a systematic count of opinions.

Implementation: Data Collection and Strategies to Avoid Biases

In general, qualitative approaches allow researchers to explore and understand the experiences, opinions, and perspectives of their informants in greater depth than that offered by quantitative approaches. The use of qualitative approaches entails sacrifices in terms of generalizability and comparability--areas in which quantitative methods excel because of their use of large and probabilistic samples. Interviewers were clear on these limitations, as well as their neutral role that aimed to limit courtesy and social desirability bias, a situation in which a respondent gives an answer that he or she feels the interviewer wants to hear.

Data collection occurred over a period of 2 weeks, between February 27 and March 10, 2017. The evaluation team trained data collectors before they went into the field. We held a plenary training to introduce participants to the study, the supervision process, logistics, interviewing techniques, and the interview and observation protocols, as well as to conduct mock interviews. During training, the research team improved the items on the interview protocols by incorporating the input from local interviewers. During the initial days in the field, interviewers noted challenges in administering the surveys and interview protocols, which we discussed during debriefings during the first phase of fieldwork.

The team collected data using notes and digital recordings. All of the interviews and focus group discussions were transcribed. The team ensured complete anonymization and protection of confidentiality for research participants.

Limitations of the Study

As noted before, the methodology employed with the quantitative design assumes that there is no selection bias due to unobserved characteristics and that the observable characteristics we capture determine programme participation. However, due to the unobservable nature of these potential additional characteristics, this assumption is untestable. Additionally, while our qualitative research provided insights into the uptake rates of the programme, we were not able to determine these rates quantitatively, because tracking down targeted farmers from 2006 was not feasible. This limitation prevented us from analysing what impact, if any, the programme had on the likelihood of a dairy farmer in 2006 remaining one to the present day.

Programme or Policy: Design, Methods, and Implementation

The SDCP is a joint programme between the GoK and IFAD. The Programme commenced in July 2006 and will be complete in September 2019. The SDCP was designed to reach Dairy Groups engaged in milk production in nine milk producing counties. The overall goal of SDCP was to increase the income of poor rural households that depend substantially on production and trade of dairy products for their livelihoods. The SDCP was implemented through various inter-related components. First SDCP provided training on organisational, managerial, and enterprise skills (e.g., bookkeeping, accounting, financial planning) to farmers. Second, the programme targeted household production by aiming to enhance dairy farming productivity and reduce production costs through grants, trainings, field days, and demonstrations. Lastly, SDCP aimed to strengthen market linkages for small-scale milk producers by improving road infrastructure and conducting additional trainings to on milk handling practices and value addition opportunities. The project operated through Dairy Groups and was intended to reach 24,000 smallholder dairy farming households with members participating in 600 Dairy Groups across nine milk producing counties.

Since the implementation began in 2006, we did not have a monitoring system to track implementation rollout. Similarly, since the evaluation was an ex-post evaluation, we did not provide incentives for participation in the study groups. For the quantitative data collection and the Key Informant Interviews, we did not provide any compensation or incentives for completion of the survey or interviews, respectively. For participation in the qualitative focus group discussions we provided a transport allowance and snack.

Impact Analysis and Results of Key Evaluation Questions

Descriptive Statistics and Quality of Counterfactual

In this section, we present some of the key explanatory variables collected for the study. These variables help us to understand the context in which the SDCP was implemented and serve as controls in the econometric models. This section also describes the outcomes of interest and presents the impact results.

The variables that were used for the matching procedure at the division level, were collected by ILRI for programme placement before 2005. These include milk density, proportion of poor households, and travel time to the nearest urban centre, which we have included in Table 3. The original ILRI data also included the number of households and area, although the ILRI final targeting exercise only utilised milk density and proportion of poor households. As shown in Table 3, milk density in the study area is on average 83,000 litres/km2/year. The difference in milk density between the SDCP areas and the control areas is not statistically significant. From examining the proportion of poor households in the division, we see that more than half of the households in our dataset are poor. The proportion of poor people in the SDCP households and control households is similar at 53%. In terms of distance, SDCP households travel 0.39 to hours to the nearest urban centre as do households in the comparison group.

The averages for these three key results are similar for both the treatment and control groups. More specifically, Table 3 also presents the results of a linear model for the probability of being in the treatment group as a function of the three division-level characteristics used for program placement. This regression is conducted at the dairy group level and it includes county fixed effects, which is the administrative level at which the initial matching of divisions was conducted. The results show that none of the three placement variables is a good predictor of the probability of being in the treatment group. In fact, we are not able to reject the hypothesis that the three variables are jointly equal to zero using an F test (p-value = 0.86). These results provide evidence that we are able to find divisions in the same counties where SDCP operates that could have been chosen for the programme given based on the three placement characteristics, but were not chosen due to programme capacity constraints.

Note, however, that the finding control divisions that are similar to SDCP division does not guarantee that the individual dairy groups within these divisions are similar. To increase comparability beyond the study site selection, we included a short filter questionnaire at the beginning of the household survey to capture key time-invariant characteristics and retrospective information on dairy farming to improve the matching process between each one of households in the treatment group and a similar farm household in a comparison area. In Table 4 we specify a linear probability model to assess to what extent the treatment and control households differ in terms of key household and agricultural characteristics. The table shows the mean of each explanatory variable, the change in the propensity of being in the treatment group after a change of one unit on a given characteristic, the standard error of each estimate, and the corresponding p-value.

In terms of household demographics, the data show that the typical household speaks local languages (only 8 percent of the households speak English), has around three people in the working age group (people 14- to 65-years old) and 0.5 children 5 years of age and younger. Also, on average, the SDCP household head is 52 years old and has 13 years of education. Lastly, about 24% of households in our data have a female head of household. The percentage of female-headed households is 11 percentage points higher in the SDCP group than in the control group, a difference that is statistically significant.

Regarding the household socio-economic and agriculture characteristics, the results show that there are no statistically significant differences between treatment and control households in terms of land size, slope of farm plots, access to irrigation, and area cultivated. The average household in the sample has 0.19 acres with a legal title, 87 percent of the farms are either flat or have a slight slope, and does not have access to irrigation. In turn, there are some variables where treatment and control households exhibit differences that are statistically significant. A one standard-deviation increase in the indices of consumer durables and agricultural implement increase the probability of being in the treatment group by approximately 2 percentage points. Also, treatment households are 10 percentage points more likely to have cultivated a crop in the primary agricultural season. Lastly, there were some differences in the breed composition of the division where the household is located as measured in 2005. (4) That is, an increase of 100 zebu heads in 2005 on the division where the household is located decreases the probability of being in the treatment group by 10 percentage points. Alternatively, an increase of 100 cross heads increases the probability of being in the treatment group by 10 percentage points. There are no differences back in 2005 between treatment and control households in terms of the presence of grade cattle in their corresponding divisions.

Lastly, treated and comparison households have similar probabilities of having received other agricultural or livestock development programmes.

Overall, the results show that, with the exception of few variables, treatment and control households have very similar household and socio-economic characteristics. Nevertheless, some of the reported differences in observable characteristics could be an indication that there are differences in unobservable characteristics between the two groups that may ultimately bias programme impacts. Indeed, the balance plot in Exhibit 3 below shows there is a skew towards relatively high propensity scores for the treated households which motivated the use of the Inverse-probability-weighted regression adjustment (IPWRA). It is reassuring for the identification strategy that the weights produce very similar densities of the propensity score.

Moreover, we also conduct balance tests for each covariate considered in the treatment equation, where each observation is weighted by a an inverse function of the estimated propensity score. As shown in Table 5, all the covariates considered are balanced between the treatment and comparison groups when the full sample is considered. In addition to that, we conducted covariate balance tests by strata of the propensity score to check whether the different covariates where also balanced at different points over the distribution of the propensity score. For this analysis, we considered seven blocks or strata of the propensity score to ensure that the mean propensity score is not different for treated and controls in each block. Given that our treatment equation includes 20 covariates, we conducted a total 140 tests (=7*20) of the balancing property. Of those tests, only 5 of tests (3%) reject the hypothesis of T and C having the same mean for a given variable within a given block. Overall the results indicate there is good covariate balance after implementing a strategy based on inverse-probability weighting.

Table 6 indicates that more than half of the households in our dataset reported that there are other agriculture-based projects in their communities other than SDCP. Most of those projects are agricultural ones; the ones that focus on livestock represent only a small portion of the projects available.

Mixed Methods Analysis (Quantitative and Qualitative)

Methods: Regression techniques

As indicated above, at the household level the identification strategy used to estimate SDCP impacts combines regression and propensity score methods in a three-step approach to estimate treatment effects. In the first step, we estimate the probability of programme participation through a logit or probit such as:

[T.sub.i] [PHI] ([X.sub.i] * [beta]' + [[epsilon].sub.i])

where [T.sub.i] is a dummy for having received the programme, [X.sub.i] is a vector of individual and division-level characteristics, and [[epsilon].sub.i] is an error term. The observable characteristics considered in the treatment equation include the 2006 division level characteristics used for the initial programme placement, namely, milk density (litre/km2/year), the proportion of poor people in the division, and the travel time (in hours) to the nearest urban centre. In addition to these variables, we include other 2006 division level variables constructed by ILRI that account for the estimated number of heads from the different breed categories (i.e., local, crossed, and exotic), which are a good indication of the division breed composition, which are good predictors of the ability to increase the prevalence of higher milk producing breeds. We also control for household level characteristics such as the gender and age of the household head, the head's maximum number of years of schooling, the language spoken at home, the number of working age members and the number of children under the age of 5. We also control for indices of consumer durables and agricultural implements as a way to control for household income differences as well as dummies for the existence of other agricultural and livestock projects (different from the SDCP) in the last years in the village where the respondent lives. In order to account for differences in agricultural production in the estimation of the propensity score, we also control for a set of agricultural production characteristics such as indicator variables for producing any crops in the primary and secondary rainy seasons, the natural log of the hectares cultivated in each one of the two seasons, dummies for the slope of the main plot being flat or slightly flat, and indicator variables for the availability of irrigation in at least one of the plots and the percent of plots being served by an irrigation scheme. We also control for county level fixed effects to ensure that the comparison of treatment and comparison households are done within a given county.

Then we generate propensity scores, [P.sub.i], the probability of receiving treatment, as

[mathematical expression not reproducible]

Second, using the estimated inverse-probability weights, weighted regression models are fit for the outcome equation for each treatment level and obtained the treatment-specific predicted outcomes for each subject. Lastly, means of the treatment-specific predicted outcomes are computed and the difference of these averages provides the estimate of the average treatment effect (ATE) of the programme. More specifically, we estimated programme impacts using the teffects ipwra in STATA using the osample option. Note that all observations in the original sample are used in analysis as there are no observations that violate the overlap assumption.

Intermediate Outcomes

Improved Animal Management

We start the presentation of the programme impacts by looking at the results on animal management. In Table 7 we show the results for the impact of the programme on feeding and water practices farmers use with their cattle. Although the programme did not have a statistically significant impact across all dimensions of feeding and water practices, positive impacts existed related to the grazing practices, keeping practices, and feeding of concentrate feeds. Specifically, we find that SDCP increases the probability that a farmer practices zero grazing with their cattle by 8 percentage points. SDCP staff stated that practicing zero grazing was one of the key messages that the programme delivered to farmers in order to keep control of how much the cows eat and the nutritional value of what they eat as well as reducing losses in potential milk production from food searching in large areas. Improved animal management was the most frequently discussed benefit of the SDCP among FGD participants. One farmer described the changes she made in animal management, "Initially I would send the children to go and graze, [but] I now practice zero grazing. I realized I used to lose a lot on milk yields. At least I can produce more milk, I treat [the cows] when they fall sick, clean them, and ensure they are vaccinated."

We also find that the SDCP increases the probability that farmers keep their cattle in a paddock that has a stall floor made of concrete by 5 percentage points. This increase is promising since concrete floors are easier to clean, which reduces the change that the milk could get contaminated, which may have a positive effect on the selling price charged to some customers. We also find that the SDCP increases the probability that farmers feed cattle concentrate feeds and mineral supplements by 13 percentage points up. The programme also shows a positive impact on the probability of feeding cattle off the ground in a trough when milking, which helps protect food from contamination and promotes a more natural grazing posture. These result suggests that farmers were receptive to programme education regarding the nutritional content and feeding practices, which may have a positive effect on milk production.

The SDCP also led to an improvement in breeding services. Specifically, the SDCP increased the propensity of households monitoring their cattle on a regular basis and the use of Artificial Insemination (AI) services. Table 8 shows that treatment farmers were 7 percentage points more likely to report they monitor their cattle regularly. Similarly, treatment farmers were 12 percentage points more likely to report that they have used the AI Service in the last 12 months. However, we did not find any statistically significant differences between the treatment and comparison groups related to availability and use of own bull service for breeding.

The positive quantitative findings related to access to and use of AI are especially promising considering that many farmers discussed ongoing challenges with AI. The use of AI seems to be a high cost to the farmer, and misperceptions about the purpose and potential of AI services mean that farmers are not able to make informed decisions in these transactions. For example, many farmers seemed to be unclear on the proper timing for AI, and a few farmers thought they could pay more for the AI provider to select the sex when they inseminate. Other challenges included that farmers don't believe that government-subsidized AI services provide the best breeds, and that they have to purchase AI services too often because they are often unsuccessful.

However, in the qualitative data, many farmers discussed ongoing challenges with AI. The use of AI seems to be a high cost to the farmer, and misperceptions about the purpose and potential of AI services mean that farmers are not able to make informed decisions in these transactions. For example, many farmers seemed to be unclear on the proper timing for AI, and a few farmers thought they could pay more for the AI provider to select the sex when they inseminate. Other challenges included that farmers don't believe that government-subsidized AI services provide the best breeds, and that they have to purchase AI services too often because they are often unsuccessful.

The program had positive impacts related to animal health services, with the impacts concentrated on improved access to and use of vaccination services and curative treatment services. Table 9 shows that the SDCP increased the probability that treatment farmers both had access to and used vaccination services by 25 percentage points. Similarly, SDCP farmers are 10 percentage points more likely to have access to curative treatment services and 7 percentage points more likely to use them than farmers in the comparison group. However, the programme does not seem to have changed access to change other health services such as deworming (anthelmintics) or tick control services.

Improved Access to Extension and Information

The SDCP was implemented through several components where the dissemination of information on different areas of the productive process were largely conducted through extension services and trainings of dairy group members. In particular, SDCP provided training on organisational, managerial, and enterprise skills (e.g., bookkeeping, accounting, financial planning) to farmers. In addition, the programme also conducted trainings, field days, and demonstrations specifically designed to enhance dairy farming productivity and reduce production costs. In this section, we assess the effect of the program on making these services available and to what extend the information provided is adopted.

Table 10 presents the results on the reported availability of extension visits, field days, and demonstrations. All these outcomes consistently show that farmers in SDCP areas have a higher access to these type of services. That is, program farmers are on average between 12 to 18 percentage points more likely to report that the different types of extension services are available in their villages.

We also investigated if the programme had any effects on the probability of receiving any information on specific aspects of the production process as well as whether farmers who attended those trainings adopted the recommended practices. The largest program effects are seen on receiving information on general livestock practices (15 percentage points), milk processing and quality control (10 percentage points), fodder establishment and fresh milk marketing (7 percentage points). However, positive impacts are observed for almost all type of cattle-related topics, which may be a consequence of the programme tailoring specific trainings according to the needs and preferences of each dairy group. The results on adopting specific practices closely resemble the impacts reported on Table 11. Interestingly, SDCP farmers did not report having higher access to information on crop best practices relative to the comparison group, which is reassuting as the SDCP programme did not provide any training on crop practices.

Improved Dairy Group Performance

Following the "structure-conduct-performance" literature, here we will first present results on structure, then conduct, and finally, performance. We surveyed 94 dairy groups in 73 SDCP communities and 90 dairy groups in 83 control communities; we note here that enumerators surveyed the "most important" dairy groups within a community. The first difference to note is that there were more dairy group questionnaires implemented in SDCP than in control communities. Also, the results we present here are naive differences between SDCP and control dairy groups. Finally, we only summarize the main results of a larger report on dairy groups, and full results can be found in that report.

In terms of basic structure, nearly all dairy groups have an elected chair, and nearly all of the groups (97%) are legally registered and have by-laws or a written constitution. Additionally, there was similar representation of women as respondents to the dairy group questionnaire and in terms of positions held within the dairy group--just over 50% for both SDCP and control dairy groups. On the other hand, SDCP groups are more likely to also have an elected secretary and treasurer. We expect that groups with more formalized structures should perform better in this context, primarily because one of the main goals of the programme is to link the dairy groups to the milk value chain rather than focusing exclusively on intercommunity activities and also because dues are collected and managed, meaning accountability is important. Committees may also improve the performance of the dairy group, enabling subgroup members to focus on specific tasks and enabling members with specific expertise to exploit their comparative advantages within a committee. Of four committees asked about explicitly, plus and "other" category, SDCP groups were not more likely to have specific committees overall and were significantly less likely to have financial committees. Additionally, SDCP groups are more likely to have monthly meetings, but control groups are more likely to hold weekly meetings; more meetings may mean greater opportunities to share information and engage in decision-making, but too many meetings may also be inefficient. Thus, evidence on structure is somewhat mixed; there are more elected positions but fewer committees, and meetings are held less often.

We have only two aspects of conduct: how decisions are made and sources of financing accessed. In terms of decision-making, there are very few differences between SDCP and control dairy groups across decisions on three broad areas: financing, marketing and information acquisition, and dissemination. We allowed for decisions to be made by consensus, by the chair with other key members, and by committee. The only statistically significant difference is that information decisions for the SDCP group are more likely to be made by the chair plus other key members. For the control dairy groups, information decisions were more evenly spread across the three decision-making mechanisms. On the other hand, it is interesting to note that while finance and marketing decisions were most often made by consensus in more than 50% of both SDCP and control communities, consensus-based decision making was less important for information decisions--between 32% and 36% for SDCP and control communities, respectively. Next, we asked about four specific types of financing and a fifth "other" category. Here, SDCP and control groups exhibited a number of statistically significant differences. SDCP groups were far more likely to obtain financing from micro-finance institutions (14% vs. 1%), from "other" sources that were mainly different types of local credit and savings groups (55% vs. 43%), and from commercial banks (5% vs. 1%). Both SDCP and control groups, however, also relied heavily on members' dues, with 93% stating they relied on member dues for both groups. Overall, both SDCP and control groups exhibit similar patterns in decision-making procedures, but SDCP groups are more likely to obtain financing from a wider range of financial institutions.

Finally, we have a number of measures of dairy group performance, including the different types and frequencies of conflict, different types of trainings, and various "other" dairy group activities. With respect to conflicts, we asked about unpaid dues, amount of dues, participation in trainings, financial issues, management issues, and "other" conflicts. For the most part, SDCP and control groups faced similar percentages of conflicts, with the exception of unpaid dues. In SDCP groups, only 21% said there were conflicts over unpaid dues, versus 37% for control groups. This is interesting because SDCP groups were much less likely to have financial committees, but they were also more likely to have an elected treasurer. Evidence suggests having an elected treasurer may be more important than having specific financing committees, but we did not ask specifically about how members' dues were collected, which could shed more light on this issue. Also of interest is the fact that 39% of both SDCP and control groups indicated that there had been conflicts. Outside of unpaid dues, the most important categories of conflicts included management issues, amount of dues, and "other" issues.

Where we see the most significant difference between SDCP and control dairy groups is in trainings made available to dairy group members. Of nine dairy group topics, SDCP groups stated that at least one training had occurred was statistically greater than control groups on all nine topics. The differences were particularly pronounced for the following topics: dairy group management, proposal writing, and fodder management. Finally, we asked about a number of other services provided by the dairy groups, including organizing educational exchange tours, collecting/sharing milk price data, facilitating links between members and input suppliers, facilitating links between members and milk purchasers, contracting with input suppliers on behalf of members, contracting with milk purchasers on behalf of members, and "other." There was very little difference between SDCP and control dairy groups on the percent of groups that offered the different services, except that SDCP groups were more likely to contract with milk purchasers on behalf of members, 11% versus 2%. It is also interesting to note that over 70% of groups, both SDCP and control, arranged educational exchange tours, while less than 30% of groups undertook any of the other services.

Overall, the evidence from the dairy group survey suggests that SDCP dairy groups function and perform a bit differently than comparison groups. In particular, they are more likely to have elected officials below the chairperson; more likely to access multiple sources of financing; less likely to face conflicts over collecting dues from members; and more likely to contract with milk purchasers on behalf of members, albeit at a relatively low rate (11%). The most striking difference, however, is in the number of trainings provided primarily by SDCP to dairy group members. The latter suggests that there is room to improve the performance of SDCP dairy groups, particularly in terms of the services that the SDCP promotes to reduce smallholders' transactions costs in accessing dairy input and milk markets, thereby increasing net revenues accruing to smallholders.

Main Outcomes

One of the key goals of the SDCP is to increase the income of poor rural households that depend substantially on production and trade of dairy products for their livelihood. To attain this goal, the programme focused on improving productivity, making some inputs more accessible, and emphasize the importance of value adding and more reliable trade relations. In this section, we investigate program impacts on some key final outcomes of the programme.

First, we look at the effects that the programme had on the cattle size and composition. As shown in Table 12, we find that, relative to the comparison group, SDCP farmers not only own 0.5 more cattle heads (including cows, males, heifers, calves, pre-weaning males), but also have a higher number of cows (calved at least once) as well as animals they are currently milking. Interestingly, these higher impacts on number of animals is mostly explained by a higher number of cross breed cattle and not of the exotic (i.e., grade) more productive breeds.

We then investigate programme impacts on milk production, commercialisation, and total value. Our findings indicate that SDCP farmers are 8 percentage points more likely to sell milk both in the morning and the evening, although only the morning propensity of selling is statistically significant. The evidence suggests SDCP farmers are selling more milk in the market although the results again are not statistically significant at the 10 percent level of significance. Nevertheless, those farmers selling to the market are able to obtain selling price that is 31 percent higher than the selling price received by non-beneficiaries. Overall, the total value of milk sold, calculate as the quantity of milk sold times the price, observed by SDCP farmers is 43 percent higher than the value of the comparison group. Moreover, SDCP farmers show a higher level of milk production for animals at calving as well as the total production calculate as the sum of milk sold, consumed in the household, and lost.

These quantitative results are supported by qualitative data on the perceived effect of the SDCP on farmers' incomes from dairy farming. Without being prompted, FGD participants said they believed SDCP-sponsored activities had enabled them to more easily pay bills and send their children to school. One participant said, "My milk production has increased and that's more income. With it, I have taken my children to school." The perceived effect of the programme on income could partly result from the SDCP focus on recordkeeping. For example, one farmer said, "There is recordkeeping--tracking milk production and noting fall in its production--and getting to understand source of the problem and how to solve it." Though qualitative data cannot accurately indicate whether there was a real nominal increase in production and productivity per animal, FGD participants from treatment groups generally perceived a difference in production as a result of SDCP. Multiple anecdotes indicate farmers' perceptions of increased productivity. One farmer described her increased yields: "Initially I would only get 3 cups of milk, but currently my cow is producing 7 bottles." She also added that this increase helps to pay for school fees.

Increased Food Security

FGD participants said they believed increased income as a result of the SDCP had enabled them to keep a variety of foods available in their households. One farmer said that his family's general health had improved, while others said they now consistently have tea with milk in their house. Farmers' perception of increased food security may partially result from understanding better practices for growing crops, including the use of cow dung for harvesting. One farmer said, "The animal manure from both the goats and cattle is channeled on the farm, which in turn fastens the growth of crops and more yields--this too has ensured a consistency supply of food in key household."

This evidence from qualitative data goes in line with the estimated impacts on the ability of SDCP households to translate higher incomes into higher levels of food security. The survey includes details about food consumption within households to get a sense of higher income has led to more dietary diversity, which is a proxy for food security. One such measure of food security was developed by the Food and Nutrition Technical Assistance Project (FANTA); greater values of the FANTA measure indicate more food insecurity. The FANTA Project includes guidelines for a dietary diversity questionnaire that can be used at the household level. Specifically this diversity measures involves calculating dietary diversity scores by summing the number of food groups consumed by anyone in the household over a reference period (our questionnaire has set this to be the last 7 days). Thus, the dietary diversity scores consist of a simple count of food groups that the household has consumed in the last 7 days.

The results in Table 14 show that SDCP households are more likely to have a more diverse food basket, specially foods with larger levels of animal and vegetable proteins (red meats, milk products, and legumes such as beans, peas, lentils, and nuts), and lower levels of tuber and fruit consumption, which are nonetheless still quite common among SDCP farmers. Overall, the results provide some evidence that programme beneficiaries were able to exhibit higher levels of food diversification towards more nutritional food items.

Gender Differential Effects

In this section, we investigate differential programme effects by gender of the household head. As indicated earlier, one of SDCP goals is to serve women dairy farmers and the dairy groups they form. As a result, being a female headed household increases the probability of participating in the programme by 11 percentage points (see Table 4). To the extent that female headed households tend to have a lower socio-economic status than male headed households (e.g., female headed households have fewer working age members, have fewer consumer durables or agricultural implements, and have fewer extensions of cultivated land), program impacts may be different in terms of this gender dimension.

We first explore programme impacts on household decision making. As part of the household survey, we collected information on numerous variables regarding who in the household participates in making decisions about dairy activities, including use of inputs and providers, management of money from milk sales, and request of dairy-related trainings. All the questions considered allow respondents to choose from five options about who makes the decision: household male, household female, joint household (male & female), non-household member, and other. For the analytical purposes, we created indicator variables for all these decision making questions where the variable is equal to 1 if a female member made the decision and 0 otherwise.

We present the results for some key variables in Table 15. (5) For most of the outcomes considered, we see a positive effect of the programme on the probability of making a decision relative to the comparison group. Treated households are nine percentage points more likely to have a female managing the cash from fresh milk sold relative to non-SDCP households. Also, SDCP households are 7 and 11 percentage points more likely than the comparison group to have a female deciding the provider for bull and artificial insemination services respectively. Lastly, household beneficiaries are more likely to have a female deciding over the use of different services such as use of anthelmintics, tick control, vaccination, and curative treatments.

The overall results indicate that the SDCP programme had a positive impact on the welfare of female dairy farmers by allowing women to be more empowered when making decisions related to dairy production. These quantitative results are encouraging since the qualitative results did not provide insight into the gender dynamics of dairy farming. Gender was included as a topic in FGDs, but farmers had very little interesting information to share about their perspectives on gender.

Cost-Analysis

The total cost of the SDCP is 1,515,614,910 KSH for the period from 2006 through 2016. Appendix Table J1 includes the yearly expenses across categories. The largest category of expenses was the "Technical Assistance, Training, & Workshops" expense, which amounted to 33.4% of the overall budget. This large share of the budget is not surprising since many of the programme activities related to this line item. Other higher categories of expenses included "Salaries & Allowances," "Vehicle & Office Operating Costs," and "Vehicles, Equipment, & Materials," at 17.5%, 13.2%, and 14.5% respectively.

To compare the costs to the benefit that each farmer receives from the program, we first need an estimate of the costs per beneficiary. To determine the number of beneficiaries, we examined the Dairy Groups Inventory from the programme counties and divisions, which we summarized in Appendix Table J2. Most of the inventories were updated in 2012, with some having been revised in 2011. Across treatment divisions, there were 505 dairy groups with 15,535 total dairy group members. On average each dairy group had 30.8 members. Taking the total costs from Appendix Table J1 and the total number of beneficiaries from Appendix Table J2, we can estimate that the total costs per beneficiary were approximately 97,561 KSH (equal to 1,515,614,910 KSH divided by 15,535).

To compare the costs to the benefit that each farmer receives from the program, we next need a measure of the benefits of the programme at the farmer level. While we recognize that the programme is associated with a variety of farmer-level benefits, we focus on the benefit of increased milk production for the purpose of this analysis, since that was one of the main goals of the SDCP. The unconditional average of litres of milk produced yesterday for a farmer in the comparison group was 3.81 litres, where this estimate includes zero values for comparison group farmers who produced no milk yesterday. Our impact estimates included in Table 13 suggest that the program increases milk production by 37%, suggesting that the additional amount of milk produced yesterday by a farmer in the treatment group is 1.41 litres. Because our estimate of the milk produced yesterday includes values of zero for when the farmer did not produce milk that day, these estimates reflect the amount produced on an average day. We assume this average level of production throughout the year and multiply the estimate of 1.41 additional litres per day by 365 days in a year to estimate that the additional milk produced by a farmer in a year is 514.65 litres. In monetary terms this is equivalent to a benefit of 20,586 KSH per farmer since the unconditional mean, which excludes zeros for those who did not sell milk, of the price per litre of milk is 40 KSH. This benefit estimate corresponds to the 2016 time frame as the data on yesterday's milk production was collected in 2016.

With an estimated total cost per beneficiary of 97,561 KSH and an estimated per farmer benefit of 20,586 KSH in the year 2016, we calculate that it would take approximately 4.74 years (equal to 97,561 KSH divided by 20,586 KSH) to "break even," that is for the benefits to equal the costs, assuming the benefits of 20,586 KSH remain constant across years. This estimate of the number of years to break even seems reasonable for although there likely were no benefits at the beginning of the programme (when upfront costs were associated with setting-up the SDCP), farmers likely began benefiting from the programme prior to 2016. Furthermore, to the extent farmers continue to employ the best practices advocated by the programme, the benefits may extend into future years. In addition, these estimates seem reasonable because we only examined the benefit of increased milk production and the SDCP benefited treatment farmers in a variety of dimensions.

Discussion and Conclusions

Overall, the results suggest that the SDCP was successful in increasing milk production, but with more limited--though positive--impacts on increasing milk marketing and increasing milk prices received by smallholders. Thus, we first summarize results pertaining to milk production, and then on milk marketing. We end the section with a summary of the key challenges remaining.

Milk Production

SDCP households surveyed were more likely to have received information on all of the practices being promoted by SDCP versus control households, and they were also more likely to have adopted those practices as well. In addition, SDCP households were more likely to have cross-bred cows, to use AI services, and to have obtained a wider range of health services. Finally, they were also more likely to adopt recommended management practices and investments, including practicing zero grazing, having concrete floors, and feeding concentrates. In fact, control households did not perform better on any measures of input use, management and investment, with just one exception; SDCP households were less likely to clean cows teats before milking. Overall, these improved input and management practices led to greater milk.

In addition to increasing milk production per cow, another key objective of SDCP was to smooth milk production across the year, by increasing access to adequate feed and fodder throughout the year. Because of the difficulty in obtaining recall data on milk production throughout a year, we have no direct evidence on whether SDCP farmers actually have smoother outputs. We did find that SDCP farmers were more likely to adopt fodder establishment, hay making, silage making and conservation of crop residues, all of which should contribute to increasing availability of adequate fodder throughout the year. However, adoption rates for each of these practices are below 15%, suggesting there may still be room to improve activities and trainings aimed at smoothing fodder availability and thus milk production throughout the year.

Results from the household survey are consistent both with results from the dairy group survey and the FGDs. For instance, SDCP dairy group respondents noted higher access to trainings on almost all topics. In particular, 79% of SDCP dairy groups noted that at least one training had been provided on fodder management, well above the 36% in control dairy groups. However, household respondents were less likely to receive information on various practices associated in both groups, implying that relatively few households in the dairy group attended these trainings. In fact, the number of dairy group leaders and members attending various trainings was somewhat limited, at about 16 attendees for SDCP trainings and slightly less in control groups. The latter implies scope for increasing dissemination of knowledge learned at trainings to the wider group of dairy farmers who were not able to attend.

FGD participants thought that the SDCP project helped them achieve higher milk yields, higher income and greater food security. The most frequently discussed benefit was improved animal management. Many participants also discussed the need to make and preserve fodder to ensure year-round availability of fodder for their dairy cows, and SDCP farmers appear to have a more sophisticated knowledge of fodder management. However, they also expressed barriers some farmers faced in adopting best practices. In part, FGD participants thought increased availability and consistency of technical services provided by SDCP would help increase the number of adopters. Additionally, many farmers had misperceptions about how AI works, and many also thought the price was too high, particularly given that success is not ensured the first time.

Milk and Input Markets

Both total milk production and milk sold was higher for SDCP, though the impact is higher on production versus milk marketed. In part this appears to be due to the fact that SDCP producers are more likely to have sold any milk the preceding day, and in particular, they are more likely to have sold milk in the evening and to have sold both in the morning and the evening. Thus, this evidence suggests that there were positive impacts on milk marketing. On the other hand, from the household survey, the percent of households receiving information on market-related topics was in general lower than for production-related and farm management topics, for both SDCP and control groups, though SDCP households were more likely to receive market-related information than controls. For instance, just 9% of households noted that they had received information on fresh milk marketing, with even fewer having received information on value addition and cooperative milk marketing. Similarly, in the dairy group surveys, just 59% reported trainings related to marketing, significantly below the number reporting trainings on production related activities such as fodder management (79%).

The dairy group survey also reflects more limited impacts on marketing services they provide to their members. First, there are limited differences between treated and control dairy groups in the types of services offered. Just 20% of SDCP groups facilitate links between members and input suppliers, and just 24% facilitate linking members to milk purchases--similar to the percentages observed in control groups. Just 9% of SDCP dairy groups contract with input suppliers on behalf of members, and 11% contract with milk purchasers on behalf of members. However, while 11% is fairly low, it is significantly higher than control groups who contract with milk purchasers, at 2%. At least some treatment FGDs mentioned that they thought the project had reduced transactions costs, and one participant said that it had become easier to find a steadier customer base. On the other hand, many participants also noted entering the market was still a challenge for many farmers, and the FGD interviewers noted that farmers expressed a lack of understanding of how to expand their business.

One of the more successful initiatives to help farmers access input and output markets appears to be the expansion of access to credit, as primarily documented in the dairy group surveys and FGDs. SDCP diary groups were more likely to access a wider range of finance sources, including micro-finance and commercial, but particularly local savings & loan clubs. Similarly, treatment farmers in FGDs frequently mentioned SDCP trainings which led to the establishment of merry-go-round and table banking groups.

Challenges

Overall, SDCP activities contribute to a positive overall picture of the impact and perceptions of the SDCP on Kenyan dairy farmers, particularly for dairy production. However, insights from the qualitative data also point to challenges in service provision, utilization of technical knowledge and skills, and institutional coordination that could enhance the program.

Uptake and Ongoing Services

Key informants notably expressed different primary challenges to dairy farming commercialization in Kenya than their beneficiary counterparts, albeit that the challenges were also elements that the SDCP actively aims to address. In addition to mentioning challenges associated with increasing uptake of improved feed management, marketing, and dairy farm management, key informants' top two most frequently mentioned challenges were (1) the availability and consistency of technical services that relate to the SDCP and (2) the lack of uptake and utilization of available knowledge and technical skills among farmers. Discussion of these two primary challenges was related. Many key informants particularly mentioned extension among the government-provided services that suffer because of lack of staff and funding. One informant said, "All development projects within the county go through the same officers within the county, [and the] number of staff are of staff are not enough to reach out to those communities." This statement highlights that despite the provision of SDCP activities, a staffing bottleneck may prevent these services from being implemented as intended. One trainer from the ministry of agriculture recommended they, "Visit the groups mid-year to see if they implement what we have trained them on and if they have any technical challenges."

Given that such services aim to help farmers with technical knowledge, it makes sense that utilization of available knowledge and technical skills was also a notable challenge. Even in cases where key informants believed that information about technical skills in farm management had reached the farmers, they then mentioned challenges with farmers being able to utilize the knowledge because of capital constraints. One key informant pointed out this constraint, "Much as we offer the extension, services may not be adequate and [...] farmers are slow in taking up the services that we offer." The same key informant connected the challenge of services with financial challenges on the part of the farmer, "Like in A.I--as much it is subsidized farmers still, complain that the price is too high. The farmers have not fully the idea of commercialization."

In addition to lack of capital to consistently implement farm and animal management practices, farmers also have challenges with negotiating entrance into the market. One dairy group chairperson said of the SDCP, "now that they have improved our lives by providing machines for handling milk, they should try to continue supporting us to see that we can even package and market for ourselves." This indicates that despite having had trainings on marketing, not all farmers have put the changes into practice. Farmers were also unable to concretely identify how to go about expanding their business in other ways. For example, one dairy group member said, "I would like if we could get capital to invest in our business of making feeds and we market them and sale to other places." However, capital was an issue in this group, as well as others. For example, another farmer said, "If we can get our own doctors and at least our group to pay for the treatment then they can deduct the cost in milk, then we would really prosper."

Institutional Coordination and Linkages

FGDs indicated there are challenges with linking farmers and dairy groups to various service providers, and that farmers do not always have the knowledge to negotiate to their benefit. Key informants agreed that bringing service providers together continued to be a challenge. One key informant said there continues to be "inadequate coordination of the value chain actors, especially in terms of contract terms and contractual agreements." Farmers generally did not discuss interactions with value chain actors other than AI providers and veterinary officers. One farmer said, "The service providers it's on need basis, while the agricultural extension officers we meet them on weekly basis as they liaise with other stake holders like world bank, [or others who] deal with breeding of animals and rearing of chicken ," indicating that they rely on extension officers for coordination of major animal management activities, in some cases.

There was no mention among dairy group farmers or their chairpeople of any formal contractual arrangements with banks, service providers, or other actors on the value chain. The lack of agreements was evident when one key informant said he wished there could be, "Proper structures on marketing of milk and increased production of milk from individual farmers. We should not experience case that happened in Nyandarua where they poured milk because they did not have a place to take it." One farmer did explain that his group entered into a marketing agreement: "we have approached an organization called Inter Region Economic Network and they came in and researched and decided they will bring us marketers for our products and as a result we shall sell our milk as individuals. So the agreement was they sell the milk in very huge amounts and they keep 15% and us the rest 85%." However, there was no other mention of such arrangements.

Funds Disbursement

Finally, a major challenge to program implementation cited by multiple key informants was the inability to get SDCP funds through the various levels of the government to enable timely implementation of initiatives on the ground. Key informants mentioned particular elements of funding including disbursing funds from the national level of the government, enabling payments to service providers, and--even among farmers--the inability to purchase services, which some informants speculated comes from an impression among farmers that they should not have to pay for services or get monetary provisions for attending trainings. This challenge among farmers aligns with the prior idea that farmers have not fully taken up the trainings. One informant reiterated, "The challenges on finances [...] hamper implementation, [including with] group issues on governance, leadership and resource mobilization. They may have the knowledge, but still those conflicts arise."

At the level of government implementation, one key informant said there was a challenge with "Authority to incur expenditure; you cannot actually be allowed to spend before it's signed." At the time of the interview, it had been more than a month since the office was supposed to receive the authority to incur, yet they had not received it. Key informants provided mixed responses on whether devolution caused any of these challenges with SDCP implementation, though some respondents thought having an extra layer of bureaucracy may contribute to the slowdown of funds and information flow (despite that the program is still implemented from the national level).

Internal and External Validity

The results of the evaluation of the SDCP programme can be used to inform the design and implementation of similar dairy policies and projects both elsewhere in Kenya and in other developing counties. First, there are at least 16 counties in Kenya that, although not being served by the programme, have comparable resource poor dairy farmers who we would expect will benefit in similar ways from SDCP activities should they receive the programme. Although the road densities might differ from the densities selected by IFAD and GoK, the selected districts had large road densities and were also relatively contiguous in order to ease SDCP management. Thus, the results should extend to similar areas in Kenya with large road densities that are not necessarily contiguous. Second, many of the features and challenges that smallholder dairy farmers face in Kenya are shared by many dairy farmers in Eastern Africa. Most small dairy farmers from neighbouring countries face a sub-tropical geography suitable for dairy cattle and produce milk as an important part of their diet (Thorpe et. al., 2000). These dairy farmers also exhibit similar challenges such as lack of training and availability of production inputs that prevent them from commercializing their dairy products in the most efficient way.

More generally, the effects of the SDCP programme can provide some insights about other types of agricultural extension programmes to smallholder farmers in developing countries. A key objective of agricultural extension is to increase the knowledge of farmers about agricultural practices, which in turn should have an effect on productivity. The traditional extension model, known as "Training and Visit" (T&V) extension, is characterized by government-employed extension agents visiting farmers individually or in groups to demonstrate agricultural best practices (Anderson and Birner, 2007). However, the T&V approach has been criticized for a series of reasons. First, given the limited transportation infrastructure in rural areas and the high costs of delivering information in person in some developing countries the reach of extension programs are limited. Second, as it is costly to provide agricultural extension to farmers on a recurring basis, the infrequent and irregular meetings limit the ability to provide timely information, which further limits the ability of farmers to provide timely information. Lastly, usually the information provided to farmers is too top-down, which results in an inadequate diagnosis of the difficulties currently facing farmers, as well as information that is often too technical for semi-literate farming populations.

The SDCP programme has some key features that may help overcome some of the limitations of this traditional extension approach. In particular, a key aspect of the programme is that it aims at building the capacity of groups of farmers through trainings that allow them to identify their most pressing needs and design work plans to request more targeted assistance from the programme in terms of both information and resources. Also, by increasing the capacity of groups and not only individual farmers, the programme may not only be more cost efficient, but also generate some positive externalities to other farmers in the area who may later become part of the groups. Third, the programme incentivizes farmers to organize and compete for additional funds to improve their productivity. Our results are consistent with this approach to extension; however, our results also suggest that more needs to be done to help group members who participate in trainings to subsequently share the knowledge they learn to group members who did not participate. In sum, we think the results of this evaluation and the analysis of its costs, the design of other types of extension and approaches in other agricultural sectors and countries.

We also believe that the quantitative results provide useful information on the impacts of a complex project on smallholder dairy farmers in other regions in Kenya that nonetheless face similar market-shed characteristics, and in other neighboring countries with similar market characteristics. Given specificities associated with both milk production and marketing, we do not expect that results would be easily generalizable to other cash-crop markets, however.

References

Anderson, J., & Birner, R. (2007). How to make agricultural extension demand-driven?: The case of India's Agricultural Extension Policy (IFPRI Discussion Paper 00729). Washington, DC: International Food Policy Research Institute.

Atieno, R., & Kanyinga, K. (2008). The revitalization of Kenya Cooperative Creameries: the politics of policy reforms in the dairy sector in Kenya. Future Agricultures Research Paper 009. Future Agricultures Consortium.

Bernard, H. R. (2011). Research methods in anthropology: Qualitative and quantitative approaches (5th ed.). Lanham, MD: AltaMira Press.

Burke, W.J., Jayne, T., Freeman, H., & Kristjansen, P. (2007). Factors associated with rural farm households' movements into and out of poverty: the rising importance of livestock. International Development Working Paper 90, Department of Agricultural Economics, Michigan State University.

Burke, W, Myers, M., & Jayne, T. (2015). A triple-hurdle model of production and market participation in Kenya's dairy market. American Journal of Agricultural Economics, 97(4), 1227-1246

FAO Global Information and Early Warning System Country Brief on Kenya (April 25, 2017) available at http://www.fao.org/giews/countrybrief/country.jsp?code=KEN accessed June 16, 2017.

Gelan, A., & Muriithi, B. (2015). Examining returns to scale in smallholder dairy farms in East Africa. Quarterly Journal of International Agriculture, 54(3), 237-259.

Glaser, B., & Strauss, A. (2009). The discovery of grounded theory: Strategies for qualitative research. Chicago, IL: Alpine.

Heckman, J., Ichimura, H., Smith, J., & Todd, P. (1998). Characterizing selection bias using experimental data. Econometrica, 66(5), 1017-1098.

Heckman, J., Ichimura, H., & Todd, P. (1997). Matching as an econometric evaluation estimator: evidence from evaluating a job training programme. Review of Economic Studies, 64(4), 605-654.

Herrero, M., Thornton, P., Bernues, A., Baltenweck, I., Vervoort, J., van de Steeg, J., Makokha, S., van Wijk, M., Karanja, S., Rufino, M., & Staal, S. (2014). Exploring future changes in smallholder farming systems by linking scio-economic scenarios with regional and household models. Global Environmental Change, 24, 165-182.

Holloway, G., Nicholson, C., Delgado, C., Stall, S., & Ehui, S. (2000). Agroindustrialization through institutional innovation: transactionis costs, cooperatives and milk-market development in the East African highlands. Agricultural Economics, 23(3), 279-288.

Imbens, G., & Wooldridge, J. (2009). Recent developments in the econometrics of program evaluation. Journal of Economic Literature, 47(1), 5-86.

McKenzie, D. (2011, November 20). Power calculations for propensity score matching. In: Development Impact Blog. Retrieved from http://blogs.worldbank.org/impactevaluations/power-calculations-for-propensity-score-matching

Muriuki, H. G. (2003). A review of the small scale dairy sector--Kenya. Retrieved from Report. http://www.fao.org/fileadmin/templates/ags/docs/dairy/P1assessmentkenya.pdf

Nafula, S. (2013). The impact of enhanced nutritional and feeding practices on growth and health of dairy calves in Mukurweini District of Kenya. Masgters of Veterinary Medicine Thesis, University of Nairobi. Last accessed May 12 at: http://erepository.uonbi.ac.ke/bitstream/handle/11295/72378/The%20Impact%20Of%20Enhanced%20Nutritional%20And%20Feeding%20Practices%20On%20Growth%20And%20Health%20Of%20Dairy%20Calves%20In%20Mukurweini%20District%20Of%20Kenya.pdf?sequence=3&isAllowed=y

Ouma, R., Njoroge, L., Romney, D., Ochungo, P., Staal, S., & Baltenweck, I. (2007). Targeting dairy interventions in Kenya: A guide for development planners, researchers and extension workers (ILRI Manuals and Guides No. 1). Nairobi, Kenya: International Livestock Research Institute.

Robins, J., & Rotnitzky, A. (1995). Semiparametric Efficiency in Multivariate Regression Models with Missing Data. Journal of the American Statistical Association, 90(429), 122-29.

Robins, J., Rotnitzky, A., & Zhao, L. (1995). Analysis of Semiparametric Regression Models for Repeated Outcomes in the Presence of Missing Data. Journal of the American Statistical Association, 90(429), 106-21.

Sacerdote, B. (2004). Fixing broken experiments using the propensity score. In: A. Gelman & X-L Meng (Eds.), Applied Bayesian modeling and causal inference from incomplete-data perspectives (pp. 61-71). Hoboken, NJ: Wiley.

Staal, S., Delgado, C., & Nicholson, C. (1997). Smallholkdeer dairying under transactions costs in East Africa. World Development, 25, 779-794.

Thorpe, W., Muriuki, H. G., Omore, A., Owango, M. O., & Staal, S. (2000). Development of smallholder dairying in Eastern Africa with particular reference to Kenya. Paper prepared for the UZ/RVAU/DIAS/DANIDA-ENRECA Project Review Workshop, Harare, Zimbabwe.

Van der Laan, M., & Robins, J. (2003). Unified Methods for Censored Longitudinal Data and Causality. New York: Springer, Physica-Verlag.

Appendices

Appendix A. Field notes and other information from formative work

Appendix C. Survey instruments (qualitative and quantitative)

Appendix E. Sample Size and Power Calculations

To calculate a sufficient sample size for this evaluation, we conducted a number of power analyses based on existing farm-level data for SDCP beneficiaries that were collected as part of a 2014 survey commissioned by SDCP to Capital Guardian Consulting. The data focused on a series of outcomes--such as household monthly income and expenditures, as well as milk production per cow per day--that we used for the power calculations. Note that the data were collected for 78 dairy groups in the nine programme counties.

The power analysis calculations are based on the following assumptions:

* Minimum detectable effect size (MDES) for income and production related outcomes of 0.2 standard deviations. This effect size is equivalent to a 27% increase in income and a 20% increase in milk production. Note that Monitoring and Evaluation (M&E) data from SDCP show that programme beneficiaries have increased their income and milk production over time by almost three times. While these estimates are confounded by the growth of economic conditions that would have happened even in the absence of the programme, we believe the assumed MDES are rather conservative.

* We will collect information for 154 dairy groups, which will serve as clusters for the analysis (77 treated and 77 comparison). For each cluster, we will randomly select 15 households.

* The estimated intra-cluster correlation (ICC) is 0.11 for monthly income, 0.08 for monthly expenditures, and 0.15 for milk production. Thus, to be conservative, we set the ICC to the largest of these values.

* The proportion of the outcome variance explained by the exogenous covariates is set to 0.1, which we assume to be rather low given the absence of baseline data for these outcomes. However, we will control for exogenous household characteristics and geographical fixed effects that should enable us to explain at least 10% of the outcome variation.

* Power is set to 0.80

* Alpha level is set to 0.05

* Given these assumptions, the estimated sample size to detect an effect size of at least 0.2 standard deviations is 2,309 households.

* However, we assume that 10% of observations will not fall within the common support of the propensity score matching estimation. Thus, it will be necessary to drop these observations, and so we inflate the sample size by the proportion of observations that fall within the common support, a procedure suggested by McKenzie (2011). As a result, we proposed to collect farm-level data from 2,565 households (=2,309/0.9).

* We do not expect to lose a large proportion of observations due to falling outside the common support, because through all of the activities, we are proposing to increase the comparability of the treatment and comparison groups, including the two-stage targeting process (observation- and criteria-based targeting) and the filter questionnaire.

Although one of the main goals of the SDCP is to reduce the poverty levels of programme beneficiaries, it is not feasible at this point to credibly determine the effect that the expected increase in income will have on the number of households that will be moved out of poverty. To do so, we would need the income distribution of the farmers in the comparison areas to estimate the proportion of households that will be placed above the poverty line as a result of the programme. In any case, we will estimate the effects of the programme on this indicator as part of the evaluation.

Appendix J. Cost Data for the Programme
Appendix Table J1. SDCP Yearly Expenditure by Category (in thousands of
KSH)

                                2006/07  2007/08  2008/09  2009/10

Civil Works                         32     3,310    9,548      839
Contracts for Service              216    10,650    7,604   25,350
Providers
Dairy Goat Development costs         0         0        0        0
Grant for Dairy Goat
Development for Women                0       166       78    5,303
Incremental Operating Costs:    11,871    20,015   25,226   26,531
Salaries & Allowances
International Technical              0         0        0        0
Assistance for DTI
International Technical              0         0        0        0
Assistance for Project
Management & Coordination
National Technical Assistance    1,515     3,181    3,252    4,102
& Policy
National Technical Assistance        0         0        0        0
except for Support to Policy
and Studies
Technical Assistance, Training   5,676    18,881   42,938   42,030
& Workshops
Vehicles & Office Operting       6,211    22,281   12,366   21,016
Costs
Vehicles, Equip & Materials     44,460    25,981   12,289    5,492
Total                           69,980   104,464  113,300  130,661

                                2010/11  2011/12  2012/13  2013/14

Civil Works                       5,774    1,872    1,904   52,795
Contracts for Service            21,552    2,943    1,320    5,031
Providers
Dairy Goat Development costs          0        0   14,680        0
Grant for Dairy Goat
Development for Women             4,241    6,381    8,175    5,391
Incremental Operating Costs:     28,116   29,796   30,770   29,933
Salaries & Allowances
International Technical               0        0        0        0
Assistance for DTI
International Technical               0        0        0        0
Assistance for Project
Management & Coordination
National Technical Assistance     4,333    7,206      481        0
& Policy
National Technical Assistance         0        0        0        0
except for Support to Policy
and Studies
Technical Assistance, Training   60,016   66,036   61,482   46,321
& Workshops
Vehicles & Office Operting       29,478   28,412   26,638   26,600
Costs
Vehicles, Equip & Materials      14,918   31,000   35,586   33,360
Total                           168,428  173,646  181,037  199,431

                                2014/15  2015/16  Total      % of Total

Civil Works                       2,247   20,226     98,546    6.5%
Contracts for Service               739   13,642     89,048    5.9%
Providers
Dairy Goat Development costs     33,735    9,991     58,406    3.9%
Grant for Dairy Goat
Development for Women             9,389    6,079     45,203    3.0%
Incremental Operating Costs:     29,794   33,050    265,103   17.5%
Salaries & Allowances
International Technical               0        0          0    0.0%
Assistance for DTI
International Technical               0        0          0    0.0%
Assistance for Project
Management & Coordination
National Technical Assistance     7,797        0     31,866    2.1%
& Policy
National Technical Assistance       767        0        767    0.1%
except for Support to Policy
and Studies
Technical Assistance, Training   86,101   77,278    506,757   33.4%
& Workshops
Vehicles & Office Operting       20,104    6,540    199,645   13.2%
Costs
Vehicles, Equip & Materials       9,404    7,784    220,274   14.5%
Total                           200,076  174,591  1,515,615  100.0%

Note: Analyzed from file "SDCP - Total Cumulative Expenditure" provided
from SDCP Technical team.

Appendix Table J2: Dairy Groups Inventory (Revised as of 2011-2012)

County   Division            # of    Total #  Avg. # of
                             Dairy   of       Members
                             Groups  Members

Bomet    Ndarawet             15        316    21
Bomet    Kembu                19        450    24
Bomet    Sugumerga            18        652    36
Nyamira  Borabu               22      2,649   120
Nyamira  Ekerenyo             25        889    36
Nyamira  Nyamira Peri-Urban   24        670    28
Central  Keumbu               20        604    30
Kisii
Central  Mosocho              23        634    28
Kisii
Central  Suneka               25        672    27
Kisii
Nakuru   Subukia-Kabazi       13        360    28
Nakuru   Njoro                12        286    24
Nakuru   Rongai                8        163    20
Lugari   Likuyani             12        258    22
Lugari   Lugari               32        723    23
Lugari   Matete               22        500    23
Nandi    KABISAGA-SIGOT       18        507    28
Nandi    Kapsabet             17        420    25
Nandi    KABIEMITLOLINGERIT   18        425    24
Bungoma  Kanduyi              23        571    25
Bungoma  Ndivisi              24        554    23
Bungoma  Tongaren             23        579    25
Trans    Endebess             20        644    32
Nzoia
Trans    WAITALUK             23        677    29
Nzoia
Trans    Kiminini             18        471    26
Nzoia
Uasin    Kapsaret             14        472    34
Gishu
Uasin    MOI'S BRIDGE         17        389    23
Gishu
Total                        505     15,535    30.8


Appendix K.. do files

(1) The SDCP grouped the counties geographically into clusters to assist programme management

(2) The comparison area for Lugari division was initially Ikolomani. However, when data collection began, we learned that Ikolomani had only a farming cooperative, not farming groups. So, we replaced Ikolomani with Lurambi, the next best match in terms of propensity score.

(3) In Kenya, the following crops are produced during the main rainy season: sorghum, which is harvested by the end of September; beans and millet, which are harvested by the end of October; and maize and wheat, which are harvested by the end of November. Barley, maize, millet, sorghum, and beans are also produced during the short rainy season and are harvested at the end of March. See Kenyan Crop Calendar available at FAO Global Information and Early Warning System Country Brief on Kenya.

(4) These variables were also constructed by ILRI in 2005 at the division level as part of the programme placement exercise, but were not used as selection variables.

(5) Note that for those decision making variables with a low number of females making a given decision, the doubly robust estimator did not converge.

Juan Bonilla, American Institutes for Research (AIR)

Nancy McCarthy, AIR

Simon Mugatha, AIR

Nisha Rai, AIR

Andrea Coombes, AIR

Joshua Brubaker, AIR

Grantee Final Report

Accepted by 3ie: October 2017

Note to readers

This impact evaluation has been submitted in partial fulfilment of the requirements of grant TW4.2.01 awarded under Thematic Window 4. This version is being published online as it was received. A copy-edited and formatted version will be available in the 3ie Impact Evaluation Report Series in the near future. All content is the sole responsibility of the authors and does not represent the opinions of 3ie, its donors or its board of commissioners. Any errors and omissions are the sole responsibility of the authors. All affiliations of the authors listed in the title page are those that were in effect at the time the report was accepted. Any comments or queries should be directed to the corresponding author, Juan Bonilla at jbonilla@air.org.

Suggested citation: Bonilla, J, McCarthy, N, Mugatha, S, Rai, N, Coombes, A and Brubaker, J 2017. Impact evaluation of the smallholder dairy commercialization programme in Kenya, 3ie Grantee Final Report. New Delhi: International Initiative for Impact Evaluation (3ie)

Funding for this impact evaluation was provided by IFAD.

3ie's Agricultural Innovation Thematic Window is supported by the Alliance for Green Revolution in Africa, the Bill & Melinda Gates Foundation, the International Fund for Agricultural Development and UK aid

Acknowledgements

We recognize the contributions of many individuals and organizations, without whom it would not have been possible to complete this study. Our thanks go to the International Initiative for Impact Evaluation (3ie); Smallholder Dairy Commercialization Programme (SDCP); International Fund for Agricultural Development (IFAD); International Livestock Research Institute (ILRI); and Research Solutions Africa for the opportunity to carry out this study and for the financial and technical support that they provided. We further appreciate 3ie for the technical review and support provided throughout the study.

Our special thanks go to Mr. Jasper Grosskurth and Mrs. Jasper Gosselt (Research Solutions Africa) for carrying out the data collection, and to Ms. Pamela Ochungo for her help with recreating the original ILRI targeting maps. The value of the technical and logistical support obtained from Mr. Luke Kessei cannot be overemphasized. Everyone at the SDCP provided valuable logistical support during data collection, including program background information.

Our acknowledgments would be incomplete without mentioning our team of very able research assistants in Kenya. Specifically, we acknowledge the input of the team of enumerators and supervisors from Research Solutions Africa, whose dedication during data collection ensured that high-quality data were collected. The highly competent team of programming and data cleaning personnel at Research Solutions Africa is also gratefully acknowledged.

We also wish to acknowledge the patience exercised by the Kenyan households, community leaders, and community members during interviews. It is our hope that insights from the information they provided will translate into valuable interventions in their communities.
Acronyms

3ie    International Initiative for Impact Evaluation
AI     Artificial Insemination
AIR    American Institutes for Research
ATE    Average Treatment Effect
DCA    Dairy Commercialization Areas
DFID   Department for International Development
FANTA  Food and Nutrition Technical Assistance Project
GDP    Gross Domestic Product
GOK    Government of Kenya
HDDS   Household dietary diversity score
ICC    Intra-cluster correlation
IFAD   International Fund for Agricultural Development
ILRI   International Livestock Research Institute
IPWRA  Inverse-probability-weighted regression adjustment
IRB    Institutional Review Board
MDES   Minimum detectable effect size
M&E    Monitoring & Evaluation
SDCP   Smallholder Dairy Commercialization Programme
T&V    Training and Visit

Table 1: SDCP Divisions and Evaluation Study Areas

County   SDCP Divisions   SDCP Division(s)  Comparison Area Divisions
                          Included in the   Included in the Evaluation
                          Evaluation

Cluster: Bomet, Nyamira, and Central Kisii

Bomet    Bomet Central    Sigor             Ndanai
         Longisa
         Sigor
Nyamira  Borabu           N/A               N/A
         Ekerenyo
         Nyamira
Central  Keumbu           N/A               N/A
Kisii    Mosocho
         Suneka

Cluster: Nakuru

Nakuru   Rongai           Rongai            Kamara
Nakuru   Njoro            Njoro             Keringet
Nakuru   Subukia-Kabazi:  Subukia-Kabazi:   Elburgon and Olenguruone
         Mbogoini and     Mbogoini and
         Bahati           Bahati

Cluster: Lugari, Bungoma, Nandi, Trans Nzoia, and Uasin Gishu

Lugari   Lugari           Lugari            Lurambi (2)
Lugari   Likuyanki        Likuyanki         Ileho
Lugari   Matete           Matete            Kabras
Bungoma  Kanduyi          Kanduyi           Chwele
         Ndivisi          N/A               N/A
         Tongaren
Nandi    Kabiyet
         Kapsabet
         Kilibwoni
         Kosirai
Trans    Endebess         N/A               N/A
Nzoia    Kiminini
Uasin    Kapsaret         N/A               N/A
Gishu    Soy
         Turbo

Table 2: Qualitative Sampling

Stakeholder Group         Geographical  Number of              Component
                              Level     Interviews               Focus

SDCP Coordinating Unit    National      4                      5 (1, 2)
Dairy Training Institute  National      1                      3
Kenya Animal Genetic      National      1                      3
Resources Centre
Department of Livestock   Regional      3                      3, 4
Community Artificial      County        Nakuru, 2; Lugari, 1   3
Insemination Station
Input Supplier            County        Nakuru, 1; Lugari, 1   3, 4
County Programme          County        Nakuru, 2; Lugari, 3   5 (1, 2)
Coordination Team
Treatment Dairy Group     County        5                      1, 2
Chairpersons
Treatment Smallholder     County        12 (1 man, 1 woman     1, 2
Dairy Groups                            for six dairy groups)
Comparison Dairy Group    County        1                      1, 3
Chairpersons
Comparison Dairy Groups   County        4 (1 man, 1 woman      1, 3
                                        for two dairy groups)

Table 3: Pre-Programme, Division-Level Characteristics used for
Matching (N=183)

Placement          Mean       Mean       Coefficient  Standard  p-value
variables          control    treatment               error

Milk density       85,389     81,099     -0.000008    0.000009  0.398
(litres
/k[m.sup.2]/year)
Proportion of           0.53       0.53   0.15        3.49      0.967
poor people
Travel time to          0.39       0.40  -0.55        0.92      0.557
nearest urban
centre

Table 4: Determinants of treatment probability

Independent Variable               Mean     Coefficient  SE     p-value

Household demographics

Main language is English = 1          0.08  -0.011       0.028  0.70
No. people in working-age group       2.94  -0.005       0.006  0.43
No. children 5 years of age and       0.48   0.017       0.012  0.14
younger
Age of household head                52.3    0.003       0.001  0.00
Max. years of schooling              13.0    0.003       0.004  0.46
Female head of household = 1          0.24   0.112       0.025  0.00

Income and agriculture
characteristics

Consumer durables index (SD)          0.00   0.018       0.010  0.08
Ag. Implements index (SD)             0.00   0.019       0.008  0.01
Total size of landholdings (acres     0.19   0.015       0.027  0.59
purchased with title)
Farm is flat or with slight           0.87  -0.008       0.028  0.77
slope = 1
Plot with any system of               0.02  -0.082       0.075  0.28
irrigation = 1
Plot with any irrigation              0.02  -0.057       0.079  0.47
structure = 1
Household cultivated any crop in      0.88   0.100       0.033  0.00
primary season
No. Zebu cattle in division in     1071.1   -0.001       0.000  0.00
2005
No. Cross cattle in division in    1388.2    0.001       0.000  0.00
2005
No. grade cattle in division in     959.1    0.00003     0.000  0.83
2005

Development projects in community

Any agriculture-based project         0.55  -0.002       0.088  0.99
other than SDCP = 1
Any livestock-based project = 1       0.12   0.078       0.090  0.39
N                                  2558

Notes: This table shows the results of a linear model for the
probability of being in the treatment group as a function of household,
agricultural, and community characteristics. The table shows the mean
for each explanatory variable, the change in the propensity of being in
the treatment group after a change of one unit on a given
characteristic, the standard error of each estimate, and the
corresponding p-value.

Table 5: Covariate balance tests over treatment groups with
inverse-probability weighting

Dependent Variable                   Mean   Coefficient  SE     p-value

Household demographics

Main language is English = 1          0.08  -0.015       0.017  0.365
Number of people in the working       2.94  -0.105       0.098  0.285
age group
Number of children 5 years of age     0.48  -0.004       0.032  0.903
and younger
Age of household head                52.3    0.002       0.012  0.838
Max. years of schooling of           13.0   -0.140       0.163  0.390
household member
Female head of household              0.24   0.002       0.020  0.914

Income and agriculture
characteristics

Consumer durables index (SD)          0.00  -0.083       0.088  0.344
Ag. Implements index (SD)             0.00  -0.069       0.086  0.422
Size of landholdings (acres           0.19   0.004       0.016  0.782
purchased with title)
Household cultivated any crop in      0.88  -0.005       0.015  0.739
primary season
Household cultivated any crop in      0.02  -0.007       0.022  0.737
secondary season
Area cultivated in primary season     0.22  -0.013       0.021  0.552
(log of ha)
Area cultivated in secondary season   0.05  -0.011       0.021  0.599
(log of ha)
Farm is flat or with slight           0.87  -0.006       0.015  0.683
slope = 1
Any system of irrigation = 1          0.02  -0.009       0.012  0.430
Any irrigation structure = 1          0.02  -0.009       0.011  0.444

Development projects in community

Community has any agriculture-based   0.55  -0.007       0.022  0.768
project other than SDCP = 1
Any project in community is focused   0.12  -0.014       0.017  0.412
on livestock

Notes: Each row in this table corresponds to an OLS regression of the
specified variable on the treatment indicator. The table shows the mean
for each dependent variable, the difference in means between the
treatment and control groups (coefficient), the standard error for each
difference, and the corresponding p-value.

Table 6: Availability of Other Agricultural and Livestock Support
Programs

Characteristics                                     SDCP  Control  All

Community has any agriculture-based project other   0.56  0.54     0.55
than SDCP
Any project in community is focused on agriculture  0.42  0.45     0.43
Any project in community is focused on livestock    0.15  0.09     0.12

Table 7: Impacts on feeding practices

Dependent Variable            Impact        Comparison  Treatment  N
                              Estimate      Mean        Mean
                              (1)           (2)         (3)

Practice zero grazing           0.08 (***)  0.09        0.17       2558
                               (0.02)
Paddock/boma/stall floor        0.05 (***)  0.07        0.12       2558
made of concrete               (0.02)
Purchase protein-rich fodder    0.01        0.02        0.03       2558
                               (0.01)
Feed cattle crop residues      -0.00        0.84        0.83       2558
                               (0.03)
Feed cattle concentrate         0.13 (***)  0.66        0.79       2558
feeds/mineral supplements      (0.04)
Feeding off the ground in       0.12 (***)  0.18        0.29       2558
trough when milking            (0.03)
Vary feeding by lactation       0.03        0.55        0.58       2558
stage                          (0.04)
Experience shortage of feeds   -0.01        0.74        0.73       2558
from their farm               (-0.03)

Notes: Impacts are estimated using the inverse-probability-weighted
regression adjustment (Doubly-robust estimator). Robust standard errors
are in parentheses. All estimations control for household and household
head characteristics; plot and crop production characteristics for
primary and secondary rainy seasons; availability of other agricultural
and livestock support programs in village; pre-program division-level
characteristics including milk density in liters per square kilometers,
proportion of people below the poverty line, travel time to the nearest
cooling center and to nearest urban center. (***) p<0.01; (***) p<0.05;
(***) p<0.1

Table 8: Impacts on access and use of livestock technology

Dependent Variable            Impact       Comparison  Treatment  N
                              Estimate     Mean        Mean
                              (1)          (2)         (3)

Household monitors cattle      0.07 (***)  0.88        0.94       2558
regularly                     (0.02)
Technology for own bull       -0.02        0.29        0.27       2558
service is available          (0.04)
Own Bull Service used in the  -0.01        0.13        0.12       2558
last 12 months                (0.03)
Technology for AI service is   0.05        0.60        0.65       2558
available                     (0.04)
AI Service used in the last    0.12 (***)  0.24        0.36       2558
12 months                     (0.03)

Notes: Impacts are estimated using the inverse-probability-weighted
regression adjustment (Doubly-robust estimator). Robust standard errors
are in parentheses. All estimations control for household and household
head characteristics; plot and crop production characteristics for
primary and secondary rainy seasons; availability of other agricultural
and livestock support programs in village; pre-program division-level
characteristics including milk density in liters per square kilometers,
proportion of people below the poverty line, travel time to the nearest
cooling center and to nearest urban center. (***) p<0.01; (***) p<0.05;
(***) p<0.1

Table 9: Impacts on animal health services

Dependent Variable           Impact       Comparison  Treatment  N
                             Estimate     Mean        Mean
                             (1)          (2)         (3)

Had access to anthelmintics   0.02        0.90        0.92       2558
                             (0.02)
Used anthelmintics            0.02        0.89        0.91       2558
                             (0.02)
Had access to tick control    0.00        0.89        0.89       2558
service                      (0.03)
Used tick control service     0.00        0.88        0.89       2558
                             (0.03)
Had access to vaccination     0.25 (***)  0.46        0.71       2558
service                      (0.04
Used vaccination service      0.26 (***)  0.43        0.68       2558
                             (0.04)
Had access to curative        0.10 (***)  0.21        0.30       2558
treatment service            (0.04)
Used curative treatment       0.07 (**)   0.21        0.28       2558
service                      (0.03)

Notes: Impacts are estimated using the inverse-probability-weighted
regression adjustment (Doubly-robust estimator). Robust standard errors
are in parentheses. All estimations control for household and household
head characteristics; plot and crop production characteristics for
primary and secondary rainy seasons; availability of other agricultural
and livestock support programs in village; pre-program division-level
characteristics including milk density in liters per square kilometers,
proportion of people below the poverty line, travel time to the nearest
cooling center and to nearest urban center. (***) p<0.01; (***) p<0.05;
(***) p<0.1

Table 10: Impacts on availability on cattle extension services

Dependent Variable            Impact       Comparison  Treatment  N
                              Estimate     Mean        Mean
                              (1)          (2)         (3)

Extension visits available     0.14 (***)  0.19        0.33       2558
                              (0.03)
Field days available           0.18 (***)  0.18        0.36       2558
                              (0.04)
Demonstrations are available   0.12 (***)  0.14        0.27       2558
                              (0.03)

Notes: Impacts are estimated using the inverse-probability-weighted
regression adjustment (Doubly-robust estimator). Robust standard errors
are in parentheses. All estimations control for household and household
head characteristics; plot and crop production characteristics for
primary and secondary rainy seasons; availability of other agricultural
and livestock support programs in village; pre-program division-level
characteristics including milk density in liters per square kilometers,
proportion of people below the poverty line, travel time to the nearest
cooling center and to nearest urban center. (***) p<0.01; (***) p<0.05;
(***) p<0.1

Table 11: Impacts on probability of receiving information from
trainings

Dependent Variable             Impact       Comparison  Treatment  N
                               Estimate     Mean        Mean
                               (1)          (2)         (3)

Crop best practices             0.04        0.17        0.21       2558
                               (0.03)
Livestock best practices        0.15 (***)  0.22        0.37       2558
                               (0.03)
Improved fermentation of milk   0.06 (***)  0.01        0.07       2558
best practices                 (0.01)
Other milk processing and       0.10 (***)  0.03        0.13       2558
quality control best           (0.01)
practices
Managerial, bookkeeping,        0.04 (***)  0.03        0.07       2558
accounting, and finance        (0.01)
Fodder Establishment            0.07 (***)  0.05        0.13       2558
                               (0.02)
Hay Making                      0.06 (***)  0.06        0.12       2558
                               (0.02)
Silage Making                   0.04 (**)   0.05        0.09       2558
                               (0.02)
Use of chaff cutter             0.05 (***)  0.01        0.05       2558
                               (0.01)
Conservation (crop residues)    0.03 (***)  0.02        0.04       2558
                               (0.01)
Animal registration             0.03 (***)  0.01        0.04       2558
                               (0.01)
Fresh milk marketing            0.07 (***)  0.02        0.09       2558
                               (0.01)
Value addition marketing        0.05 (***)  0.01        0.06       2558
(e.g., mala, yoghurt)          (0.01)
Group/Cooperative Milk          0.02 (**)   0.02        0.04       2558
Marketing                      (0.01)
Market Information Searching    0.02 (***)  0.01        0.02       2558
                               (0.01)

Notes: Impacts are estimated using the inverse-probability-weighted
regression adjustment (Doubly-robust estimator). Robust standard errors
are in parentheses. All estimations control for household and household
head characteristics; plot and crop production characteristics for
primary and secondary rainy seasons; availability of other agricultural
and livestock support programs in village; pre-program division-level
characteristics including milk density in liters per square kilometers,
proportion of people below the poverty line, travel time to the nearest
cooling center and to nearest urban center. (***) p<0.01; (***) p<0.05;
(***) p<0.1

Table 12: Impacts on number of animals

Dependent Variable            Impact        Comparison  Treatment  N
                              Estimate      Mean        Mean
                              (1)           (2)         (3)

Number of cattle owned          0.49 (***)  2.31        2.80       2558
                               (0.15)
  Number of exotic breed        0.14        1.18        1.14       2558
  cattle owned                (-0.28)
  Number of crossed breed       0.29 (**)   0.68        0.97       2558
  cattle owned                 (0.14)
Number of cows owned            0.35 (***)  0.95        1.31       2558
                               (0.08)
Number of milking cows          0.44 (***)  0.57        1.01       2558
                               (0.07)
Number of exotic breed cows     0.03        0.57        0.60       2558
owned                          (0.08)
Number of crossed breed cows    0.16 (***)  0.28        0.44       2558
owned                          (0.06)

Notes: Impacts are estimated using the inverse-probability-weighted
regression adjustment (Doubly-robust estimator). Robust standard errors
in parentheses. All estimations control for household and household
head characteristics; plot and crop production characteristics for
primary and secondary rainy seasons; availability of other agricultural
and livestock support programs in village; pre-program division-level
characteristics including milk density in liters per square kilometers,
proportion of people below the poverty line, travel time to the nearest
cooling center and to nearest urban center. (***) p<0.01; (***) p<0.05;
(***) p<0.1

Table 13: Impacts on milk production and total value

Dependent Variable            Impact       Comparison  Treatment  N
                              Estimate     Mean        Mean
                              (1)          (2)         (3)

Sold milk yesterday any time   0.08 (**)   0.42        0.50       2558
                              (0.04)
Sold milk in the morning       0.07        0.41        0.48       2558
                              (0.04)
Sold milk in the evening       0.06 (**)   0.20        0.26       2558
                              (0.03)
Total litres of milk sold in   0.12        0.86        0.98       2558
the morning (yesterday)       (0.10)
Total litres of milk sold in   0.11        0.38        0.49       2558
the evening (yesterday)       (0.07)
Total litres of milk sold      0.18        0.96        1.14       2558
(yesterday)                   (0.11)
Selling price of milk/litre    0.31 (*)    1.81        2.13       2558
(yesterday)                   (0.18)
Total value of milk sold       0.43 (*)    2.47        2.90       2558
                              (0.25)
Total milk production at       0.58 (***)  1.33        1.92       2558
calving                       (0.12)
Total milk production          0.37 (***)  1.13        1.50       2558
(yesterday)                   (0.11)

Notes: Impacts are estimated using the inverse-probability-weighted
regression adjustment (Doubly-robust estimator). Robust standard errors
are in parentheses. All estimations control for household and household
head characteristics; plot and crop production characteristics for
primary and secondary rainy seasons; availability of other agricultural
and livestock support programs in village; pre-program division-level
characteristics including milk density in liters per square kilometers,
proportion of people below the poverty line, travel time to the nearest
cooling center and to nearest urban center. (***) p<0.01; (***) p<0.05;
(***) p<0.1. All production values and selling price are expressed in
logs. Total milk production includes milk sold, consumed in the
household, and lost.

Table 14: Impacts on food categories consumed in last 7 days

Dependent Variable             Impact       Comparison  Treatment  N
                               Estimate     Mean        Mean
                               (1)          (2)         (3)

Cereals/grains                 -0.00        0.99        0.98       2558
                               (0.01)
Potatoes/yams/cassava          -0.14 (***)  0.89        0.75       2558
                               (0.03)
Vegetables                      0.00        0.99        0.99       2558
                               (0.01)
Fruits                         -0.11 (***)  0.89        0.78       2558
                               (0.03)
Beans/peas/lentils/nuts         0.03 (*)    0.88        0.91       2558
                               (0.02)
Red meats/other organ meats     0.17 (***)  0.36        0.54       2558
                               (0.04)
Poultry                         0.00        0.35        0.36       2558
                               (0.03)
Eggs                            0.02        0.60        0.62       2558
                               (0.04)
Fresh/dried fish/shellfish     -0.02        0.37        0.35       2558
                               (0.03)
Milk/cheese/yogurt/other milk   0.09 (***)  0.80        0.89       2558
product                        (0.03)
Oils and fats                   0.02        0.88        0.90       2558
                               (0.03)
Sweets/sugar/honey              0.03        0.91        0.95       2558
                               (0.02)
Other (condiments, coffee,      0.06 (**)   0.89        0.95       2558
tea)                           (0.02)
Household dietary diversity     0.16        9.81        9.97       2558
score (HDDS)                   (0.14)

Notes: Impacts are estimated using the inverse-probability-weighted
regression adjustment (Doubly-robust estimator). Robust standard errors
are in parentheses. All estimations control for household and household
head characteristics; plot and crop production characteristics for
primary and secondary rainy seasons; availability of other agricultural
and livestock support programs in village; pre-program division-level
characteristics including milk density in liters per square kilometers,
proportion of people below the poverty line, travel time to the nearest
cooling center and to nearest urban center. (***) p<0.01; (***) p<0.05;
(***) p<0.1

Table 15: Impacts on women decision making outcomes

Dependent Variable            Impact       Comparison  Treatment  N
                              Estimate     Mean        Mean
                              (1)          (2)         (3)

Female manages money from      0.09 (*)    0.21        0.30       2558
fresh milk sold               (0.05)
Female decides own bull        0.07 (*)    0.11        0.18       2558
service provider              (0.04)
Female decides AI service      0.11 (***)  0.03        0.15       2558
provider                      (0.03)
Female requested livestock     0.11 (***)  0.10        0.20       2558
best practices training       (0.03)
Female requested cattle best   0.06        0.17        0.23       2558
practices training            (0.04)
Female decides use of          0.27 (***)  0.17        0.45       2558
anthelmintic                  (0.05)
Female decides use of tick     0.19 (***)  0.21        0.40       2558
control service               (0.06)
Female decides use of          0.19 (***)  0.11        0.31       2558
vaccination service           (0.05)
Female decides use of          0.05 (*)    0.06        0.11       2558
curative treatment service    (0.03)

Notes: Impacts are estimated using the inverse-probability-weighted
regression adjustment (Doubly-robust estimator). Robust standard errors
are in parentheses. All estimations control for household and household
head characteristics; plot and crop production characteristics for
primary and secondary rainy seasons; availability of other agricultural
and livestock support programs in village; pre-program division-level
characteristics including milk density in liters per square kilometers,
proportion of people below the poverty line, travel time to the nearest
cooling center and to nearest urban center. (***) p<0.01; (***) p<0.05;
(***) p<0.1
COPYRIGHT 2017 International Fund for Agricultural Development (IFAD)
No portion of this article can be reproduced without the express written permission from the copyright holder.
Copyright 2017 Gale, Cengage Learning. All rights reserved.

Article Details
Printer friendly Cite/link Email Feedback
Author:Bonilla, Juan; McCarthy, Nancy; Mugatha, Simon; Rai, Nisha; Coombes, Andrea; Brubaker, Joshua
Publication:IFAD Impact Assessment Series
Date:Oct 1, 2017
Words:25664
Previous Article:Democratic Republic of Sao Tome e Principe: Smallholder Commercial Agriculture Project (PAPAC) and Participatory Smallholder Agriculture and...
Next Article:Ethiopia: Participatory Small Irrigation Development Programme I (PASIDP I) Results from a High Frequency Data Collection.
Topics:

Terms of use | Privacy policy | Copyright © 2019 Farlex, Inc. | Feedback | For webmasters