Impact evaluation of the smallholder dairy commercialization programme in Kenya.
Contents Page Acknowledgements i Acronyms ii Plain Language Summary iii Introduction 1 Intervention, Theory of Change, and Research Hypotheses 3 Development of Intervention 3 Theory of Change 5 Primary Outcomes and Impacts of Interest 9 Context 10 Timeline 16 Evaluation: Design, Methods, and Implementation 16 IRB 16 Quantitative Design 16 Qualitative Design 20 Limitations of the Study 23 Programme or Policy: Design, Methods, and Implementation 23 Impact Analysis and Results of Key Evaluation Questions 24 Descriptive Statistics and Quality of Counterfactual 24 Mixed Methods Analysis (Quantitative and Qualitative) 28 Intermediate Outcomes 29 Main Outcomes 36 Gender Differential Effects 39 Cost-Analysis 41 Discussion and Conclusions 42 Internal and External Validity 46 References 48 Appendix A Field notes and other information from formative work 1 Appendix C Survey instruments (qualitative and quantitative) 1 Appendix E Sample Size and Power Calculations 1 Appendix J Cost Data for the Programme 1 Appendix K do files 1 Tables Page Table 1: SDCP Divisions and Evaluation Study Areas 14 Table 2: Qualitative Sampling 21 Table 3: Pre-Programme, Division-Level Characteristics used for Matching (N=183) 25 Table 4: Determinants of treatment probability 26 Table 5: Standardized differences in observable characteristics with and without weights Error! Bookmark not defined. Table 6: Availability of Other Agricultural and Livestock Support Programs 28 Table 7: Impacts on feeding practices 30 Table 8: Impacts on access and use of livestock technology 31 Table 9: Impacts on animal health services 32 Table 10: Impacts on availability on cattle extension services 32 Table 11: Impacts on probability of receiving information from trainings 33 Table 12: Impacts on number of animals 36 Table 13: Impacts on milk production and total value 37 Table 14: Impacts on food categories consumed in last 7 days 38 Table 15: Impacts on women decision making outcomes 40 Table 16: Impacts on milk production and total value (female household heads only) Error! Bookmark not d Appendix Table J1 SDCP Yearly Expenditure by Category (in thousands of KSH) Appendix Table J2: Dairy Groups Inventory (Revised as of 2011-2012) Exhibits Page Exhibit 1: Recreation of Original Programme Targeting 12 Exhibit 2: Evaluation Treatment and Comparison Divisions and Non-Evaluation Treatment Divisions 15 Exhibit 3: Balance Plot 27
Plain Language Summary
The dairy sector in Kenya is one of the largest and most developed in Sub-Saharan Africa, accounting for four percent of its gross domestic product. Despite high production volumes, the sector is still dominated by smallholder farmers who rely on livestock for income and food security. Dairy farmers face a number of barriers to increased profitability, including animal diseases, lack of access to artificial insemination and other veterinary services, high costs of improved technologies, such as silage equipment, inadequate access to markets, poor rural infrastructure and unsteady supply of quality animal fodder.
The Smallholder Dairy Commercialisation Programme (SDCP) was funded by the International Fund for Agricultural Development (IFAD) and implemented by the Government of Kenya from 2005-2015. It was designed to reach 600 dairy groups (24,000 smallholder dairy farmers) in nine counties. SDCP provided training to dairy farmers to build their enterprise, managerial and organisational skills. Aside from training, the programme also aimed to enhance dairy farming productivity and reduce production costs through demonstration, field days and grants. To strengthen market linkages, SDCP invested in improving road infrastructure and conducted additional training on milk-handling practices and value-added opportunities.
The programme identified three main areas where barriers to improving dairy income potentially operate: dairy group activities, household production and market intermediaries. Programme designers hypothesised that increasing net dairy income for smallholder farmers can occur through four primary contextual factors (1) increasing milk production; (2) increasing milk prices; (3) decreasing the costs of producing milk; and (4) decreasing the transaction costs of participation in input and output markets. They assumed that increased net income will lead to improved food security and increased participation by women and marginalised communities.
Impact evaluation overview
A key objective of agricultural extension is to increase farmers' knowledge about agricultural practices, which in turn could have an effect on productivity. Evaluating the impact of the SDCP can provide useful insights for the design of other agricultural extension programmes for smallholder farmers in developing countries. 3ie funded this grant under 3ie's Agricultural Innovation Thematic Window. The main evaluation questions were the following:
1. Does SDCP improve the well-being of dairy farmers through improved animal management, improved efficiency, increased production and productivity, and increase in farmer incomes and food security?
2. Are there differences in the participation in programme activities between female and male dairy farmers? Does the programme have sex-differentiated effects?
3. How does the process by which SDCP is implemented influence the effectiveness of the programme? Given this process, how do contextual factors affect programme success?
Methodology and identification strategy
The ex-post evaluation used a quasi-experimental design using propensity score matching. The sample of 2,500 famers was split equally between 1,250 SDCP beneficiaries and 1,250 non-programme participants. Key information was collected through a survey (conducted by Research Solutions Africa) and the programme causal chain, including the underlying assumptions, was mapped out. To capture key time-invariant characteristics, as well as retrospective information on dairy farming, the study included a short filter questionnaire at the beginning of the household survey for accurately matching households in the treatment group with households in the comparison group.
The study used qualitative data to explain the quantitative findings and understand the implementation of the programme. It compared accounts of local government officials with that of dairy farmers, by using semi-structured interviews and focus group discussions.
The findings suggest that the SDCP was successful in increasing milk production but had limited impacts on increasing either the quantity of milk sold or prices received by famers. Regarding intra-household bargaining power, SDCP farmer households were more likely to have women managing cash from the sale of milk relative to non-SDCP households. This was also found to be true for decisions relating to use of services such as artificial insemination, anthelmintic drugs, tick control, vaccination and curative treatments.
Qualitative research highlighted the challenges in programme implementation, such as those related to linking dairy groups to various service providers and limited knowledge among farmers to negotiate terms favourable to them.
We calculate that it would take approximately 4.74 years for the program to "break even," that is for the benefits to equal the costs, assuming the benefits of increased milk production remain constant across years. This estimate of the number of years to break even seems reasonable for although there likely were no benefits at the beginning of the programme (when upfront costs were associated with setting-up the SDCP), farmers likely began benefiting from the programme prior to 2016. Furthermore, to the extent farmers continue to employ the best practices advocated by the programme, the benefits may extend into future years.
The International Initiative for Impact Evaluation (3ie) on behalf of the International Fund for Agricultural Development (IFAD) contracted the American Institutes for Research (AIR), in partnership with Lead Analytics and Research Solutions Africa to evaluate the Smallholder Dairy Commercialization Programme (SDCP) in Kenya.
The Smallholder Dairy Commercialisation Programme (SDCP), implemented by the Government of Kenya (GoK) from 2005 to 2015, was designed to reach Dairy Groups engaged in milk production in nine milk-producing counties. The overall goal of SDCP was to increase the income of poor rural households that depend substantially on production and trade of dairy products for their livelihoods. The SDCP was implemented through various interrelated components. First, SDCP provided training on organisational, managerial, and enterprise skills (e.g., bookkeeping, accounting, financial planning) to farmers. Second, the programme targeted household production by aiming to enhance dairy farming productivity and reduce production costs through grants, trainings, field days, and demonstrations. Lastly, SDCP aimed to strengthen market linkages for small-scale milk producers by improving road infrastructure and conducting additional trainings on milk handling practices and value-addition opportunities.
Many researchers, nongovernmental organizations, governments, and donors have long held the position that smallholder dairy can be a particularly effective mechanism for alleviating poverty and increasing food security in regions well-suited for dairy production, such as those located in western Kenya (Staal, et al., 1997; Thorpe, Muriuki, Omore, Owango, & Staal, 2000; Burke et al., 2007). While there had been some successes prior to 2006 in promoting the production and marketing of milk in these types of regions in Kenya, the SDCP was designed to address a number of barriers to increased profitability of smallholder dairy farming in the country, including relatively high transactions costs for production and marketing (Staal, et al., 1997 (World Development)), and under performance of dairy groups and cooperatives in reducing those transactions costs (Holloway et al., 2000 (Agricultural Economics, 23(3): 279-288); Atieno and Kanyinga, 2008). However, empirical evidence on the impacts of reducing transactions costs--either through analyses of household surveys or through rigorous impact assessments of specific projects--remains relatively limited. A few studies report results that would suggest that reducing transactions costs will improve farm productivity and incomes, primarily by increasing access to improved production methods and marketing information and by being able to link to private sector actors in the value chain. For instance, Burke et al. (2015) find that smallholders closer to electricity sources and with access to private sector value chain actors are more likely to be dairy producers and net sellers, and that "active" dairy cooperatives also induce greater participation by smallholders in dairy markets. Gelan and Muriithi (2015), and references cited therein, find that practices such as zero grazing and adoption of improved cows, led to increased milk production efficiency, while Nafula (2013) provides evidence of the importance of enhanced nutritional and feed practices on the health of dairy calves using a randomized control trial method. The SDCP project promotes all of these activities and also seeks to reduce marketing-related transactions costs.
Understanding the impacts of the SDCP is particularly valuable given the importance of the dairy sector in Kenya. The dairy sector in Kenya is one of the largest and most developed in Sub-Saharan Africa, accounting for 4% of the country's gross domestic product (GDP). Despite high production volumes, the dairy industry is dominated by smallholder farmers who rely on livestock for income and food security. However, smallholder dairy farmers face a number of barriers to increased profitability, including steady supplies of quality animal fodder and feed throughout the year; animal diseases; inadequate access to markets; artificial insemination (AI) services and other veterinary services; poor rural infrastructure; high costs of improved technologies, such as silage equipment; limited skills with which to bargain with input suppliers and output purchasers; limited knowledge to run their dairy activities as a commercially oriented enterprise, including maintaining animal health and providing high-quality milk; and dependency on traders and limited ability to maintain milk quality, putting downward pressure on farmgate milk prices.
While the project undertook a study of the programme in the form of household surveys at the end of the project in 2015, no formal impact assessment had previously been conducted. Specifically, the 2015 survey did not include households from "control" areas. Instead, the questionnaire included retrospective questions, going back many years. Thus, it is possible that findings from the 2015 survey might suffer from recall bias and from lack of data from households on which to construct a counterfactual. The study AIR and Lead Analytics have conducted is the first rigorous impact assessment designed to examine the effects of the SDCP. AIR and Lead Analytics designed the evaluation to address the knowledge gaps that related to project's impacts on smallholder dairy producers, including measures of production efficiency and profitability, as well as impacts of project activities aimed at ensuring that women and the resource-poor farmers benefitted from the project.
The study investigated programme impacts and implementation of the SDCP. Our main research question was whether SDCP improves the wellbeing of dairy farmers through improved animal management, increased production and productivity, improved efficiency (e.g., input and transaction cost reductions), and a rise in farmer incomes and food security. We also investigated whether there are differences in programme participation and effects by the gender of the farmer. Lastly, we looked at the process by which SDCP is implemented and how contextual factors may affect programme success.
We addressed these research questions through a mixed methods design. Quantitatively, we used a quasi-experimental approach using a matching ex-post design. Since programme implementation is complete, an ex-post analysis was appropriate to inform future scale up of similar efforts. Our design involved two steps of matching at the division and then household level. At the division level, we relied heavily on a two-step targeting approach. First, we used existing administrative information from the original programme targeting to identify similar nonprogramme areas that were not affected by the SDCP due to capacity constraints (observation-based targeting). Second, we discuss the filtered control communities with local experts and stakeholders to determine which of the preselected control areas were more similar to treatment communities at project inception (criteria-based targeting). One of the major benefits of the criteria-based targeting exercise is that, based upon the insight of the experts, we were able to exclude divisions where other policies or actions may have had differential effects over time, which is especially beneficial since this is an ex-post evaluation for a long-running programme. Furthermore, we matched treatment and comparison divisions within the same counties, which helps ensure that both study areas received the same level of support from their county over time, especially after devolution of power to the county, which occurred in Kenya in 2013 At the household level, to estimate programme impacts, we further used matching through our use of the doubly robust estimator, which combines regression and propensity score methods, to achieve some robustness to mis specification of the parametric models (Imbens & Wooldridge, 2009). Combining regression and weighting (with weights derived from the probability of being part of the program) can lead to additional robustness by removing the correlation between omitted covariates and by reducing the correlation between omitted and included variables. We complemented these analyses with qualitative research in the form of focus group discussions and key informant interviews.
Our findings indicate that the programme had statistically significant positive impacts on improved animal management, including grazing and feeding practices, keeping practices, and animal health services. These results suggest that farmers were receptive to programme education regarding the nutritional content and feeding practices, which may have a positive effect on milk production. The SDCP also led to an improvement in breeding services. Specifically, the SDCP increased the propensity of households monitoring their cattle on a regular basis and the use of Artificial Insemination (AI) services. The positive quantitative findings related to access to and use of AI are especially promising considering that many farmers discussed ongoing challenges with AI. We found that SDCP treatment farmers have improved access to extension visits, field days, and demonstrations, services which the programme provided. We also found an increase in the probability of receiving any information on specific aspects of the production process. The largest increase in probabilities was for receiving information on general livestock practices, milk processing and quality control, and fodder establishment and fresh milk marketing, although positive impacts are observed for almost all type of cattle-related topics, as well as topics related to enterprise skill. The results also suggest that the SDCP was successful in increasing milk production, but with more limited --though positive--impacts on increasing milk marketing and increasing milk prices received by smallholders.
The report is structured as follows. First, we describe the intervention and research hypotheses. Then we describe the context and the timeline for the project and evaluation. We describe the evaluation's and the policy's design, methods, and implementation. We then present the impact analysis followed by a discussion and recommendations for policy and practice.
Intervention, Theory of Change, and Research Hypotheses
Development of Intervention
During the 1990s, the monopolistic dairy cooperative structure in Kenya collapsed. While plagued by inefficiencies, the structure provided some support to dairy farmers. The collapse led to great distrust of new collectives despite the potential gains from collective action in the sector and due to economies of scale in linking smallholders to input suppliers and milk purchases (e.g., through bulk purchasing of fodder and bulk milk sales). At the outset of the project in 2006, the GoK was restructuring at least some of its legislation related to smallholder participation in dairy markets, with the aim of reducing costs that smallholders face in joining formal markets. One of the project's goals was to work with the government and relevant ministries to further enhance the policy and legislative environments for smallholders, primarily with respect to issues dealing with animal breeding.
The project was built on empirical evidence on smallholder dairy systems produced by the International Livestock Research Institute (ILRI), with support from Department for International Development (DFID). Given evidence on the level of dairy commercialization potential, smallholder dairy producers, poverty rates, and indicators of rural infrastructure, the project selected to work in nine districts in central and central-western Kenya. Smallholders dominate the production of milk in these districts, which also exhibit high-poverty rates. At the same time, smallholder dairy farmers face a number of barriers to increased profitability, including steady supplies of quality animal fodder and feed throughout the year; animal diseases; inadequate access to markets; AI services and other veterinary services; poor rural infrastructure; high costs of improved technologies, such as silage equipment; limited skills with which to bargain with input suppliers and output purchasers; limited knowledge to run their dairy activities as a commercially oriented enterprise, including maintaining animal health and providing high-quality milk; and depending on traders with limited ability to maintain milk quality, putting downward pressure on farmgate milk prices. Additionally, women play a key role in the smallholder dairy sector. Research showed that many female-headed households in these districts had dairy cows, and even in male-headed households, females controlled over 60% of the income from dairy activities. Women dairy farmers have been traditionally even further disadvantaged in terms of receiving extension advice and playing leadership roles in dairy groups to ensure their specific needs are addressed.
The SDCP--funded by IFAD, the GoK, and the local community, and implemented by the GoK from 2005 to 2015--was designed to address some of the constraints faced by smallholder dairy farmers. The overall goal of SDCP was to increase the income of poor rural households that depend substantially on production and trade of dairy products for their livelihoods. The SDCP was implemented through various interrelated components. First, the programme provided dairy group activities by training beneficiaries on organisational, managerial, and enterprise skills (e.g., bookkeeping, accounting, financial planning) to fully benefit from market-driven milk commercialisation. Further, capacity building of Dairy Groups was complemented by competitive access to investment grants for improving forage production and feed milling; milk bulking, chilling, and processing; and management and market information systems. Second, the programme targeted household production by aiming to enhance dairy farming productivity and to reduce production costs through trainings, field days, and demonstrations. Lastly, SDCP aimed to strengthen relationships with market intermediaries by enhancing market linkages for small-scale milk producers by improving road infrastructure and conducting additional trainings to beneficiaries on milk handling practices and value-addition opportunities.
The primary beneficiaries of the project were resource-poor smallholder dairy farmers, with an emphasis on ensuring women's participation in all project activities. The project intended to reach 24,000 smallholder dairy farming households with members participating in 600 dairy groups across nine milk-producing counties. IFAD and GoK determined the target of 600 dairy groups as a number large enough for an impact but still within capacity constraints. Based on dairy group inventories as of 2011 and 2012, the project ended up working with 15,535 smallholder dairy farming households with members from 505 dairy groups across the nine counties. In addition to resource-poor smallholders (85% of targeted beneficiaries), the project also targeted smallholders already engaging in more intensive production. Finally, the project also targeted 300 milk traders and 90 milk bar/milk processors to improve milk quality and contractual arrangements with buyers and sellers.
Theory of Change
The project first reviewed the substantial evidence on factors associated with low milk production, productivity per cow, and relatively low participation in milk markets with marked seasonal fluctuations. The project identified three main areas where impediments to improving dairy incomes operated: dairy group activities, household production, and market intermediaries. These three areas conform to three of the five project components, with support to policy and institutions and to project management comprising the fourth and fifth components.
Increasing dairy incomes for smallholders can occur through three primary channels: increasing milk production, increasing prices received for milk sold, and decreasing costs of producing and marketing milk. Here we focus on the first three components, as these components address all of the channels to varying degrees.
Component 4 (support to policy and institutions) concerns activities to shape national-level policies and regulations, primarily concerned with regulating AI, registering (improved) breed births, and milk-related phytosanitary regulations. To the extent that these have been adopted and implemented, they would affect all dairy cattle owners in Kenya, that is dairy cattle owners in both the treatment and comparison groups in this study. Since both groups would benefit from these policy and regulatory changes, it will be impossible to evaluate these impacts using the treatment and comparison households. Component 5 (support to project management) affects only treated households, so it is more of an IFAD-specific procedure than a separate activity. In fact, most development projects do not put project management as a separate component of activities.
In other words, all beneficiaries receive all five components, whereas comparison farmers would receive benefits from Component 4. Since beneficiaries receive all five components, isolating the effects is not feasible with the quantitative evaluation. However, since the main difference between treatment and comparison groups is found in Components 1-3, we focus on these components. These components are linked in that farmer level interventions (Component 2) would interact with how groups of farmers performed (Component 1), in order to be able to take advantage of greater market opportunities (Component 3).
Component 1: Organization and Enterprise Skills of Dairy Groups
a. Disorganized groups with limited business/commercialization skills and knowledge
b. Limited or weak links with input suppliers and output purchasers, leading to missed opportunities to secure lower input prices or higher and more reliable output prices
c. Limited ability to disseminate relevant production advice to farmers, particularly disadvantaged groups, including women and resource-poor farmers
Main inputs to address these three barriers:
I1. Extensive training on group organization and management, business skills, development of enterprise plans, and preparation of business proposals that are then eligible for dairy enterprise grants.
I2. Training on establishing and maintaining links with input and service providers and output purchasers
I3. Linking groups to advisory and extension systems
I4. Reaching out to women and resource-poor farmers
Main intended outcomes:
O1. Dairy groups with financially viable and sustainable business plans, and the ability to develop and successfully obtain external grant funds (due mainly to I1 input). The main assumption here is that dairy group members were able to successfully understand the training materials and translate that knowledge into business plans and proposal writing. Another key assumption is that there were real business opportunities that relatively small and resource-poor dairy groups could take advantage of.
O2. Transactions costs and input costs reduced, and output prices potentially increased (due mainly to I2 and to some extent I1). The main assumption is that access to milk markets and market players can be increased by knowledge gained in training. It should be noted the project was undertaken in areas determined to be high-potential commercialization areas, which was intended to limit the impact of other barriers such as geographic isolation and very high transportation costs.
O3. Dairy group members, including women and resource-poor farmers, increase knowledge on dairy production and markets, leading to higher and more stable milk production and to lower transactions costs of participating in markets (due mainly to I3 and to some extent I1).
Component 2: Technical Support to Dairy Producers
d. Poor breeds of dairy cows
e. Lack of adoption of improved management practices, such as zero-grazing and keeping milk records, and producing hay and silage
f. Poor animal nutrition, and pronounced seasonality in fodder and feed use
g. Animal diseases
Main inputs to address these four barriers:
I1. Trainings and demonstrations to disseminate information about benefits to improving breeds through AI and benefits from animal registration, improved husbandry and dairy enterprise management practices, improved fodder production and management and supplemental feed use, and animal diseases and disease management
I2. Establishing community AI schemes
I3. Establishing a revolving community-based animal health fund
I4. Training community resource persons to aide in disseminating information and linking farmers to relevant resources
Main intended outcomes:
O1. Better-bred dairy cows, leading to higher milk per cow and total output (I1 and I2). Assumes that quality AI seed is available, and that farmers see the value in improving breeds, which provides delayed benefits.
O2. Greater production and better management of fodder and feed, leading to lower costs of milk production and greater stability in milk output throughout the year (I1 and I4). Assumes limited opportunity costs associated with fodder being put to other uses, and that options for extending forage availability throughout the year (e.g., storage) are profitable.
O3. Healthier cows producing more milk (I1, I3 and I4). Assumes that information on disease management and access to revolving funds are sufficient to address substantial issues with tick-borne disease control.
O4. Better overall management practices, leading to greater production and potentially lower costs of production (I1 and I4). Assumes that training materials contain relevant and understandable information that farmers can apply in practice. Also assumes relatively low opportunity costs, investment costs, and input costs, or alternatively, that switching to new systems is actually profitable.
O5. More milk to meet household needs and to participate year-round in the milk market (I1, enterprise management and fodder production and management; and to some extent I4). Assumes that market linkages are established (e.g., through the dairy groups) and transactions costs are sufficiently lowered.
Component 3: Development of Milk Marketing Chains
h. Disorganized markets
i. Market traders with limited skills to maintain high-quality milk (e.g., hygiene standards)
j. Market traders with limited enterprise management skills
k. Limited generation and dissemination of milk market information
l. Limited market infrastructure (e.g., bulk milk facilities)
m. Limited production and marketing of dairy goat milk
n. Inefficient contractual arrangements between dairy groups, milk collectors, cooling centres, and processors
o. Limited access by smallholders to rural finance
Main inputs to address these eight barriers:
I1. Developing a low-cost market information system and strengthening the Dairy Information Centre.
I2. Linking activities between smallholders and rural finance operators
I3. Capacity building for milk marketing groups
I4. Pilot testing school milk programmes as an opportunity to expand milk marketing.
I5. Performing a study on milk marketing opportunities
I6. Trainings on hygienic milk handling
I7. Establishing milk bulking facilities and other infrastructure (e.g., cooling facilities)
I8. Training and demonstrations on dairy goat production and marketing, as well as procurement and distribution of dairy goats to resource-poor smallholders
I9. Developing new and improved contractual arrangements
Main intended outcomes:
O1. Reduced transactions costs of participating in the market (I1, I2, and I7). Assumes that farmers and other market players can access new information sources, and that information is relevant and understandable.
O2. Increased size of the market (I3, I4, and I5). Assumes there is scope for expanding the milk market. External evidence suggests there is such scope.
O3. Increased quality of milk in the market, increasing value added and potentially leading to higher prices for smallholders (I6 and I7). Assumes that knowledge is disseminated in a practical and useful way, and more importantly, that sufficient access to technologies and infrastructure exist throughout the entire value chain.
O4. More effective contractual arrangements increases the quantity of milk in the market throughout the year. Assumes that contract terms are currently inefficient and there is scope to make improvements.
O5. Increased participation in the dairy goat milk market by women and resource-poor smallholders (I8).
From Outcomes to Impacts: As noted above, the primary impact is expected to be higher net milk incomes, though increased production and productivity per animal, reduced input costs, reduced transactions costs, and potentially higher farmgate milk prices. The outcomes clearly map to this impact. The second expected impact is greater participation by women and resource-poor farmers in milk markets and as leaders in dairy groups, which follow from: Component 1, Outcome 3 and from Component 3, Outcome 2. While we can analyse this impact through heterogeneous effects analysis, the greater participation by women and resource-poor farmers is an impact in and of itself, since the project aims to ensure that women and the most vulnerable were indeed included. The third expected impact is increased food security. In part, this is related to higher net dairy incomes, and thus related to two of the four pillars of food security, access, and availability. Additionally, greater stability of milk production and sales throughout the year is related to the stability pillar. Finally, trainings on hygiene increases milk safety for consumers, which is related to the utilisation pillar.
Primary Outcomes and Impacts of Interest
The programme's key intended outcomes included more knowledgeable dairy farmers, healthier and better-bred milk cows, reduced seasonality in milk production, increased participation and labour opportunities in milk markets (generating the impact of higher net dairy incomes for smallholder dairy farmers); well-developed business plans and project proposals, and establishment of reliable trade relations with input suppliers and output purchasers for dairy groups (generating the impact of commercially viable and sustainable dairy organizations); and increased networks and higher quality milk for milk traders (generating the impact of greater value addition).
In this study we investigate programme impacts and implementation of the SDCP. Our main research question is the following:
1. Does SDCP improve the wellbeing of dairy farmers through improved animal management (e.g., better husbandry practices), increased production and productivity (e.g., litres/cow/day), improved efficiency (e.g., input and transaction cost reductions), and increased incomes for farmers (e.g., gross margins and higher milk prices)? Does this lead to increased income from dairy farming and ultimately improved food security?
To better understand our primary research question, we consider the following secondary questions:
1. Are there differences in programme participation by the sex of the farmers? Does the programme have differential effects by the sex of the farmers?
2. How does the process by which SDCP is implemented influence the effectiveness of the programme? Given this process, how do contextual factors affect programme success?
The first secondary question is critical since women play a key role in dairy production in Kenya. However, they face a number of constraints that may alter the extent to which they benefit from the programme, such as owning smaller farms, which affects their access to credit using land as collateral, or being less educated, which limits their access to technical information for enhancing production. The second secondary question is especially important because the impacts of a programme are ultimately a function of the manner in which it is implemented. Understanding the implementation of this programme is a critical aspect of this evaluation because it largely determines the type of dairy farmer the programme reaches. There are clear implications if the programme does not reach certain segments of the population (e.g., women, poorer farmers, certain locations, etc.). The process can also lead to variation in programme effectiveness in that it may vary by contextual factors, such as the population density of a region or the type of farming.
Agriculture is one of the leading sectors in Kenya, employing 70% of the rural population and accounting for 25% of the GDP. More specifically, the dairy sector in Kenya is one of the largest and most developed in Sub-Saharan Africa, accounting for 4% of the country's GDP. Dairy farms are concentrated in the highlands and former provinces of the Rift Valley in the central and eastern parts of Kenya. There are more than 3.5 million heads of purebred Friesian--Holstein, Ayrshire, Guernsey, and Jersey cattle and their crosses, with a total yearly production of about 2 billion litres of milk (Muriaki, 2003). Despite these high production volumes, the dairy industry is dominated by smallholder farmers. Approximately 80% of people engaged in agricultural activities are smallholder farmers, producing three quarters of the agricultural output in farms that commonly have an area of less than 3 ha. Smallholders produce over 50% of the predominant crops in the country--including maize, coffee, and tea--and 80% of milk and beef products. There are over 1 million smallholder farmers who depend on dairy farming for their livelihood. Less than 15% of marketed milk flows through milk processors (Thorpe et al. 2000), with the rest being sold as raw milk through direct sales to consumers by farm households, dairy cooperative societies, and individual traders.
As with many other countries in the sub-Saharan region, smallholder dairy farmers in Kenya rely on livestock for income and food security. However, milk production is regularly threatened by inadequate access to markets, animal diseases, poor quality and unstable supplies of animal food and feeds with corresponding decreases in market milk supply, poor rural infrastructure, and inadequate access to AI services and other veterinary services. More importantly, small dairy farmers face a series of constraints that prevent them from effectively commercializing their milk, including large seasonal fluctuations in milk output and prices, poor rural infrastructure (roads and electricity), limited skills with which to bargain with input suppliers and output purchasers, lack of management and business skills, and inefficiencies in the post-harvest segment of the dairy value chain. Furthermore, the dependency on traders who have limited ability to maintain milk quality puts downward pressure on farmgate milk prices. The inability of farmers to avoid and respond to these risks and constraints associated with milk production is partly explained by the lack of access to productive inputs and improved technologies, such as silage equipment, and also to a lack of knowledge on how to maintain animal health and increase milk productivity through better production practices.
In 2005, IFAD and the GoK commissioned a study by the ILRI to lead the initial targeting of the programme. Three key indicators were initially considered to determine the programme area: milk production and production potential, incidence of rural poverty, and market access. Milk production was measured as litres of milk produced per square kilometre per year, an indicator that reflects well the productivity by individual cow; number of animals in a given area; and percentage of lactating animals. Poverty incidence was drawn from data constructed by the Central Bureau of Statistics in 2003. Finally, market access was defined as distance from farms to main milk cooling centres, as well as distance from farms to main urban centres. Upon realization that the three key indicators were too restrictive in terms of potential programme locations, the programme targeting instead focused on the two key indicators of milk production and production potential and incidence of rural poverty. For these indicators, thresholds were defined to target the initial participating districts/counties. First, the programme selected districts with high (>90,000 litres/km2/year) and medium 60,000 (<90,000 litres/km2/year) milk densities. Second, they chose districts with a poverty rate of at least 46%.
To select beneficiaries, the project relied on "Targeting Pro-Poor Investment in the Dairy Sub-Sector," a report produced by ILRI in March 2005, which contained rich analysis on the smallholder dairy subsector. Using additional information from dairy farmer surveys, the project first selected nine districts out of the 25 milk-producing districts in Kenya. Then, because not all divisions/subcounties (the next administrative unit after districts/counties) within the nine selected districts met the targeting criteria, SDCP focused operations on only 27 divisions (out of a total of 53 divisions in the nine districts). These divisions corresponded to Dairy Commercialization Areas (DCA), and the GoK selected locations and sublocations within the 27 DCAs, with the aim of identifying 600 farmer groups. The project attempted to work with already existing farmer groups that had at least some activities focused on dairy farming. Each DCA had 500-800 dairy farmers, which was considered a manageable number of farmers to be handled by divisional government staff.
Once the geographical units for the programme were selected, SDCP operationalized programme activities in the field through dairy groups. Dairy groups are formally registered as self-help groups, cooperative societies, or common interest groups that have a common interest in dairy farming. On average, each SDCP dairy group has 30 dairy farmers. At the end of 2015, SDCP had worked with 527 dairy groups, with more than 16,000 members, of which 60% were women. The selection of dairy groups within programme areas was done using a participatory selection process that gave priority to resource-poor dairy farmers, including women and youth. More specifically, SDCP chose individual dairy groups based on the following observable characteristics: (i) farmers had a maximum of two cows; (b) farms had an average of 2.5 acres (0.91 hectares (ha)) of productive land; and (c) the group was not marketing more than 30% of its milk before joining the programme. The programme targeted the dairy group; thus, all participating farmers in the dairy group were eligible. Lastly, the identification of existing dairy groups in each district was facilitated by the fact that in Kenya, all common interest groups need to be registered with the national and local governments. While IFAD and GoK targeted 600 dairy groups, numerous other dairy groups existed in the districts who were not treated due to limited implementation.
Study Site Selection
To select the sites for the study, we replicated the targeting process that ILRI conducted in 2005. This replication process involved the use of observation-based and criteria-based targeting, as proposed by Ouma and colleagues (2007). Observation-based targeting involved determining where the SDCP was adopted, plotting those sites on a map, and identifying the common characteristics the sites shared. To conduct the observation-based targeting, the AIR and Lead Analytics team worked with contacts at ILRI to examine the data that was originally used for the targeting exercise. We engaged with Ms. Pamela Ochungo, a geographic information systems analyst from Kenya who conducted the original targeting exercise on behalf of ILRI, to recreate the maps used for the original targeting of the programme. Exhibit 1 shows the areas that met the threshold for high and medium milk production and poverty levels.
We used this information to determine a group of potential nonprogramme areas that had characteristics similar to SDCP areas before the programme started. Because IFAD and GoK determined the target of 600 dairy groups in part due to capacity constraints, there were additional nontreated areas similar to the treated areas that could serve as a comparison group. After recreating the original targeting map, we performed propensity score matching at the level of the division by using the original data to match treatment divisions to potential comparison divisions within each county. We calculated from the original ILRI data, a propensity score for the division based on the 2005 values of the variables of milk density, proportion of poor households, proportion of Grade cows in dairy households, proportion of dairy households, and travel distances to urban and cooling centres. We matched similar comparison divisions to treatment divisions within the same county. That is, we simultaneously selected treatment and comparison divisions for the study areas on the basis of their being the most comparable pairs according to observed 2005 data. Restricting to the same county was important because over the programme implementation years, there was variability in county-level support for dairy farming after devolution. Thus, to mitigate specific influences at the county level, we needed to compare treatment divisions with comparison divisions in the same county.
In addition to the results of the matching exercise, we used criteria-based targeting to refine the selection of the comparison sites. Criteria-based targeting is based on the opinion of experts, who determine to what extent nontargeted areas could have been chosen for the programme. Experts relied on historic variables that are likely to be associated with the uptake of the intervention in 2005, such as climate, market access, and other agro-ecological conditions. Mr. Luke Kessei, the Ministry Desk Officer for the SDCP from 2005 to 2015, played a key role in designing and implementing the programme and is now an independent consultant. He, along with the SDCP technical team, served as our experts for the criteria-based targeting. In October 2016, we met with IFAD and SDCP officers in Nairobi and Nakuru. Through the meetings with the SDCP technical team, we discussed at a high level which of the potential comparison areas might serve as the best counterfactual in terms of similarities to SDCP treatment divisions.
We presented to the team a list of the nine programme counties along with all the potential divisions (all the beneficiary divisions which could serve as study treatment areas and the non-beneficiary divisions which could serve as comparison areas). With the help of our experts, we narrowed down the list of SDCP treatment and comparison divisions to include in the evaluation. In these discussions, the experts considered eight criteria: (1) whether the evaluation included at least one county from each of the three geographic programme clusters (1); (2) the variation in county-level support after devolution, where areas with high levels of support were excluded; (3) the variation in support provided by other dairy-focused programmes, where areas with high levels of support were excluded; (4) the violence that happened in certain areas during the 2007 election, which resulted in factories being destroyed, livestock being killed, and some areas receiving government support in response to the violence. Areas that had very high levels of violence were excluded; (5) the similarities in farmer composition, since some potential comparison divisions had large-scale farmers which would not serve as a valid counterfactual to the smallholder farmers participating in the SDCP. Areas where the farmers tended to be larger-scale were excluded; (6) the geographic proximity of the potential comparison areas to the SDCP treatment areas, since comparison areas that neighboured the SDCP areas would be at higher risk of contamination from spillover effects. Comparison areas that closely neighboured SDCP treatment areas were excluded; (7) the focus of the farmers in the area, since some farmers were focused more on tea farming than on dairy farming. Areas where the farmers tended to be mostly tea farmers were excluded; and (8) the feasibility of data collection requirements in very remote divisions, which were excluded. One of the major benefits of the criteria-based targeting exercise is that, based upon the insight of the experts, we were able to exclude divisions where other policies or actions may have had differential effects over time, which is especially beneficial since this is an ex-post evaluation for a long-running programme. Specifically, we excluded divisions where county level policies provided a high level of support, where other dairy programmes provided a high level of support, and where 2007 election violence was extreme.
We cross-referenced the sites recommended by the experts from the criteria-based targeting exercise to the results of the division-level propensity score matching from the observation-based targeting exercise to arrive at the final sample. The results of the two-part targeting exercise are detailed in Table 1 and further visualized in Exhibit 2.
Since the evaluation sampled from all three geographic clusters of programme implementation, the results should be generally representative of the SDCP population. That is, the results should apply to the average dairy farmer who is located in areas with medium to high levels of milk production and poverty. However, because we excluded certain divisions that would have posed difficulties in forming our study group, our sample lacks representativeness on certain dimensions. Specifically, the study did not include areas where there was a high level of support provided by other dairy-focused programmes, where the aftermath of the 2007 election was severe, where large-scale farmers were more common than smallholders, where farmers were less focused on dairy farming, and where farmers were located far from each other. Nevertheless, the observation-based and criteria-based targeting exercises ensured that we had comparison areas that are similar to the treated areas. However, there are some limitations to the external validity of the study sites. Specifically, as indicated by the upper left corner of Exhibit 1, implementation of the SDCP covered districts/counties in the western region of Kenya. Thus, the results do not necessarily extend to other regions of Kenya where dairy farming is a less central focus of smallholder farmers. However, lessons may well be drawn for other East African countries with similar ecological and socioeconomic characteristics as those found in western Kenya, including those in the central highlands of Kenya, as well as the highlands of Uganda, Rwanda, Tanzania, and to a lesser extent, Ethiopia (Herrero et al. 2014, Global Environmental Change, and references cited therein).
The following timeline depicts the stages of implementation and evaluation. As an ex-post evaluation, all evaluation activities occurred after implementation of the program.
Evaluation: Design, Methods, and Implementation
AIR is registered with Office of Human Research Protection as a research institution (IORG0000260) and conducts research under its own Federal wide Assurance (FWA00003952). AIR's Institutional Review Board (IRB) (IRB00000436) reviewed our procedures for minimizing the risks to participants, along with the instruments and protocols. To ensure ethical research, RSA read a consent statement that explained the purposes of the research and the expected duration of the subject's participation, and that described the procedures to be followed.
It was necessary to establish a clear counterfactual to conduct a valid assessment of the impact of SDCP on smallholder farmers. To address the question of what would have happened to programme participants had they not received the intervention, we used a rigorous quasi-experimental methodology. An experimental design was not possible because of the ex-post nature of the evaluation. However, estimating programme impacts by comparing a treatment group with a nonexperimental comparison group may be biased because participants self-select into the programme or implementing partners specifically target those beneficiaries that are more likely to experience the largest programme impacts. Within the context section above, we describe how the SDCP targeted programme beneficiaries and how we used that information along with observation-based and criteria-based targeting to select the study sites. That is, we first, used existing administrative information from the original programme targeting to identify (through propensity score matching at the division level) similar nonprogramme areas that were not affected by the SDCP due to capacity constraints (observation-based targeting). Second, we discussed the filtered control communities with local experts and stakeholders to determine which of the preselected control areas were more similar to treatment communities at project inception (criteria-based targeting).
After matching comparison areas to treatment areas for the study sites, RSA, with oversight from Lead Analytics, conducted a comprehensive dairy survey to a sample of 2,562 dairy farmers that covered key information needed to map out the causal chain among inputs, activities, outputs, outcomes, and impacts as well as the underlying assumptions of the SDCP. The sample of 2,562 was split between 1,297 SDCP beneficiaries (from 95 dairy groups) and 1,265 matched comparison farmers (from 89 dairy groups).
To capture key time-invariant characteristics and retrospective information on dairy farming, we included a short filter questionnaire at the beginning of the household survey to improve the matching process between each one of households in the treatment group to a similar farm household in a comparison area. That is, only farm households that were eligible as potential comparisons based on predetermined variables were part of the sample. The idea behind the filter questionnaire is to mimic the selection mechanism that SDCP staff used when defining the dairy groups that later joined the programme.
The identification strategy proposed to estimate the causal effects of the programme relies on the doubly robust estimator developed by Robins and Rotnitzky (1995); Robins, Rotnitzky, and Lue Ping Zhao (1995); and van der Laan and Robins (2003). At the household level, the proposed approach combines regression and propensity score matching methods in a three-step approach to estimate treatment effects. In the first step, a treatment model is defined that explains the probability of programme participation. From this step, inverse-probability weights are derived from the estimated propensity score. Second, using the estimated inverse-probability weights, weighted regression models are fit for the outcome equation for each treatment level and obtained the treatment-specific predicted outcomes for each subject. Lastly, means of the treatment-specific predicted outcomes are computed and the difference of these averages provides the estimate of the average treatment effect of the programme. Intuitively, weighting can be interpreted as removing the correlation between the treatment condition and other covariates that may be correlated with treatment, and regression as removing the direct effect of such variables on the outcomes of interest (Imbens & Wooldridge, 2009).
This approach assumes that programme participation is exogenous to potential outcomes conditional on observable characteristics--that is, that there is no selection bias due to unobserved characteristics and that the observable characteristics we capture determine programme participation. Due to the unobservable nature of these potential additional characteristics, this assumption is untestable. Nevertheless, we employ a series of strategies to reduce the potential threat of the impact estimates being driven by unobserved characteristics of programme participants. Specifically, in addition to replicating the division selection process that was conducted to determine programme placement, we use a filter questionnaire to replicate the selection of potentially eligible dairy groups, and to collect numerous covariates as controls that are good predictors of programme participation. Several authors have argued that social programmes can be evaluated using matching methods, as long as there is access to a rich set of variables that determine programme participation, and that the nonexperimental comparison group is drawn from the same local region as participants (Heckman, Ichimura, & Todd, 1997; Heckman, Ichimura, Smith, & Todd, 1998). We are confident that our proposed empirical strategy will allow us to estimate the causal effect of the SDCP on smallholder dairy farmers.
Sample Size Determination
We determined that a sample of 2,500 farmers, split evenly between the treatment and comparison groups, would be sufficiently large to detect meaningful programme effects. To calculate the sample size, we conducted a number of power analyses based on existing farmlevel data for SDCP beneficiaries that were collected as part of a 2014 survey commissioned by SDCP to Capital Guardian Consulting. These calculations are described in more detail in Appendix E. The calculations account for the fact that the dairy group serves as the clustering variable. We proposed to collect information for 154 dairy groups (77 treated and 77 comparison). For each dairy group, we proposed to randomly select 15 households. In practice, we conducted a full survey instrument to the sample of 2,562 observations (1,297 treatment and 1,265 comparison dairy farm households), coming from 183 dairy groups (95 treatment and 89 control). In the survey, we collected key information to map out the causal chain among inputs, activities, outputs, outcomes, and impacts, as well as the underlying assumptions.
To increase comparability beyond the efforts we took for the study site selection, we included a short filter questionnaire at the beginning of the household survey to capture key time-invariant characteristics and retrospective information on dairy farming to improve the matching process between each one of households in the treatment group to a similar farm household in a comparison area. We included only dairy farmers who met characteristics that should have increased comparability between the treatment and comparison groups. We have included the Filter questionnaire in Appendix C. The filter questionnaire ensured that only those potential comparison group farmers who were most similar to the treated farmers remained in the final sample. Furthermore, this procedure represented an efficient use of project funds and helped to reduce the time burden placed on the farmers. By including a filter questionnaire that eliminates dissimilar farmers, we ensured that project resources spent on the full household survey were allocated to those farmers most relevant for inclusion in the study. Additionally, these procedures reduce the time burden associated with the survey, since we administered the survey to only relevant farmers.
AIR, in conjunction with LEAD Analytics, designed the quantitative questionnaire, adapting questionnaires used by ILRI in Tanzania and the World Bank Living Standards Measurement Study-Integrated Surveys on Agriculture in Malawi. The questionnaire was designed to collect detailed data about milk production, cost, and sales to generate information on net milk income and milk sales, which are two primary outcomes of interest to assess project impact. The project also aimed to reduce seasonality of milk production, so that net incomes would be higher and less variable throughout the year. Thus, the questionnaire also collected data on practices, such as second-season fodder grass production, associated with less pronounced seasonality in milk production. Finally, the hoped-for impacts include increased food security. In part, lower seasonality should contribute to more smooth consumption patterns throughout the year, reducing or eliminating the lean season. Second, while a full-scale consumption module was not included, we did include a module to capture dietary diversity, based on recent recall data.
Building on evidence from the literature, the questionnaire also included sections to recover information on the most important control variables at the household level, in order to improve precision of estimating project impact. These included basic household demographics and wealth variables; landholdings; and access to extension and other sources of information, density of social networks, etc.
Importantly, we also designed a dairy group questionnaire. The functioning of dairy groups (i.e., structure, conduct, and performance) is likely to have a strong impact on the ability of households to benefit from project activities, many of which were carried out through the dairy group leadership. Indicators of dairy group performance can serve as controls and can also provide valuable additional insights to feed into future project designs. The dairy group questionnaire also included a module on the history of presence of other development projects in addition to SDCP, which could prove to be useful control information, as well as basic information on community characteristics. In control villages, where no dairy groups are currently functioning, the community-level questions were addressed to village leaders. Appendix C presents the household and dairy group questionnaires.
The paper questionnaire was translated to tablet, primarily by our local survey partners, RSA, with inputs from LEAD Analytics during the training and piloting phases. The questionnaires were conducted in the field by RSA between November 15 and December 20, 2016. This time frame corresponds to the secondary rainy season in Kenya, which happens for a few weeks in November and December and is followed by a dry season of hot weather through March. The heaviest rainy season in Kenya occurs in late-April, May and early June. The survey time period closely corresponds to the end of harvest period for the main rainy season. (3) A Lead Analytics' field manager oversaw the training and piloting of data collection and, after coordinating with the other AIR and Lead Analytics team members, provided daily feedback to the enumerators and survey programmers throughout both processes.
Strategies to Avoid Biases
Training for both our quantitative and qualitative data collection was key to ensuring that interviewers understood the study, the interview protocols, interviewing techniques, and the importance of understanding questions exactly as written and recording responses exactly as stated--all of which improve data quality and reduce bias. In addition, the training covered administering and obtaining consent from every participant, to ensure compliance with AIR's IRB. The training covered the following topics:
* Overview of SDCP
* Overview of AIR's evaluation of SDCP
* Ethics, consent, and confidentiality
* Basics of qualitative research (interview protocols and observations)
* Professional conduct
* Data security
* In-depth practice of all protocols, including translations and role playing
* Team debriefs and protocol revisions
* Field plans and logistics
We further reduced biases by choosing comparison areas, as opposed to comparison households. The use of comparison areas helped overcome concerns about not capturing the true programme impacts due to spillover effects. If farmers in the comparison areas are outside the SDCP's catchment area, the benefits would be less likely to flow to the comparison areas. We investigated the validity of this requirement through the farm-level surveys by asking farmers in comparison areas about how much they know about IFAD's SDCP and other agricultural development programmes. Since numerous development and other agricultural programmes have been implemented in Kenya, it is important that the comparison and treatment groups have the same level of exposure to these programmes over time, with the exception of SDCP. Many of these other development and agricultural programmes are implemented at a county level. By choosing comparison areas that are within the same counties as the treated DCAs, we can account for exposure to other dairy programmes. Although we chose comparison areas within the same counties, we made sure not to choose comparison divisions that directly neighboured a treatment division. Ensuring that treated and comparison areas are sufficiently spaced apart helps to minimize the risk of spillovers.
An obvious concern when evaluating the programme is that due to a sense of reciprocity (i.e., Hawthorne Effect), respondents in the treatment group may offer answers that they believe the research team seeks. We structured the questionnaire in such a way that the questions related to the programme intervention are nonconspicuous and occur after questions related to the measurement of important outcomes, so as to delay activation of potential Hawthorne effects. In addition to that, we do not believe this evaluation exhibited John Henry Effects where nontreated farmers react to overcome the disadvantage of being in the control group, because we do not expect comparison farmers to know much about the SDCP, since they will be drawn from different geographical areas.
Addressing the research questions required a combination of qualitative and quantitative methods. In qualitative research, questions--and the responses they elicit--tend to be discursive and descriptive, while the analysis privileges explanation and interpretation over quantification. In general, qualitative approaches allow researchers to explore and understand the experiences, opinions, and perspectives of their informants in greater depth than that offered by quantitative approaches. In turn, the use of qualitative approaches entails sacrifices in terms of generalizability and comparability--areas in which quantitative methods excel because of their use of large and probabilistic samples. Samples chosen for qualitative studies are often nonrandomized--or "purposively" selected--and always smaller: "There is growing evidence that 10-20 knowledgeable people are enough to uncover and understand the core categories in any well-defined cultural domain or study of lived experience" (Bernard, 2011, p. 154).
We drew from qualitative methods to augment the quantitative surveys by capturing interaction among complex and changing contextual factors that could influence the impact of the SDCP and evaluated programme fidelity. The process component also aimed to assess gaps in implementation that may have affected impacts and how dairy farmers may have changed their practices based on what they learned with regard to Components 1-3. The contextual information we obtained through the qualitative information helped to clarify how the programme impacted individuals, thus contributing to the transferability of study findings to other settings (i.e., external validity).
We conducted semistructured interviews with key informants directly involved in the programme at the national, county, and sub-county levels. These discussions aimed to clarify uptake of the programme and the coordination of county departments of agriculture with the farmers. Second, we conducted key informant interviews with private service providers who are unrelated to the programme to ascertain how the dairy sector functions outside of the programme and public sector.
We conducted 12 focus groups using semistructured protocols with 6-8 farmers each from 12 dairy groups across two counties. We included two counties with three SDCP administrative areas each in the qualitative sample, just in case there were any notable regional or within-county differences by administrative area. We included dairy groups with varying proportions of male and female farmers. The focus group sample was half male and half female, and we separated the groups by male and female farmers to capture a better understanding of experiences by gender. Finally, we included groups who were considered to be high performing and low performing based on consultations with stakeholders familiar with the dairy groups.
We structured the questions to understand whether all of the groups engaged similarly with the various components, or whether and how some of the components were stronger for some groups than others. To compare these data with how groups function in the absence of the programme, we also conducted four focus group discussions (two male, two female) with farmers in comparison areas. We aimed to cover all components with some piece of the qualitative work (Table 2).
To ensure reliability of the findings, the research team employed several analytic methods to systematize the data review and coding: (a) content coding, (b) comparison of findings among researchers, and (c) grounded theory to guide analysis using the qualitative software programme (NVivo qualitative data analysis software, QSR International Pty Ltd., Version 10, 2012).
The first step in analysing qualitative data is to develop a coding structure that helps to systematically categorize information. Researchers "open code" data to identify primary categories of interest. These categories form the basis for the coding structure that the team uses to categorize raw data from interviews and focus group discussions into the primary findings. While categorizing the data, researchers utilized grounded theory (Glaser & Strauss, 2009) to deduct new themes from the findings, rather than testing an existing theory. Combining the use of grounded theory with a rigorous impact evaluation design enabled us to triangulate the research findings. The quantitative research served to test predefined hypotheses, and the qualitative research enabled us to ground new ideas on why the programme positively influenced some but not all outcomes of interest among respondents' real-life experiences.
After coding, the research team attempted to quantify the data, where applicable. This method helped to characterize the prevalence of responses to deduce which themes were common and which were outliers. It is important to note, however, qualitative data are not best analysed using a systematic count of opinions.
Implementation: Data Collection and Strategies to Avoid Biases
In general, qualitative approaches allow researchers to explore and understand the experiences, opinions, and perspectives of their informants in greater depth than that offered by quantitative approaches. The use of qualitative approaches entails sacrifices in terms of generalizability and comparability--areas in which quantitative methods excel because of their use of large and probabilistic samples. Interviewers were clear on these limitations, as well as their neutral role that aimed to limit courtesy and social desirability bias, a situation in which a respondent gives an answer that he or she feels the interviewer wants to hear.
Data collection occurred over a period of 2 weeks, between February 27 and March 10, 2017. The evaluation team trained data collectors before they went into the field. We held a plenary training to introduce participants to the study, the supervision process, logistics, interviewing techniques, and the interview and observation protocols, as well as to conduct mock interviews. During training, the research team improved the items on the interview protocols by incorporating the input from local interviewers. During the initial days in the field, interviewers noted challenges in administering the surveys and interview protocols, which we discussed during debriefings during the first phase of fieldwork.
The team collected data using notes and digital recordings. All of the interviews and focus group discussions were transcribed. The team ensured complete anonymization and protection of confidentiality for research participants.
Limitations of the Study
As noted before, the methodology employed with the quantitative design assumes that there is no selection bias due to unobserved characteristics and that the observable characteristics we capture determine programme participation. However, due to the unobservable nature of these potential additional characteristics, this assumption is untestable. Additionally, while our qualitative research provided insights into the uptake rates of the programme, we were not able to determine these rates quantitatively, because tracking down targeted farmers from 2006 was not feasible. This limitation prevented us from analysing what impact, if any, the programme had on the likelihood of a dairy farmer in 2006 remaining one to the present day.
Programme or Policy: Design, Methods, and Implementation
The SDCP is a joint programme between the GoK and IFAD. The Programme commenced in July 2006 and will be complete in September 2019. The SDCP was designed to reach Dairy Groups engaged in milk production in nine milk producing counties. The overall goal of SDCP was to increase the income of poor rural households that depend substantially on production and trade of dairy products for their livelihoods. The SDCP was implemented through various inter-related components. First SDCP provided training on organisational, managerial, and enterprise skills (e.g., bookkeeping, accounting, financial planning) to farmers. Second, the programme targeted household production by aiming to enhance dairy farming productivity and reduce production costs through grants, trainings, field days, and demonstrations. Lastly, SDCP aimed to strengthen market linkages for small-scale milk producers by improving road infrastructure and conducting additional trainings to on milk handling practices and value addition opportunities. The project operated through Dairy Groups and was intended to reach 24,000 smallholder dairy farming households with members participating in 600 Dairy Groups across nine milk producing counties.
Since the implementation began in 2006, we did not have a monitoring system to track implementation rollout. Similarly, since the evaluation was an ex-post evaluation, we did not provide incentives for participation in the study groups. For the quantitative data collection and the Key Informant Interviews, we did not provide any compensation or incentives for completion of the survey or interviews, respectively. For participation in the qualitative focus group discussions we provided a transport allowance and snack.
Impact Analysis and Results of Key Evaluation Questions
Descriptive Statistics and Quality of Counterfactual
In this section, we present some of the key explanatory variables collected for the study. These variables help us to understand the context in which the SDCP was implemented and serve as controls in the econometric models. This section also describes the outcomes of interest and presents the impact results.
The variables that were used for the matching procedure at the division level, were collected by ILRI for programme placement before 2005. These include milk density, proportion of poor households, and travel time to the nearest urban centre, which we have included in Table 3. The original ILRI data also included the number of households and area, although the ILRI final targeting exercise only utilised milk density and proportion of poor households. As shown in Table 3, milk density in the study area is on average 83,000 litres/km2/year. The difference in milk density between the SDCP areas and the control areas is not statistically significant. From examining the proportion of poor households in the division, we see that more than half of the households in our dataset are poor. The proportion of poor people in the SDCP households and control households is similar at 53%. In terms of distance, SDCP households travel 0.39 to hours to the nearest urban centre as do households in the comparison group.
The averages for these three key results are similar for both the treatment and control groups. More specifically, Table 3 also presents the results of a linear model for the probability of being in the treatment group as a function of the three division-level characteristics used for program placement. This regression is conducted at the dairy group level and it includes county fixed effects, which is the administrative level at which the initial matching of divisions was conducted. The results show that none of the three placement variables is a good predictor of the probability of being in the treatment group. In fact, we are not able to reject the hypothesis that the three variables are jointly equal to zero using an F test (p-value = 0.86). These results provide evidence that we are able to find divisions in the same counties where SDCP operates that could have been chosen for the programme given based on the three placement characteristics, but were not chosen due to programme capacity constraints.
Note, however, that the finding control divisions that are similar to SDCP division does not guarantee that the individual dairy groups within these divisions are similar. To increase comparability beyond the study site selection, we included a short filter questionnaire at the beginning of the household survey to capture key time-invariant characteristics and retrospective information on dairy farming to improve the matching process between each one of households in the treatment group and a similar farm household in a comparison area. In Table 4 we specify a linear probability model to assess to what extent the treatment and control households differ in terms of key household and agricultural characteristics. The table shows the mean of each explanatory variable, the change in the propensity of being in the treatment group after a change of one unit on a given characteristic, the standard error of each estimate, and the corresponding p-value.
In terms of household demographics, the data show that the typical household speaks local languages (only 8 percent of the households speak English), has around three people in the working age group (people 14- to 65-years old) and 0.5 children 5 years of age and younger. Also, on average, the SDCP household head is 52 years old and has 13 years of education. Lastly, about 24% of households in our data have a female head of household. The percentage of female-headed households is 11 percentage points higher in the SDCP group than in the control group, a difference that is statistically significant.
Regarding the household socio-economic and agriculture characteristics, the results show that there are no statistically significant differences between treatment and control households in terms of land size, slope of farm plots, access to irrigation, and area cultivated. The average household in the sample has 0.19 acres with a legal title, 87 percent of the farms are either flat or have a slight slope, and does not have access to irrigation. In turn, there are some variables where treatment and control households exhibit differences that are statistically significant. A one standard-deviation increase in the indices of consumer durables and agricultural implement increase the probability of being in the treatment group by approximately 2 percentage points. Also, treatment households are 10 percentage points more likely to have cultivated a crop in the primary agricultural season. Lastly, there were some differences in the breed composition of the division where the household is located as measured in 2005. (4) That is, an increase of 100 zebu heads in 2005 on the division where the household is located decreases the probability of being in the treatment group by 10 percentage points. Alternatively, an increase of 100 cross heads increases the probability of being in the treatment group by 10 percentage points. There are no differences back in 2005 between treatment and control households in terms of the presence of grade cattle in their corresponding divisions.
Lastly, treated and comparison households have similar probabilities of having received other agricultural or livestock development programmes.
Overall, the results show that, with the exception of few variables, treatment and control households have very similar household and socio-economic characteristics. Nevertheless, some of the reported differences in observable characteristics could be an indication that there are differences in unobservable characteristics between the two groups that may ultimately bias programme impacts. Indeed, the balance plot in Exhibit 3 below shows there is a skew towards relatively high propensity scores for the treated households which motivated the use of the Inverse-probability-weighted regression adjustment (IPWRA). It is reassuring for the identification strategy that the weights produce very similar densities of the propensity score.
Moreover, we also conduct balance tests for each covariate considered in the treatment equation, where each observation is weighted by a an inverse function of the estimated propensity score. As shown in Table 5, all the covariates considered are balanced between the treatment and comparison groups when the full sample is considered. In addition to that, we conducted covariate balance tests by strata of the propensity score to check whether the different covariates where also balanced at different points over the distribution of the propensity score. For this analysis, we considered seven blocks or strata of the propensity score to ensure that the mean propensity score is not different for treated and controls in each block. Given that our treatment equation includes 20 covariates, we conducted a total 140 tests (=7*20) of the balancing property. Of those tests, only 5 of tests (3%) reject the hypothesis of T and C having the same mean for a given variable within a given block. Overall the results indicate there is good covariate balance after implementing a strategy based on inverse-probability weighting.
Table 6 indicates that more than half of the households in our dataset reported that there are other agriculture-based projects in their communities other than SDCP. Most of those projects are agricultural ones; the ones that focus on livestock represent only a small portion of the projects available.
Mixed Methods Analysis (Quantitative and Qualitative)
Methods: Regression techniques
As indicated above, at the household level the identification strategy used to estimate SDCP impacts combines regression and propensity score methods in a three-step approach to estimate treatment effects. In the first step, we estimate the probability of programme participation through a logit or probit such as:
[T.sub.i] [PHI] ([X.sub.i] * [beta]' + [[epsilon].sub.i])
where [T.sub.i] is a dummy for having received the programme, [X.sub.i] is a vector of individual and division-level characteristics, and [[epsilon].sub.i] is an error term. The observable characteristics considered in the treatment equation include the 2006 division level characteristics used for the initial programme placement, namely, milk density (litre/km2/year), the proportion of poor people in the division, and the travel time (in hours) to the nearest urban centre. In addition to these variables, we include other 2006 division level variables constructed by ILRI that account for the estimated number of heads from the different breed categories (i.e., local, crossed, and exotic), which are a good indication of the division breed composition, which are good predictors of the ability to increase the prevalence of higher milk producing breeds. We also control for household level characteristics such as the gender and age of the household head, the head's maximum number of years of schooling, the language spoken at home, the number of working age members and the number of children under the age of 5. We also control for indices of consumer durables and agricultural implements as a way to control for household income differences as well as dummies for the existence of other agricultural and livestock projects (different from the SDCP) in the last years in the village where the respondent lives. In order to account for differences in agricultural production in the estimation of the propensity score, we also control for a set of agricultural production characteristics such as indicator variables for producing any crops in the primary and secondary rainy seasons, the natural log of the hectares cultivated in each one of the two seasons, dummies for the slope of the main plot being flat or slightly flat, and indicator variables for the availability of irrigation in at least one of the plots and the percent of plots being served by an irrigation scheme. We also control for county level fixed effects to ensure that the comparison of treatment and comparison households are done within a given county.
Then we generate propensity scores, [P.sub.i], the probability of receiving treatment, as
[mathematical expression not reproducible]
Second, using the estimated inverse-probability weights, weighted regression models are fit for the outcome equation for each treatment level and obtained the treatment-specific predicted outcomes for each subject. Lastly, means of the treatment-specific predicted outcomes are computed and the difference of these averages provides the estimate of the average treatment effect (ATE) of the programme. More specifically, we estimated programme impacts using the teffects ipwra in STATA using the osample option. Note that all observations in the original sample are used in analysis as there are no observations that violate the overlap assumption.
Improved Animal Management
We start the presentation of the programme impacts by looking at the results on animal management. In Table 7 we show the results for the impact of the programme on feeding and water practices farmers use with their cattle. Although the programme did not have a statistically significant impact across all dimensions of feeding and water practices, positive impacts existed related to the grazing practices, keeping practices, and feeding of concentrate feeds. Specifically, we find that SDCP increases the probability that a farmer practices zero grazing with their cattle by 8 percentage points. SDCP staff stated that practicing zero grazing was one of the key messages that the programme delivered to farmers in order to keep control of how much the cows eat and the nutritional value of what they eat as well as reducing losses in potential milk production from food searching in large areas. Improved animal management was the most frequently discussed benefit of the SDCP among FGD participants. One farmer described the changes she made in animal management, "Initially I would send the children to go and graze, [but] I now practice zero grazing. I realized I used to lose a lot on milk yields. At least I can produce more milk, I treat [the cows] when they fall sick, clean them, and ensure they are vaccinated."
We also find that the SDCP increases the probability that farmers keep their cattle in a paddock that has a stall floor made of concrete by 5 percentage points. This increase is promising since concrete floors are easier to clean, which reduces the change that the milk could get contaminated, which may have a positive effect on the selling price charged to some customers. We also find that the SDCP increases the probability that farmers feed cattle concentrate feeds and mineral supplements by 13 percentage points up. The programme also shows a positive impact on the probability of feeding cattle off the ground in a trough when milking, which helps protect food from contamination and promotes a more natural grazing posture. These result suggests that farmers were receptive to programme education regarding the nutritional content and feeding practices, which may have a positive effect on milk production.
The SDCP also led to an improvement in breeding services. Specifically, the SDCP increased the propensity of households monitoring their cattle on a regular basis and the use of Artificial Insemination (AI) services. Table 8 shows that treatment farmers were 7 percentage points more likely to report they monitor their cattle regularly. Similarly, treatment farmers were 12 percentage points more likely to report that they have used the AI Service in the last 12 months. However, we did not find any statistically significant differences between the treatment and comparison groups related to availability and use of own bull service for breeding.
The positive quantitative findings related to access to and use of AI are especially promising considering that many farmers discussed ongoing challenges with AI. The use of AI seems to be a high cost to the farmer, and misperceptions about the purpose and potential of AI services mean that farmers are not able to make informed decisions in these transactions. For example, many farmers seemed to be unclear on the proper timing for AI, and a few farmers thought they could pay more for the AI provider to select the sex when they inseminate. Other challenges included that farmers don't believe that government-subsidized AI services provide the best breeds, and that they have to purchase AI services too often because they are often unsuccessful.
However, in the qualitative data, many farmers discussed ongoing challenges with AI. The use of AI seems to be a high cost to the farmer, and misperceptions about the purpose and potential of AI services mean that farmers are not able to make informed decisions in these transactions. For example, many farmers seemed to be unclear on the proper timing for AI, and a few farmers thought they could pay more for the AI provider to select the sex when they inseminate. Other challenges included that farmers don't believe that government-subsidized AI services provide the best breeds, and that they have to purchase AI services too often because they are often unsuccessful.
The program had positive impacts related to animal health services, with the impacts concentrated on improved access to and use of vaccination services and curative treatment services. Table 9 shows that the SDCP increased the probability that treatment farmers both had access to and used vaccination services by 25 percentage points. Similarly, SDCP farmers are 10 percentage points more likely to have access to curative treatment services and 7 percentage points more likely to use them than farmers in the comparison group. However, the programme does not seem to have changed access to change other health services such as deworming (anthelmintics) or tick control services.
Improved Access to Extension and Information
The SDCP was implemented through several components where the dissemination of information on different areas of the productive process were largely conducted through extension services and trainings of dairy group members. In particular, SDCP provided training on organisational, managerial, and enterprise skills (e.g., bookkeeping, accounting, financial planning) to farmers. In addition, the programme also conducted trainings, field days, and demonstrations specifically designed to enhance dairy farming productivity and reduce production costs. In this section, we assess the effect of the program on making these services available and to what extend the information provided is adopted.
Table 10 presents the results on the reported availability of extension visits, field days, and demonstrations. All these outcomes consistently show that farmers in SDCP areas have a higher access to these type of services. That is, program farmers are on average between 12 to 18 percentage points more likely to report that the different types of extension services are available in their villages.
We also investigated if the programme had any effects on the probability of receiving any information on specific aspects of the production process as well as whether farmers who attended those trainings adopted the recommended practices. The largest program effects are seen on receiving information on general livestock practices (15 percentage points), milk processing and quality control (10 percentage points), fodder establishment and fresh milk marketing (7 percentage points). However, positive impacts are observed for almost all type of cattle-related topics, which may be a consequence of the programme tailoring specific trainings according to the needs and preferences of each dairy group. The results on adopting specific practices closely resemble the impacts reported on Table 11. Interestingly, SDCP farmers did not report having higher access to information on crop best practices relative to the comparison group, which is reassuting as the SDCP programme did not provide any training on crop practices.
Improved Dairy Group Performance
Following the "structure-conduct-performance" literature, here we will first present results on structure, then conduct, and finally, performance. We surveyed 94 dairy groups in 73 SDCP communities and 90 dairy groups in 83 control communities; we note here that enumerators surveyed the "most important" dairy groups within a community. The first difference to note is that there were more dairy group questionnaires implemented in SDCP than in control communities. Also, the results we present here are naive differences between SDCP and control dairy groups. Finally, we only summarize the main results of a larger report on dairy groups, and full results can be found in that report.
In terms of basic structure, nearly all dairy groups have an elected chair, and nearly all of the groups (97%) are legally registered and have by-laws or a written constitution. Additionally, there was similar representation of women as respondents to the dairy group questionnaire and in terms of positions held within the dairy group--just over 50% for both SDCP and control dairy groups. On the other hand, SDCP groups are more likely to also have an elected secretary and treasurer. We expect that groups with more formalized structures should perform better in this context, primarily because one of the main goals of the programme is to link the dairy groups to the milk value chain rather than focusing exclusively on intercommunity activities and also because dues are collected and managed, meaning accountability is important. Committees may also improve the performance of the dairy group, enabling subgroup members to focus on specific tasks and enabling members with specific expertise to exploit their comparative advantages within a committee. Of four committees asked about explicitly, plus and "other" category, SDCP groups were not more likely to have specific committees overall and were significantly less likely to have financial committees. Additionally, SDCP groups are more likely to have monthly meetings, but control groups are more likely to hold weekly meetings; more meetings may mean greater opportunities to share information and engage in decision-making, but too many meetings may also be inefficient. Thus, evidence on structure is somewhat mixed; there are more elected positions but fewer committees, and meetings are held less often.
We have only two aspects of conduct: how decisions are made and sources of financing accessed. In terms of decision-making, there are very few differences between SDCP and control dairy groups across decisions on three broad areas: financing, marketing and information acquisition, and dissemination. We allowed for decisions to be made by consensus, by the chair with other key members, and by committee. The only statistically significant difference is that information decisions for the SDCP group are more likely to be made by the chair plus other key members. For the control dairy groups, information decisions were more evenly spread across the three decision-making mechanisms. On the other hand, it is interesting to note that while finance and marketing decisions were most often made by consensus in more than 50% of both SDCP and control communities, consensus-based decision making was less important for information decisions--between 32% and 36% for SDCP and control communities, respectively. Next, we asked about four specific types of financing and a fifth "other" category. Here, SDCP and control groups exhibited a number of statistically significant differences. SDCP groups were far more likely to obtain financing from micro-finance institutions (14% vs. 1%), from "other" sources that were mainly different types of local credit and savings groups (55% vs. 43%), and from commercial banks (5% vs. 1%). Both SDCP and control groups, however, also relied heavily on members' dues, with 93% stating they relied on member dues for both groups. Overall, both SDCP and control groups exhibit similar patterns in decision-making procedures, but SDCP groups are more likely to obtain financing from a wider range of financial institutions.
Finally, we have a number of measures of dairy group performance, including the different types and frequencies of conflict, different types of trainings, and various "other" dairy group activities. With respect to conflicts, we asked about unpaid dues, amount of dues, participation in trainings, financial issues, management issues, and "other" conflicts. For the most part, SDCP and control groups faced similar percentages of conflicts, with the exception of unpaid dues. In SDCP groups, only 21% said there were conflicts over unpaid dues, versus 37% for control groups. This is interesting because SDCP groups were much less likely to have financial committees, but they were also more likely to have an elected treasurer. Evidence suggests having an elected treasurer may be more important than having specific financing committees, but we did not ask specifically about how members' dues were collected, which could shed more light on this issue. Also of interest is the fact that 39% of both SDCP and control groups indicated that there had been conflicts. Outside of unpaid dues, the most important categories of conflicts included management issues, amount of dues, and "other" issues.
Where we see the most significant difference between SDCP and control dairy groups is in trainings made available to dairy group members. Of nine dairy group topics, SDCP groups stated that at least one training had occurred was statistically greater than control groups on all nine topics. The differences were particularly pronounced for the following topics: dairy group management, proposal writing, and fodder management. Finally, we asked about a number of other services provided by the dairy groups, including organizing educational exchange tours, collecting/sharing milk price data, facilitating links between members and input suppliers, facilitating links between members and milk purchasers, contracting with input suppliers on behalf of members, contracting with milk purchasers on behalf of members, and "other." There was very little difference between SDCP and control dairy groups on the percent of groups that offered the different services, except that SDCP groups were more likely to contract with milk purchasers on behalf of members, 11% versus 2%. It is also interesting to note that over 70% of groups, both SDCP and control, arranged educational exchange tours, while less than 30% of groups undertook any of the other services.
Overall, the evidence from the dairy group survey suggests that SDCP dairy groups function and perform a bit differently than comparison groups. In particular, they are more likely to have elected officials below the chairperson; more likely to access multiple sources of financing; less likely to face conflicts over collecting dues from members; and more likely to contract with milk purchasers on behalf of members, albeit at a relatively low rate (11%). The most striking difference, however, is in the number of trainings provided primarily by SDCP to dairy group members. The latter suggests that there is room to improve the performance of SDCP dairy groups, particularly in terms of the services that the SDCP promotes to reduce smallholders' transactions costs in accessing dairy input and milk markets, thereby increasing net revenues accruing to smallholders.
One of the key goals of the SDCP is to increase the income of poor rural households that depend substantially on production and trade of dairy products for their livelihood. To attain this goal, the programme focused on improving productivity, making some inputs more accessible, and emphasize the importance of value adding and more reliable trade relations. In this section, we investigate program impacts on some key final outcomes of the programme.
First, we look at the effects that the programme had on the cattle size and composition. As shown in Table 12, we find that, relative to the comparison group, SDCP farmers not only own 0.5 more cattle heads (including cows, males, heifers, calves, pre-weaning males), but also have a higher number of cows (calved at least once) as well as animals they are currently milking. Interestingly, these higher impacts on number of animals is mostly explained by a higher number of cross breed cattle and not of the exotic (i.e., grade) more productive breeds.
We then investigate programme impacts on milk production, commercialisation, and total value. Our findings indicate that SDCP farmers are 8 percentage points more likely to sell milk both in the morning and the evening, although only the morning propensity of selling is statistically significant. The evidence suggests SDCP farmers are selling more milk in the market although the results again are not statistically significant at the 10 percent level of significance. Nevertheless, those farmers selling to the market are able to obtain selling price that is 31 percent higher than the selling price received by non-beneficiaries. Overall, the total value of milk sold, calculate as the quantity of milk sold times the price, observed by SDCP farmers is 43 percent higher than the value of the comparison group. Moreover, SDCP farmers show a higher level of milk production for animals at calving as well as the total production calculate as the sum of milk sold, consumed in the household, and lost.
These quantitative results are supported by qualitative data on the perceived effect of the SDCP on farmers' incomes from dairy farming. Without being prompted, FGD participants said they believed SDCP-sponsored activities had enabled them to more easily pay bills and send their children to school. One participant said, "My milk production has increased and that's more income. With it, I have taken my children to school." The perceived effect of the programme on income could partly result from the SDCP focus on recordkeeping. For example, one farmer said, "There is recordkeeping--tracking milk production and noting fall in its production--and getting to understand source of the problem and how to solve it." Though qualitative data cannot accurately indicate whether there was a real nominal increase in production and productivity per animal, FGD participants from treatment groups generally perceived a difference in production as a result of SDCP. Multiple anecdotes indicate farmers' perceptions of increased productivity. One farmer described her increased yields: "Initially I would only get 3 cups of milk, but currently my cow is producing 7 bottles." She also added that this increase helps to pay for school fees.
Increased Food Security
FGD participants said they believed increased income as a result of the SDCP had enabled them to keep a variety of foods available in their households. One farmer said that his family's general health had improved, while others said they now consistently have tea with milk in their house. Farmers' perception of increased food security may partially result from understanding better practices for growing crops, including the use of cow dung for harvesting. One farmer said, "The animal manure from both the goats and cattle is channeled on the farm, which in turn fastens the growth of crops and more yields--this too has ensured a consistency supply of food in key household."
This evidence from qualitative data goes in line with the estimated impacts on the ability of SDCP households to translate higher incomes into higher levels of food security. The survey includes details about food consumption within households to get a sense of higher income has led to more dietary diversity, which is a proxy for food security. One such measure of food security was developed by the Food and Nutrition Technical Assistance Project (FANTA); greater values of the FANTA measure indicate more food insecurity. The FANTA Project includes guidelines for a dietary diversity questionnaire that can be used at the household level. Specifically this diversity measures involves calculating dietary diversity scores by summing the number of food groups consumed by anyone in the household over a reference period (our questionnaire has set this to be the last 7 days). Thus, the dietary diversity scores consist of a simple count of food groups that the household has consumed in the last 7 days.
The results in Table 14 show that SDCP households are more likely to have a more diverse food basket, specially foods with larger levels of animal and vegetable proteins (red meats, milk products, and legumes such as beans, peas, lentils, and nuts), and lower levels of tuber and fruit consumption, which are nonetheless still quite common among SDCP farmers. Overall, the results provide some evidence that programme beneficiaries were able to exhibit higher levels of food diversification towards more nutritional food items.
Gender Differential Effects
In this section, we investigate differential programme effects by gender of the household head. As indicated earlier, one of SDCP goals is to serve women dairy farmers and the dairy groups they form. As a result, being a female headed household increases the probability of participating in the programme by 11 percentage points (see Table 4). To the extent that female headed households tend to have a lower socio-economic status than male headed households (e.g., female headed households have fewer working age members, have fewer consumer durables or agricultural implements, and have fewer extensions of cultivated land), program impacts may be different in terms of this gender dimension.
We first explore programme impacts on household decision making. As part of the household survey, we collected information on numerous variables regarding who in the household participates in making decisions about dairy activities, including use of inputs and providers, management of money from milk sales, and request of dairy-related trainings. All the questions considered allow respondents to choose from five options about who makes the decision: household male, household female, joint household (male & female), non-household member, and other. For the analytical purposes, we created indicator variables for all these decision making questions where the variable is equal to 1 if a female member made the decision and 0 otherwise.
We present the results for some key variables in Table 15. (5) For most of the outcomes considered, we see a positive effect of the programme on the probability of making a decision relative to the comparison group. Treated households are nine percentage points more likely to have a female managing the cash from fresh milk sold relative to non-SDCP households. Also, SDCP households are 7 and 11 percentage points more likely than the comparison group to have a female deciding the provider for bull and artificial insemination services respectively. Lastly, household beneficiaries are more likely to have a female deciding over the use of different services such as use of anthelmintics, tick control, vaccination, and curative treatments.
The overall results indicate that the SDCP programme had a positive impact on the welfare of female dairy farmers by allowing women to be more empowered when making decisions related to dairy production. These quantitative results are encouraging since the qualitative results did not provide insight into the gender dynamics of dairy farming. Gender was included as a topic in FGDs, but farmers had very little interesting information to share about their perspectives on gender.
The total cost of the SDCP is 1,515,614,910 KSH for the period from 2006 through 2016. Appendix Table J1 includes the yearly expenses across categories. The largest category of expenses was the "Technical Assistance, Training, & Workshops" expense, which amounted to 33.4% of the overall budget. This large share of the budget is not surprising since many of the programme activities related to this line item. Other higher categories of expenses included "Salaries & Allowances," "Vehicle & Office Operating Costs," and "Vehicles, Equipment, & Materials," at 17.5%, 13.2%, and 14.5% respectively.
To compare the costs to the benefit that each farmer receives from the program, we first need an estimate of the costs per beneficiary. To determine the number of beneficiaries, we examined the Dairy Groups Inventory from the programme counties and divisions, which we summarized in Appendix Table J2. Most of the inventories were updated in 2012, with some having been revised in 2011. Across treatment divisions, there were 505 dairy groups with 15,535 total dairy group members. On average each dairy group had 30.8 members. Taking the total costs from Appendix Table J1 and the total number of beneficiaries from Appendix Table J2, we can estimate that the total costs per beneficiary were approximately 97,561 KSH (equal to 1,515,614,910 KSH divided by 15,535).
To compare the costs to the benefit that each farmer receives from the program, we next need a measure of the benefits of the programme at the farmer level. While we recognize that the programme is associated with a variety of farmer-level benefits, we focus on the benefit of increased milk production for the purpose of this analysis, since that was one of the main goals of the SDCP. The unconditional average of litres of milk produced yesterday for a farmer in the comparison group was 3.81 litres, where this estimate includes zero values for comparison group farmers who produced no milk yesterday. Our impact estimates included in Table 13 suggest that the program increases milk production by 37%, suggesting that the additional amount of milk produced yesterday by a farmer in the treatment group is 1.41 litres. Because our estimate of the milk produced yesterday includes values of zero for when the farmer did not produce milk that day, these estimates reflect the amount produced on an average day. We assume this average level of production throughout the year and multiply the estimate of 1.41 additional litres per day by 365 days in a year to estimate that the additional milk produced by a farmer in a year is 514.65 litres. In monetary terms this is equivalent to a benefit of 20,586 KSH per farmer since the unconditional mean, which excludes zeros for those who did not sell milk, of the price per litre of milk is 40 KSH. This benefit estimate corresponds to the 2016 time frame as the data on yesterday's milk production was collected in 2016.
With an estimated total cost per beneficiary of 97,561 KSH and an estimated per farmer benefit of 20,586 KSH in the year 2016, we calculate that it would take approximately 4.74 years (equal to 97,561 KSH divided by 20,586 KSH) to "break even," that is for the benefits to equal the costs, assuming the benefits of 20,586 KSH remain constant across years. This estimate of the number of years to break even seems reasonable for although there likely were no benefits at the beginning of the programme (when upfront costs were associated with setting-up the SDCP), farmers likely began benefiting from the programme prior to 2016. Furthermore, to the extent farmers continue to employ the best practices advocated by the programme, the benefits may extend into future years. In addition, these estimates seem reasonable because we only examined the benefit of increased milk production and the SDCP benefited treatment farmers in a variety of dimensions.
Discussion and Conclusions
Overall, the results suggest that the SDCP was successful in increasing milk production, but with more limited--though positive--impacts on increasing milk marketing and increasing milk prices received by smallholders. Thus, we first summarize results pertaining to milk production, and then on milk marketing. We end the section with a summary of the key challenges remaining.
SDCP households surveyed were more likely to have received information on all of the practices being promoted by SDCP versus control households, and they were also more likely to have adopted those practices as well. In addition, SDCP households were more likely to have cross-bred cows, to use AI services, and to have obtained a wider range of health services. Finally, they were also more likely to adopt recommended management practices and investments, including practicing zero grazing, having concrete floors, and feeding concentrates. In fact, control households did not perform better on any measures of input use, management and investment, with just one exception; SDCP households were less likely to clean cows teats before milking. Overall, these improved input and management practices led to greater milk.
In addition to increasing milk production per cow, another key objective of SDCP was to smooth milk production across the year, by increasing access to adequate feed and fodder throughout the year. Because of the difficulty in obtaining recall data on milk production throughout a year, we have no direct evidence on whether SDCP farmers actually have smoother outputs. We did find that SDCP farmers were more likely to adopt fodder establishment, hay making, silage making and conservation of crop residues, all of which should contribute to increasing availability of adequate fodder throughout the year. However, adoption rates for each of these practices are below 15%, suggesting there may still be room to improve activities and trainings aimed at smoothing fodder availability and thus milk production throughout the year.
Results from the household survey are consistent both with results from the dairy group survey and the FGDs. For instance, SDCP dairy group respondents noted higher access to trainings on almost all topics. In particular, 79% of SDCP dairy groups noted that at least one training had been provided on fodder management, well above the 36% in control dairy groups. However, household respondents were less likely to receive information on various practices associated in both groups, implying that relatively few households in the dairy group attended these trainings. In fact, the number of dairy group leaders and members attending various trainings was somewhat limited, at about 16 attendees for SDCP trainings and slightly less in control groups. The latter implies scope for increasing dissemination of knowledge learned at trainings to the wider group of dairy farmers who were not able to attend.
FGD participants thought that the SDCP project helped them achieve higher milk yields, higher income and greater food security. The most frequently discussed benefit was improved animal management. Many participants also discussed the need to make and preserve fodder to ensure year-round availability of fodder for their dairy cows, and SDCP farmers appear to have a more sophisticated knowledge of fodder management. However, they also expressed barriers some farmers faced in adopting best practices. In part, FGD participants thought increased availability and consistency of technical services provided by SDCP would help increase the number of adopters. Additionally, many farmers had misperceptions about how AI works, and many also thought the price was too high, particularly given that success is not ensured the first time.
Milk and Input Markets
Both total milk production and milk sold was higher for SDCP, though the impact is higher on production versus milk marketed. In part this appears to be due to the fact that SDCP producers are more likely to have sold any milk the preceding day, and in particular, they are more likely to have sold milk in the evening and to have sold both in the morning and the evening. Thus, this evidence suggests that there were positive impacts on milk marketing. On the other hand, from the household survey, the percent of households receiving information on market-related topics was in general lower than for production-related and farm management topics, for both SDCP and control groups, though SDCP households were more likely to receive market-related information than controls. For instance, just 9% of households noted that they had received information on fresh milk marketing, with even fewer having received information on value addition and cooperative milk marketing. Similarly, in the dairy group surveys, just 59% reported trainings related to marketing, significantly below the number reporting trainings on production related activities such as fodder management (79%).
The dairy group survey also reflects more limited impacts on marketing services they provide to their members. First, there are limited differences between treated and control dairy groups in the types of services offered. Just 20% of SDCP groups facilitate links between members and input suppliers, and just 24% facilitate linking members to milk purchases--similar to the percentages observed in control groups. Just 9% of SDCP dairy groups contract with input suppliers on behalf of members, and 11% contract with milk purchasers on behalf of members. However, while 11% is fairly low, it is significantly higher than control groups who contract with milk purchasers, at 2%. At least some treatment FGDs mentioned that they thought the project had reduced transactions costs, and one participant said that it had become easier to find a steadier customer base. On the other hand, many participants also noted entering the market was still a challenge for many farmers, and the FGD interviewers noted that farmers expressed a lack of understanding of how to expand their business.
One of the more successful initiatives to help farmers access input and output markets appears to be the expansion of access to credit, as primarily documented in the dairy group surveys and FGDs. SDCP diary groups were more likely to access a wider range of finance sources, including micro-finance and commercial, but particularly local savings & loan clubs. Similarly, treatment farmers in FGDs frequently mentioned SDCP trainings which led to the establishment of merry-go-round and table banking groups.
Overall, SDCP activities contribute to a positive overall picture of the impact and perceptions of the SDCP on Kenyan dairy farmers, particularly for dairy production. However, insights from the qualitative data also point to challenges in service provision, utilization of technical knowledge and skills, and institutional coordination that could enhance the program.
Uptake and Ongoing Services
Key informants notably expressed different primary challenges to dairy farming commercialization in Kenya than their beneficiary counterparts, albeit that the challenges were also elements that the SDCP actively aims to address. In addition to mentioning challenges associated with increasing uptake of improved feed management, marketing, and dairy farm management, key informants' top two most frequently mentioned challenges were (1) the availability and consistency of technical services that relate to the SDCP and (2) the lack of uptake and utilization of available knowledge and technical skills among farmers. Discussion of these two primary challenges was related. Many key informants particularly mentioned extension among the government-provided services that suffer because of lack of staff and funding. One informant said, "All development projects within the county go through the same officers within the county, [and the] number of staff are of staff are not enough to reach out to those communities." This statement highlights that despite the provision of SDCP activities, a staffing bottleneck may prevent these services from being implemented as intended. One trainer from the ministry of agriculture recommended they, "Visit the groups mid-year to see if they implement what we have trained them on and if they have any technical challenges."
Given that such services aim to help farmers with technical knowledge, it makes sense that utilization of available knowledge and technical skills was also a notable challenge. Even in cases where key informants believed that information about technical skills in farm management had reached the farmers, they then mentioned challenges with farmers being able to utilize the knowledge because of capital constraints. One key informant pointed out this constraint, "Much as we offer the extension, services may not be adequate and [...] farmers are slow in taking up the services that we offer." The same key informant connected the challenge of services with financial challenges on the part of the farmer, "Like in A.I--as much it is subsidized farmers still, complain that the price is too high. The farmers have not fully the idea of commercialization."
In addition to lack of capital to consistently implement farm and animal management practices, farmers also have challenges with negotiating entrance into the market. One dairy group chairperson said of the SDCP, "now that they have improved our lives by providing machines for handling milk, they should try to continue supporting us to see that we can even package and market for ourselves." This indicates that despite having had trainings on marketing, not all farmers have put the changes into practice. Farmers were also unable to concretely identify how to go about expanding their business in other ways. For example, one dairy group member said, "I would like if we could get capital to invest in our business of making feeds and we market them and sale to other places." However, capital was an issue in this group, as well as others. For example, another farmer said, "If we can get our own doctors and at least our group to pay for the treatment then they can deduct the cost in milk, then we would really prosper."
Institutional Coordination and Linkages
FGDs indicated there are challenges with linking farmers and dairy groups to various service providers, and that farmers do not always have the knowledge to negotiate to their benefit. Key informants agreed that bringing service providers together continued to be a challenge. One key informant said there continues to be "inadequate coordination of the value chain actors, especially in terms of contract terms and contractual agreements." Farmers generally did not discuss interactions with value chain actors other than AI providers and veterinary officers. One farmer said, "The service providers it's on need basis, while the agricultural extension officers we meet them on weekly basis as they liaise with other stake holders like world bank, [or others who] deal with breeding of animals and rearing of chicken ," indicating that they rely on extension officers for coordination of major animal management activities, in some cases.
There was no mention among dairy group farmers or their chairpeople of any formal contractual arrangements with banks, service providers, or other actors on the value chain. The lack of agreements was evident when one key informant said he wished there could be, "Proper structures on marketing of milk and increased production of milk from individual farmers. We should not experience case that happened in Nyandarua where they poured milk because they did not have a place to take it." One farmer did explain that his group entered into a marketing agreement: "we have approached an organization called Inter Region Economic Network and they came in and researched and decided they will bring us marketers for our products and as a result we shall sell our milk as individuals. So the agreement was they sell the milk in very huge amounts and they keep 15% and us the rest 85%." However, there was no other mention of such arrangements.
Finally, a major challenge to program implementation cited by multiple key informants was the inability to get SDCP funds through the various levels of the government to enable timely implementation of initiatives on the ground. Key informants mentioned particular elements of funding including disbursing funds from the national level of the government, enabling payments to service providers, and--even among farmers--the inability to purchase services, which some informants speculated comes from an impression among farmers that they should not have to pay for services or get monetary provisions for attending trainings. This challenge among farmers aligns with the prior idea that farmers have not fully taken up the trainings. One informant reiterated, "The challenges on finances [...] hamper implementation, [including with] group issues on governance, leadership and resource mobilization. They may have the knowledge, but still those conflicts arise."
At the level of government implementation, one key informant said there was a challenge with "Authority to incur expenditure; you cannot actually be allowed to spend before it's signed." At the time of the interview, it had been more than a month since the office was supposed to receive the authority to incur, yet they had not received it. Key informants provided mixed responses on whether devolution caused any of these challenges with SDCP implementation, though some respondents thought having an extra layer of bureaucracy may contribute to the slowdown of funds and information flow (despite that the program is still implemented from the national level).
Internal and External Validity
The results of the evaluation of the SDCP programme can be used to inform the design and implementation of similar dairy policies and projects both elsewhere in Kenya and in other developing counties. First, there are at least 16 counties in Kenya that, although not being served by the programme, have comparable resource poor dairy farmers who we would expect will benefit in similar ways from SDCP activities should they receive the programme. Although the road densities might differ from the densities selected by IFAD and GoK, the selected districts had large road densities and were also relatively contiguous in order to ease SDCP management. Thus, the results should extend to similar areas in Kenya with large road densities that are not necessarily contiguous. Second, many of the features and challenges that smallholder dairy farmers face in Kenya are shared by many dairy farmers in Eastern Africa. Most small dairy farmers from neighbouring countries face a sub-tropical geography suitable for dairy cattle and produce milk as an important part of their diet (Thorpe et. al., 2000). These dairy farmers also exhibit similar challenges such as lack of training and availability of production inputs that prevent them from commercializing their dairy products in the most efficient way.
More generally, the effects of the SDCP programme can provide some insights about other types of agricultural extension programmes to smallholder farmers in developing countries. A key objective of agricultural extension is to increase the knowledge of farmers about agricultural practices, which in turn should have an effect on productivity. The traditional extension model, known as "Training and Visit" (T&V) extension, is characterized by government-employed extension agents visiting farmers individually or in groups to demonstrate agricultural best practices (Anderson and Birner, 2007). However, the T&V approach has been criticized for a series of reasons. First, given the limited transportation infrastructure in rural areas and the high costs of delivering information in person in some developing countries the reach of extension programs are limited. Second, as it is costly to provide agricultural extension to farmers on a recurring basis, the infrequent and irregular meetings limit the ability to provide timely information, which further limits the ability of farmers to provide timely information. Lastly, usually the information provided to farmers is too top-down, which results in an inadequate diagnosis of the difficulties currently facing farmers, as well as information that is often too technical for semi-literate farming populations.
The SDCP programme has some key features that may help overcome some of the limitations of this traditional extension approach. In particular, a key aspect of the programme is that it aims at building the capacity of groups of farmers through trainings that allow them to identify their most pressing needs and design work plans to request more targeted assistance from the programme in terms of both information and resources. Also, by increasing the capacity of groups and not only individual farmers, the programme may not only be more cost efficient, but also generate some positive externalities to other farmers in the area who may later become part of the groups. Third, the programme incentivizes farmers to organize and compete for additional funds to improve their productivity. Our results are consistent with this approach to extension; however, our results also suggest that more needs to be done to help group members who participate in trainings to subsequently share the knowledge they learn to group members who did not participate. In sum, we think the results of this evaluation and the analysis of its costs, the design of other types of extension and approaches in other agricultural sectors and countries.
We also believe that the quantitative results provide useful information on the impacts of a complex project on smallholder dairy farmers in other regions in Kenya that nonetheless face similar market-shed characteristics, and in other neighboring countries with similar market characteristics. Given specificities associated with both milk production and marketing, we do not expect that results would be easily generalizable to other cash-crop markets, however.
Anderson, J., & Birner, R. (2007). How to make agricultural extension demand-driven?: The case of India's Agricultural Extension Policy (IFPRI Discussion Paper 00729). Washington, DC: International Food Policy Research Institute.
Atieno, R., & Kanyinga, K. (2008). The revitalization of Kenya Cooperative Creameries: the politics of policy reforms in the dairy sector in Kenya. Future Agricultures Research Paper 009. Future Agricultures Consortium.
Bernard, H. R. (2011). Research methods in anthropology: Qualitative and quantitative approaches (5th ed.). Lanham, MD: AltaMira Press.
Burke, W.J., Jayne, T., Freeman, H., & Kristjansen, P. (2007). Factors associated with rural farm households' movements into and out of poverty: the rising importance of livestock. International Development Working Paper 90, Department of Agricultural Economics, Michigan State University.
Burke, W, Myers, M., & Jayne, T. (2015). A triple-hurdle model of production and market participation in Kenya's dairy market. American Journal of Agricultural Economics, 97(4), 1227-1246
FAO Global Information and Early Warning System Country Brief on Kenya (April 25, 2017) available at http://www.fao.org/giews/countrybrief/country.jsp?code=KEN accessed June 16, 2017.
Gelan, A., & Muriithi, B. (2015). Examining returns to scale in smallholder dairy farms in East Africa. Quarterly Journal of International Agriculture, 54(3), 237-259.
Glaser, B., & Strauss, A. (2009). The discovery of grounded theory: Strategies for qualitative research. Chicago, IL: Alpine.
Heckman, J., Ichimura, H., Smith, J., & Todd, P. (1998). Characterizing selection bias using experimental data. Econometrica, 66(5), 1017-1098.
Heckman, J., Ichimura, H., & Todd, P. (1997). Matching as an econometric evaluation estimator: evidence from evaluating a job training programme. Review of Economic Studies, 64(4), 605-654.
Herrero, M., Thornton, P., Bernues, A., Baltenweck, I., Vervoort, J., van de Steeg, J., Makokha, S., van Wijk, M., Karanja, S., Rufino, M., & Staal, S. (2014). Exploring future changes in smallholder farming systems by linking scio-economic scenarios with regional and household models. Global Environmental Change, 24, 165-182.
Holloway, G., Nicholson, C., Delgado, C., Stall, S., & Ehui, S. (2000). Agroindustrialization through institutional innovation: transactionis costs, cooperatives and milk-market development in the East African highlands. Agricultural Economics, 23(3), 279-288.
Imbens, G., & Wooldridge, J. (2009). Recent developments in the econometrics of program evaluation. Journal of Economic Literature, 47(1), 5-86.
McKenzie, D. (2011, November 20). Power calculations for propensity score matching. In: Development Impact Blog. Retrieved from http://blogs.worldbank.org/impactevaluations/power-calculations-for-propensity-score-matching
Muriuki, H. G. (2003). A review of the small scale dairy sector--Kenya. Retrieved from Report. http://www.fao.org/fileadmin/templates/ags/docs/dairy/P1assessmentkenya.pdf
Nafula, S. (2013). The impact of enhanced nutritional and feeding practices on growth and health of dairy calves in Mukurweini District of Kenya. Masgters of Veterinary Medicine Thesis, University of Nairobi. Last accessed May 12 at: http://erepository.uonbi.ac.ke/bitstream/handle/11295/72378/The%20Impact%20Of%20Enhanced%20Nutritional%20And%20Feeding%20Practices%20On%20Growth%20And%20Health%20Of%20Dairy%20Calves%20In%20Mukurweini%20District%20Of%20Kenya.pdf?sequence=3&isAllowed=y
Ouma, R., Njoroge, L., Romney, D., Ochungo, P., Staal, S., & Baltenweck, I. (2007). Targeting dairy interventions in Kenya: A guide for development planners, researchers and extension workers (ILRI Manuals and Guides No. 1). Nairobi, Kenya: International Livestock Research Institute.
Robins, J., & Rotnitzky, A. (1995). Semiparametric Efficiency in Multivariate Regression Models with Missing Data. Journal of the American Statistical Association, 90(429), 122-29.
Robins, J., Rotnitzky, A., & Zhao, L. (1995). Analysis of Semiparametric Regression Models for Repeated Outcomes in the Presence of Missing Data. Journal of the American Statistical Association, 90(429), 106-21.
Sacerdote, B. (2004). Fixing broken experiments using the propensity score. In: A. Gelman & X-L Meng (Eds.), Applied Bayesian modeling and causal inference from incomplete-data perspectives (pp. 61-71). Hoboken, NJ: Wiley.
Staal, S., Delgado, C., & Nicholson, C. (1997). Smallholkdeer dairying under transactions costs in East Africa. World Development, 25, 779-794.
Thorpe, W., Muriuki, H. G., Omore, A., Owango, M. O., & Staal, S. (2000). Development of smallholder dairying in Eastern Africa with particular reference to Kenya. Paper prepared for the UZ/RVAU/DIAS/DANIDA-ENRECA Project Review Workshop, Harare, Zimbabwe.
Van der Laan, M., & Robins, J. (2003). Unified Methods for Censored Longitudinal Data and Causality. New York: Springer, Physica-Verlag.
Appendix A. Field notes and other information from formative work
Appendix C. Survey instruments (qualitative and quantitative)
Appendix E. Sample Size and Power Calculations
To calculate a sufficient sample size for this evaluation, we conducted a number of power analyses based on existing farm-level data for SDCP beneficiaries that were collected as part of a 2014 survey commissioned by SDCP to Capital Guardian Consulting. The data focused on a series of outcomes--such as household monthly income and expenditures, as well as milk production per cow per day--that we used for the power calculations. Note that the data were collected for 78 dairy groups in the nine programme counties.
The power analysis calculations are based on the following assumptions:
* Minimum detectable effect size (MDES) for income and production related outcomes of 0.2 standard deviations. This effect size is equivalent to a 27% increase in income and a 20% increase in milk production. Note that Monitoring and Evaluation (M&E) data from SDCP show that programme beneficiaries have increased their income and milk production over time by almost three times. While these estimates are confounded by the growth of economic conditions that would have happened even in the absence of the programme, we believe the assumed MDES are rather conservative.
* We will collect information for 154 dairy groups, which will serve as clusters for the analysis (77 treated and 77 comparison). For each cluster, we will randomly select 15 households.
* The estimated intra-cluster correlation (ICC) is 0.11 for monthly income, 0.08 for monthly expenditures, and 0.15 for milk production. Thus, to be conservative, we set the ICC to the largest of these values.
* The proportion of the outcome variance explained by the exogenous covariates is set to 0.1, which we assume to be rather low given the absence of baseline data for these outcomes. However, we will control for exogenous household characteristics and geographical fixed effects that should enable us to explain at least 10% of the outcome variation.
* Power is set to 0.80
* Alpha level is set to 0.05
* Given these assumptions, the estimated sample size to detect an effect size of at least 0.2 standard deviations is 2,309 households.
* However, we assume that 10% of observations will not fall within the common support of the propensity score matching estimation. Thus, it will be necessary to drop these observations, and so we inflate the sample size by the proportion of observations that fall within the common support, a procedure suggested by McKenzie (2011). As a result, we proposed to collect farm-level data from 2,565 households (=2,309/0.9).
* We do not expect to lose a large proportion of observations due to falling outside the common support, because through all of the activities, we are proposing to increase the comparability of the treatment and comparison groups, including the two-stage targeting process (observation- and criteria-based targeting) and the filter questionnaire.
Although one of the main goals of the SDCP is to reduce the poverty levels of programme beneficiaries, it is not feasible at this point to credibly determine the effect that the expected increase in income will have on the number of households that will be moved out of poverty. To do so, we would need the income distribution of the farmers in the comparison areas to estimate the proportion of households that will be placed above the poverty line as a result of the programme. In any case, we will estimate the effects of the programme on this indicator as part of the evaluation.
Appendix J. Cost Data for the Programme
Appendix Table J1. SDCP Yearly Expenditure by Category (in thousands of KSH) 2006/07 2007/08 2008/09 2009/10 Civil Works 32 3,310 9,548 839 Contracts for Service 216 10,650 7,604 25,350 Providers Dairy Goat Development costs 0 0 0 0 Grant for Dairy Goat Development for Women 0 166 78 5,303 Incremental Operating Costs: 11,871 20,015 25,226 26,531 Salaries & Allowances International Technical 0 0 0 0 Assistance for DTI International Technical 0 0 0 0 Assistance for Project Management & Coordination National Technical Assistance 1,515 3,181 3,252 4,102 & Policy National Technical Assistance 0 0 0 0 except for Support to Policy and Studies Technical Assistance, Training 5,676 18,881 42,938 42,030 & Workshops Vehicles & Office Operting 6,211 22,281 12,366 21,016 Costs Vehicles, Equip & Materials 44,460 25,981 12,289 5,492 Total 69,980 104,464 113,300 130,661 2010/11 2011/12 2012/13 2013/14 Civil Works 5,774 1,872 1,904 52,795 Contracts for Service 21,552 2,943 1,320 5,031 Providers Dairy Goat Development costs 0 0 14,680 0 Grant for Dairy Goat Development for Women 4,241 6,381 8,175 5,391 Incremental Operating Costs: 28,116 29,796 30,770 29,933 Salaries & Allowances International Technical 0 0 0 0 Assistance for DTI International Technical 0 0 0 0 Assistance for Project Management & Coordination National Technical Assistance 4,333 7,206 481 0 & Policy National Technical Assistance 0 0 0 0 except for Support to Policy and Studies Technical Assistance, Training 60,016 66,036 61,482 46,321 & Workshops Vehicles & Office Operting 29,478 28,412 26,638 26,600 Costs Vehicles, Equip & Materials 14,918 31,000 35,586 33,360 Total 168,428 173,646 181,037 199,431 2014/15 2015/16 Total % of Total Civil Works 2,247 20,226 98,546 6.5% Contracts for Service 739 13,642 89,048 5.9% Providers Dairy Goat Development costs 33,735 9,991 58,406 3.9% Grant for Dairy Goat Development for Women 9,389 6,079 45,203 3.0% Incremental Operating Costs: 29,794 33,050 265,103 17.5% Salaries & Allowances International Technical 0 0 0 0.0% Assistance for DTI International Technical 0 0 0 0.0% Assistance for Project Management & Coordination National Technical Assistance 7,797 0 31,866 2.1% & Policy National Technical Assistance 767 0 767 0.1% except for Support to Policy and Studies Technical Assistance, Training 86,101 77,278 506,757 33.4% & Workshops Vehicles & Office Operting 20,104 6,540 199,645 13.2% Costs Vehicles, Equip & Materials 9,404 7,784 220,274 14.5% Total 200,076 174,591 1,515,615 100.0% Note: Analyzed from file "SDCP - Total Cumulative Expenditure" provided from SDCP Technical team. Appendix Table J2: Dairy Groups Inventory (Revised as of 2011-2012) County Division # of Total # Avg. # of Dairy of Members Groups Members Bomet Ndarawet 15 316 21 Bomet Kembu 19 450 24 Bomet Sugumerga 18 652 36 Nyamira Borabu 22 2,649 120 Nyamira Ekerenyo 25 889 36 Nyamira Nyamira Peri-Urban 24 670 28 Central Keumbu 20 604 30 Kisii Central Mosocho 23 634 28 Kisii Central Suneka 25 672 27 Kisii Nakuru Subukia-Kabazi 13 360 28 Nakuru Njoro 12 286 24 Nakuru Rongai 8 163 20 Lugari Likuyani 12 258 22 Lugari Lugari 32 723 23 Lugari Matete 22 500 23 Nandi KABISAGA-SIGOT 18 507 28 Nandi Kapsabet 17 420 25 Nandi KABIEMITLOLINGERIT 18 425 24 Bungoma Kanduyi 23 571 25 Bungoma Ndivisi 24 554 23 Bungoma Tongaren 23 579 25 Trans Endebess 20 644 32 Nzoia Trans WAITALUK 23 677 29 Nzoia Trans Kiminini 18 471 26 Nzoia Uasin Kapsaret 14 472 34 Gishu Uasin MOI'S BRIDGE 17 389 23 Gishu Total 505 15,535 30.8
Appendix K.. do files
(1) The SDCP grouped the counties geographically into clusters to assist programme management
(2) The comparison area for Lugari division was initially Ikolomani. However, when data collection began, we learned that Ikolomani had only a farming cooperative, not farming groups. So, we replaced Ikolomani with Lurambi, the next best match in terms of propensity score.
(3) In Kenya, the following crops are produced during the main rainy season: sorghum, which is harvested by the end of September; beans and millet, which are harvested by the end of October; and maize and wheat, which are harvested by the end of November. Barley, maize, millet, sorghum, and beans are also produced during the short rainy season and are harvested at the end of March. See Kenyan Crop Calendar available at FAO Global Information and Early Warning System Country Brief on Kenya.
(4) These variables were also constructed by ILRI in 2005 at the division level as part of the programme placement exercise, but were not used as selection variables.
(5) Note that for those decision making variables with a low number of females making a given decision, the doubly robust estimator did not converge.
Juan Bonilla, American Institutes for Research (AIR)
Nancy McCarthy, AIR
Simon Mugatha, AIR
Nisha Rai, AIR
Andrea Coombes, AIR
Joshua Brubaker, AIR
Grantee Final Report
Accepted by 3ie: October 2017
Note to readers
This impact evaluation has been submitted in partial fulfilment of the requirements of grant TW4.2.01 awarded under Thematic Window 4. This version is being published online as it was received. A copy-edited and formatted version will be available in the 3ie Impact Evaluation Report Series in the near future. All content is the sole responsibility of the authors and does not represent the opinions of 3ie, its donors or its board of commissioners. Any errors and omissions are the sole responsibility of the authors. All affiliations of the authors listed in the title page are those that were in effect at the time the report was accepted. Any comments or queries should be directed to the corresponding author, Juan Bonilla at firstname.lastname@example.org.
Suggested citation: Bonilla, J, McCarthy, N, Mugatha, S, Rai, N, Coombes, A and Brubaker, J 2017. Impact evaluation of the smallholder dairy commercialization programme in Kenya, 3ie Grantee Final Report. New Delhi: International Initiative for Impact Evaluation (3ie)
Funding for this impact evaluation was provided by IFAD.
3ie's Agricultural Innovation Thematic Window is supported by the Alliance for Green Revolution in Africa, the Bill & Melinda Gates Foundation, the International Fund for Agricultural Development and UK aid
We recognize the contributions of many individuals and organizations, without whom it would not have been possible to complete this study. Our thanks go to the International Initiative for Impact Evaluation (3ie); Smallholder Dairy Commercialization Programme (SDCP); International Fund for Agricultural Development (IFAD); International Livestock Research Institute (ILRI); and Research Solutions Africa for the opportunity to carry out this study and for the financial and technical support that they provided. We further appreciate 3ie for the technical review and support provided throughout the study.
Our special thanks go to Mr. Jasper Grosskurth and Mrs. Jasper Gosselt (Research Solutions Africa) for carrying out the data collection, and to Ms. Pamela Ochungo for her help with recreating the original ILRI targeting maps. The value of the technical and logistical support obtained from Mr. Luke Kessei cannot be overemphasized. Everyone at the SDCP provided valuable logistical support during data collection, including program background information.
Our acknowledgments would be incomplete without mentioning our team of very able research assistants in Kenya. Specifically, we acknowledge the input of the team of enumerators and supervisors from Research Solutions Africa, whose dedication during data collection ensured that high-quality data were collected. The highly competent team of programming and data cleaning personnel at Research Solutions Africa is also gratefully acknowledged.
We also wish to acknowledge the patience exercised by the Kenyan households, community leaders, and community members during interviews. It is our hope that insights from the information they provided will translate into valuable interventions in their communities.
Acronyms 3ie International Initiative for Impact Evaluation AI Artificial Insemination AIR American Institutes for Research ATE Average Treatment Effect DCA Dairy Commercialization Areas DFID Department for International Development FANTA Food and Nutrition Technical Assistance Project GDP Gross Domestic Product GOK Government of Kenya HDDS Household dietary diversity score ICC Intra-cluster correlation IFAD International Fund for Agricultural Development ILRI International Livestock Research Institute IPWRA Inverse-probability-weighted regression adjustment IRB Institutional Review Board MDES Minimum detectable effect size M&E Monitoring & Evaluation SDCP Smallholder Dairy Commercialization Programme T&V Training and Visit Table 1: SDCP Divisions and Evaluation Study Areas County SDCP Divisions SDCP Division(s) Comparison Area Divisions Included in the Included in the Evaluation Evaluation Cluster: Bomet, Nyamira, and Central Kisii Bomet Bomet Central Sigor Ndanai Longisa Sigor Nyamira Borabu N/A N/A Ekerenyo Nyamira Central Keumbu N/A N/A Kisii Mosocho Suneka Cluster: Nakuru Nakuru Rongai Rongai Kamara Nakuru Njoro Njoro Keringet Nakuru Subukia-Kabazi: Subukia-Kabazi: Elburgon and Olenguruone Mbogoini and Mbogoini and Bahati Bahati Cluster: Lugari, Bungoma, Nandi, Trans Nzoia, and Uasin Gishu Lugari Lugari Lugari Lurambi (2) Lugari Likuyanki Likuyanki Ileho Lugari Matete Matete Kabras Bungoma Kanduyi Kanduyi Chwele Ndivisi N/A N/A Tongaren Nandi Kabiyet Kapsabet Kilibwoni Kosirai Trans Endebess N/A N/A Nzoia Kiminini Uasin Kapsaret N/A N/A Gishu Soy Turbo Table 2: Qualitative Sampling Stakeholder Group Geographical Number of Component Level Interviews Focus SDCP Coordinating Unit National 4 5 (1, 2) Dairy Training Institute National 1 3 Kenya Animal Genetic National 1 3 Resources Centre Department of Livestock Regional 3 3, 4 Community Artificial County Nakuru, 2; Lugari, 1 3 Insemination Station Input Supplier County Nakuru, 1; Lugari, 1 3, 4 County Programme County Nakuru, 2; Lugari, 3 5 (1, 2) Coordination Team Treatment Dairy Group County 5 1, 2 Chairpersons Treatment Smallholder County 12 (1 man, 1 woman 1, 2 Dairy Groups for six dairy groups) Comparison Dairy Group County 1 1, 3 Chairpersons Comparison Dairy Groups County 4 (1 man, 1 woman 1, 3 for two dairy groups) Table 3: Pre-Programme, Division-Level Characteristics used for Matching (N=183) Placement Mean Mean Coefficient Standard p-value variables control treatment error Milk density 85,389 81,099 -0.000008 0.000009 0.398 (litres /k[m.sup.2]/year) Proportion of 0.53 0.53 0.15 3.49 0.967 poor people Travel time to 0.39 0.40 -0.55 0.92 0.557 nearest urban centre Table 4: Determinants of treatment probability Independent Variable Mean Coefficient SE p-value Household demographics Main language is English = 1 0.08 -0.011 0.028 0.70 No. people in working-age group 2.94 -0.005 0.006 0.43 No. children 5 years of age and 0.48 0.017 0.012 0.14 younger Age of household head 52.3 0.003 0.001 0.00 Max. years of schooling 13.0 0.003 0.004 0.46 Female head of household = 1 0.24 0.112 0.025 0.00 Income and agriculture characteristics Consumer durables index (SD) 0.00 0.018 0.010 0.08 Ag. Implements index (SD) 0.00 0.019 0.008 0.01 Total size of landholdings (acres 0.19 0.015 0.027 0.59 purchased with title) Farm is flat or with slight 0.87 -0.008 0.028 0.77 slope = 1 Plot with any system of 0.02 -0.082 0.075 0.28 irrigation = 1 Plot with any irrigation 0.02 -0.057 0.079 0.47 structure = 1 Household cultivated any crop in 0.88 0.100 0.033 0.00 primary season No. Zebu cattle in division in 1071.1 -0.001 0.000 0.00 2005 No. Cross cattle in division in 1388.2 0.001 0.000 0.00 2005 No. grade cattle in division in 959.1 0.00003 0.000 0.83 2005 Development projects in community Any agriculture-based project 0.55 -0.002 0.088 0.99 other than SDCP = 1 Any livestock-based project = 1 0.12 0.078 0.090 0.39 N 2558 Notes: This table shows the results of a linear model for the probability of being in the treatment group as a function of household, agricultural, and community characteristics. The table shows the mean for each explanatory variable, the change in the propensity of being in the treatment group after a change of one unit on a given characteristic, the standard error of each estimate, and the corresponding p-value. Table 5: Covariate balance tests over treatment groups with inverse-probability weighting Dependent Variable Mean Coefficient SE p-value Household demographics Main language is English = 1 0.08 -0.015 0.017 0.365 Number of people in the working 2.94 -0.105 0.098 0.285 age group Number of children 5 years of age 0.48 -0.004 0.032 0.903 and younger Age of household head 52.3 0.002 0.012 0.838 Max. years of schooling of 13.0 -0.140 0.163 0.390 household member Female head of household 0.24 0.002 0.020 0.914 Income and agriculture characteristics Consumer durables index (SD) 0.00 -0.083 0.088 0.344 Ag. Implements index (SD) 0.00 -0.069 0.086 0.422 Size of landholdings (acres 0.19 0.004 0.016 0.782 purchased with title) Household cultivated any crop in 0.88 -0.005 0.015 0.739 primary season Household cultivated any crop in 0.02 -0.007 0.022 0.737 secondary season Area cultivated in primary season 0.22 -0.013 0.021 0.552 (log of ha) Area cultivated in secondary season 0.05 -0.011 0.021 0.599 (log of ha) Farm is flat or with slight 0.87 -0.006 0.015 0.683 slope = 1 Any system of irrigation = 1 0.02 -0.009 0.012 0.430 Any irrigation structure = 1 0.02 -0.009 0.011 0.444 Development projects in community Community has any agriculture-based 0.55 -0.007 0.022 0.768 project other than SDCP = 1 Any project in community is focused 0.12 -0.014 0.017 0.412 on livestock Notes: Each row in this table corresponds to an OLS regression of the specified variable on the treatment indicator. The table shows the mean for each dependent variable, the difference in means between the treatment and control groups (coefficient), the standard error for each difference, and the corresponding p-value. Table 6: Availability of Other Agricultural and Livestock Support Programs Characteristics SDCP Control All Community has any agriculture-based project other 0.56 0.54 0.55 than SDCP Any project in community is focused on agriculture 0.42 0.45 0.43 Any project in community is focused on livestock 0.15 0.09 0.12 Table 7: Impacts on feeding practices Dependent Variable Impact Comparison Treatment N Estimate Mean Mean (1) (2) (3) Practice zero grazing 0.08 (***) 0.09 0.17 2558 (0.02) Paddock/boma/stall floor 0.05 (***) 0.07 0.12 2558 made of concrete (0.02) Purchase protein-rich fodder 0.01 0.02 0.03 2558 (0.01) Feed cattle crop residues -0.00 0.84 0.83 2558 (0.03) Feed cattle concentrate 0.13 (***) 0.66 0.79 2558 feeds/mineral supplements (0.04) Feeding off the ground in 0.12 (***) 0.18 0.29 2558 trough when milking (0.03) Vary feeding by lactation 0.03 0.55 0.58 2558 stage (0.04) Experience shortage of feeds -0.01 0.74 0.73 2558 from their farm (-0.03) Notes: Impacts are estimated using the inverse-probability-weighted regression adjustment (Doubly-robust estimator). Robust standard errors are in parentheses. All estimations control for household and household head characteristics; plot and crop production characteristics for primary and secondary rainy seasons; availability of other agricultural and livestock support programs in village; pre-program division-level characteristics including milk density in liters per square kilometers, proportion of people below the poverty line, travel time to the nearest cooling center and to nearest urban center. (***) p<0.01; (***) p<0.05; (***) p<0.1 Table 8: Impacts on access and use of livestock technology Dependent Variable Impact Comparison Treatment N Estimate Mean Mean (1) (2) (3) Household monitors cattle 0.07 (***) 0.88 0.94 2558 regularly (0.02) Technology for own bull -0.02 0.29 0.27 2558 service is available (0.04) Own Bull Service used in the -0.01 0.13 0.12 2558 last 12 months (0.03) Technology for AI service is 0.05 0.60 0.65 2558 available (0.04) AI Service used in the last 0.12 (***) 0.24 0.36 2558 12 months (0.03) Notes: Impacts are estimated using the inverse-probability-weighted regression adjustment (Doubly-robust estimator). Robust standard errors are in parentheses. All estimations control for household and household head characteristics; plot and crop production characteristics for primary and secondary rainy seasons; availability of other agricultural and livestock support programs in village; pre-program division-level characteristics including milk density in liters per square kilometers, proportion of people below the poverty line, travel time to the nearest cooling center and to nearest urban center. (***) p<0.01; (***) p<0.05; (***) p<0.1 Table 9: Impacts on animal health services Dependent Variable Impact Comparison Treatment N Estimate Mean Mean (1) (2) (3) Had access to anthelmintics 0.02 0.90 0.92 2558 (0.02) Used anthelmintics 0.02 0.89 0.91 2558 (0.02) Had access to tick control 0.00 0.89 0.89 2558 service (0.03) Used tick control service 0.00 0.88 0.89 2558 (0.03) Had access to vaccination 0.25 (***) 0.46 0.71 2558 service (0.04 Used vaccination service 0.26 (***) 0.43 0.68 2558 (0.04) Had access to curative 0.10 (***) 0.21 0.30 2558 treatment service (0.04) Used curative treatment 0.07 (**) 0.21 0.28 2558 service (0.03) Notes: Impacts are estimated using the inverse-probability-weighted regression adjustment (Doubly-robust estimator). Robust standard errors are in parentheses. All estimations control for household and household head characteristics; plot and crop production characteristics for primary and secondary rainy seasons; availability of other agricultural and livestock support programs in village; pre-program division-level characteristics including milk density in liters per square kilometers, proportion of people below the poverty line, travel time to the nearest cooling center and to nearest urban center. (***) p<0.01; (***) p<0.05; (***) p<0.1 Table 10: Impacts on availability on cattle extension services Dependent Variable Impact Comparison Treatment N Estimate Mean Mean (1) (2) (3) Extension visits available 0.14 (***) 0.19 0.33 2558 (0.03) Field days available 0.18 (***) 0.18 0.36 2558 (0.04) Demonstrations are available 0.12 (***) 0.14 0.27 2558 (0.03) Notes: Impacts are estimated using the inverse-probability-weighted regression adjustment (Doubly-robust estimator). Robust standard errors are in parentheses. All estimations control for household and household head characteristics; plot and crop production characteristics for primary and secondary rainy seasons; availability of other agricultural and livestock support programs in village; pre-program division-level characteristics including milk density in liters per square kilometers, proportion of people below the poverty line, travel time to the nearest cooling center and to nearest urban center. (***) p<0.01; (***) p<0.05; (***) p<0.1 Table 11: Impacts on probability of receiving information from trainings Dependent Variable Impact Comparison Treatment N Estimate Mean Mean (1) (2) (3) Crop best practices 0.04 0.17 0.21 2558 (0.03) Livestock best practices 0.15 (***) 0.22 0.37 2558 (0.03) Improved fermentation of milk 0.06 (***) 0.01 0.07 2558 best practices (0.01) Other milk processing and 0.10 (***) 0.03 0.13 2558 quality control best (0.01) practices Managerial, bookkeeping, 0.04 (***) 0.03 0.07 2558 accounting, and finance (0.01) Fodder Establishment 0.07 (***) 0.05 0.13 2558 (0.02) Hay Making 0.06 (***) 0.06 0.12 2558 (0.02) Silage Making 0.04 (**) 0.05 0.09 2558 (0.02) Use of chaff cutter 0.05 (***) 0.01 0.05 2558 (0.01) Conservation (crop residues) 0.03 (***) 0.02 0.04 2558 (0.01) Animal registration 0.03 (***) 0.01 0.04 2558 (0.01) Fresh milk marketing 0.07 (***) 0.02 0.09 2558 (0.01) Value addition marketing 0.05 (***) 0.01 0.06 2558 (e.g., mala, yoghurt) (0.01) Group/Cooperative Milk 0.02 (**) 0.02 0.04 2558 Marketing (0.01) Market Information Searching 0.02 (***) 0.01 0.02 2558 (0.01) Notes: Impacts are estimated using the inverse-probability-weighted regression adjustment (Doubly-robust estimator). Robust standard errors are in parentheses. All estimations control for household and household head characteristics; plot and crop production characteristics for primary and secondary rainy seasons; availability of other agricultural and livestock support programs in village; pre-program division-level characteristics including milk density in liters per square kilometers, proportion of people below the poverty line, travel time to the nearest cooling center and to nearest urban center. (***) p<0.01; (***) p<0.05; (***) p<0.1 Table 12: Impacts on number of animals Dependent Variable Impact Comparison Treatment N Estimate Mean Mean (1) (2) (3) Number of cattle owned 0.49 (***) 2.31 2.80 2558 (0.15) Number of exotic breed 0.14 1.18 1.14 2558 cattle owned (-0.28) Number of crossed breed 0.29 (**) 0.68 0.97 2558 cattle owned (0.14) Number of cows owned 0.35 (***) 0.95 1.31 2558 (0.08) Number of milking cows 0.44 (***) 0.57 1.01 2558 (0.07) Number of exotic breed cows 0.03 0.57 0.60 2558 owned (0.08) Number of crossed breed cows 0.16 (***) 0.28 0.44 2558 owned (0.06) Notes: Impacts are estimated using the inverse-probability-weighted regression adjustment (Doubly-robust estimator). Robust standard errors in parentheses. All estimations control for household and household head characteristics; plot and crop production characteristics for primary and secondary rainy seasons; availability of other agricultural and livestock support programs in village; pre-program division-level characteristics including milk density in liters per square kilometers, proportion of people below the poverty line, travel time to the nearest cooling center and to nearest urban center. (***) p<0.01; (***) p<0.05; (***) p<0.1 Table 13: Impacts on milk production and total value Dependent Variable Impact Comparison Treatment N Estimate Mean Mean (1) (2) (3) Sold milk yesterday any time 0.08 (**) 0.42 0.50 2558 (0.04) Sold milk in the morning 0.07 0.41 0.48 2558 (0.04) Sold milk in the evening 0.06 (**) 0.20 0.26 2558 (0.03) Total litres of milk sold in 0.12 0.86 0.98 2558 the morning (yesterday) (0.10) Total litres of milk sold in 0.11 0.38 0.49 2558 the evening (yesterday) (0.07) Total litres of milk sold 0.18 0.96 1.14 2558 (yesterday) (0.11) Selling price of milk/litre 0.31 (*) 1.81 2.13 2558 (yesterday) (0.18) Total value of milk sold 0.43 (*) 2.47 2.90 2558 (0.25) Total milk production at 0.58 (***) 1.33 1.92 2558 calving (0.12) Total milk production 0.37 (***) 1.13 1.50 2558 (yesterday) (0.11) Notes: Impacts are estimated using the inverse-probability-weighted regression adjustment (Doubly-robust estimator). Robust standard errors are in parentheses. All estimations control for household and household head characteristics; plot and crop production characteristics for primary and secondary rainy seasons; availability of other agricultural and livestock support programs in village; pre-program division-level characteristics including milk density in liters per square kilometers, proportion of people below the poverty line, travel time to the nearest cooling center and to nearest urban center. (***) p<0.01; (***) p<0.05; (***) p<0.1. All production values and selling price are expressed in logs. Total milk production includes milk sold, consumed in the household, and lost. Table 14: Impacts on food categories consumed in last 7 days Dependent Variable Impact Comparison Treatment N Estimate Mean Mean (1) (2) (3) Cereals/grains -0.00 0.99 0.98 2558 (0.01) Potatoes/yams/cassava -0.14 (***) 0.89 0.75 2558 (0.03) Vegetables 0.00 0.99 0.99 2558 (0.01) Fruits -0.11 (***) 0.89 0.78 2558 (0.03) Beans/peas/lentils/nuts 0.03 (*) 0.88 0.91 2558 (0.02) Red meats/other organ meats 0.17 (***) 0.36 0.54 2558 (0.04) Poultry 0.00 0.35 0.36 2558 (0.03) Eggs 0.02 0.60 0.62 2558 (0.04) Fresh/dried fish/shellfish -0.02 0.37 0.35 2558 (0.03) Milk/cheese/yogurt/other milk 0.09 (***) 0.80 0.89 2558 product (0.03) Oils and fats 0.02 0.88 0.90 2558 (0.03) Sweets/sugar/honey 0.03 0.91 0.95 2558 (0.02) Other (condiments, coffee, 0.06 (**) 0.89 0.95 2558 tea) (0.02) Household dietary diversity 0.16 9.81 9.97 2558 score (HDDS) (0.14) Notes: Impacts are estimated using the inverse-probability-weighted regression adjustment (Doubly-robust estimator). Robust standard errors are in parentheses. All estimations control for household and household head characteristics; plot and crop production characteristics for primary and secondary rainy seasons; availability of other agricultural and livestock support programs in village; pre-program division-level characteristics including milk density in liters per square kilometers, proportion of people below the poverty line, travel time to the nearest cooling center and to nearest urban center. (***) p<0.01; (***) p<0.05; (***) p<0.1 Table 15: Impacts on women decision making outcomes Dependent Variable Impact Comparison Treatment N Estimate Mean Mean (1) (2) (3) Female manages money from 0.09 (*) 0.21 0.30 2558 fresh milk sold (0.05) Female decides own bull 0.07 (*) 0.11 0.18 2558 service provider (0.04) Female decides AI service 0.11 (***) 0.03 0.15 2558 provider (0.03) Female requested livestock 0.11 (***) 0.10 0.20 2558 best practices training (0.03) Female requested cattle best 0.06 0.17 0.23 2558 practices training (0.04) Female decides use of 0.27 (***) 0.17 0.45 2558 anthelmintic (0.05) Female decides use of tick 0.19 (***) 0.21 0.40 2558 control service (0.06) Female decides use of 0.19 (***) 0.11 0.31 2558 vaccination service (0.05) Female decides use of 0.05 (*) 0.06 0.11 2558 curative treatment service (0.03) Notes: Impacts are estimated using the inverse-probability-weighted regression adjustment (Doubly-robust estimator). Robust standard errors are in parentheses. All estimations control for household and household head characteristics; plot and crop production characteristics for primary and secondary rainy seasons; availability of other agricultural and livestock support programs in village; pre-program division-level characteristics including milk density in liters per square kilometers, proportion of people below the poverty line, travel time to the nearest cooling center and to nearest urban center. (***) p<0.01; (***) p<0.05; (***) p<0.1