Printer Friendly

Effectiveness of antimalarial interventions in Nigeria: Evidence from facility-level longitudinal data.


With the number of global malaria-attributable deaths estimated at 438 000 in 2015, malaria is a major public health concern. (1) The disease imposes a burden on the household and the economy, leading, inter alia, to a higher probability of catastrophic expenditures and a lower level of human capital. (2) Despite the fact that malaria is treatable and related services are available at a low cost, antimalarial services remain generally underutilized. (1)

In resource-poor settings, several barriers exist. Examples include stock outages of antimalarial commodities, (3) prohibitively costly distance to the nearest provider, (4) the relative shortage of health personnel in malaria control, (3) and social insensitivities in the delivery of antimalarial services. (5) One of the most important barriers remains financial; (6,7) poorer households are unable to obtain antimalarial services, despite their availability, physical accessibility, and social acceptability.

To improve access to antimalarial services, many governments have lifted financial barriers and removed user fees. While most studies find a statistically significant association between user-fee removal and increased utilization of antimalarial services, (3,6-8) effectiveness of the policy is still regarded as inconclusive. (9)

Given that the overwhelming methodology of choice is difference-in-differences (D)[iD] with routine facility data, (7,9-11 ) shortcomings of the recent literature may be identified. They call into question the credibility of previously estimated results. First, the period within which user-fee removal is evaluated in existing studies has been considered too short. The limited data length implies that only immediate, but not longer-term, impacts are captured, leading to an imperfectly informed policy decision. (9,1112) Second, a relatively weak research design has been used. Difference-in-differences estimates in existing studies are likely to be biased, given the baseline non-comparability of intervention vis-a-vis control groups. (9,11,13,14) Finally, statistical analyses performed in earlier studies have been too simplistic. Given the increased use of longitudinal data, it has been demonstrated that existing studies have inadequately tackled the issues of non-stationarity, autocorrelation, and seasonality, which could lead to incorrect statistical inferences. (9,15,16)

This study evaluates a large-scale program that lifted user fees for antimalarial services at selected health facilities in 2010-2013 in Niger State, Nigeria--a country that has been harshly affected by the disease. The program was split into two phases, with the second phase introducing an additional intervention that was complementary to the first phase. Addressing the above shortcomings explicitly, the evaluation is performed separately for both phases. It is hypothesized that (a) the program was conducive to increased utilization of antimalarial services overall, and (b) the impacts were more pronounced in the second phase, given the complementarity of the interventions.


2.1 | Study settings

With 95 percent of the general population at risk and 50 percent of the adult population undergoing at least one episode annually, Nigeria carries the heaviest malaria burden in the world. (17) Situated in the North Central region and considered the largest state in the country, Niger State is no exception. Malaria accounts for 65 percent of all outpatient visits and constitutes a leading cause of child and maternal mortality in the state. (18)

The Niger State government has implemented numerous measures to battle malaria. Through nationwide health care reforms, malaria treatment in public facilities has been subsidized since 2010. (18)

In August 2010-March 2016, with assistance from international donors, the Niger State government was able to adopt the WHO's guidelines for malaria control, (3) and implement a program of antimalarial interventions, providing free antimalarial services and commodities at selected health facilities.

During the first phase in August 2010-June 2012, two types of interventions were provided. (18) The first intervention was malaria case management (MCM), which entailed a prompt provision of artemisinin-based combination therapies (ACTs) to febrile patients. The second intervention was the provision of long-lasting insecticidal bed nets (LLINs) and intermittent preventive treatments of malaria (IPTs) for pregnant women. With a view to preventing (rather than treating) malaria, pregnant women who visited a supported facility would be given an LLIN during their first antenatal visit as well as the first dose of sulfadoxine-pyrimethamine at 16 weeks and the second dose at 22 weeks.

In July 2012, the program expanded and included an additional intervention: parasite-based rapid diagnostic tests (RDTs). The purpose was to screen out non-malarial febrile illnesses, confirming treatment needs. This intervention and the first two interventions were implemented together from July 2012 to March 2016.

A total of 375--out of 1585--government health facilities across all 25 local government areas (LGAs) were included in the program. (18) The State Malaria Control Program and the Ministry for Local Government jointly undertook the selection. Examples of selection criteria included the fact that the facility had to (a) have antenatal services, (b) have an efficient system for drug storage, and (c) never have had prior interventions. (19) The implementation was orchestrated at the state level. Selected facilities were provided with technical assistance and fully subsidized antimalarial commodities, which were delivered directly to them. (18) They were not offered monetary incentives to participate in the program.

2.2 | Data

Data were drawn from three independent sources. Facility characteristics were retrieved from the Niger State Primary Health Care Development Agency and Local Government Malaria Control units. Utilization data were taken from monthly reports at the Niger State Malaria Elimination Program (SMEP), which, during the data-collection period, were available only in hard copy. (They were later digitized in 2016, yet the digital version has incomplete records and contains data from 2013 onward only--after the interventions had been introduced-making it unsuitable for pre-post evaluations.) Under the supervision of SMEP staff, data were manually inputted, with facility names de-identified prior to analyses.

The final sample was determined, based on data completeness and a sampling of representative LGAs. Following the SMEP's definition of LGA-specific burden of febrile illnesses, that is, the percentage of febrile patients (ie, those whose body temperature exceeds 37.8 degrees Celsius for at least 3 days) to the population within 1 year, the LGAs in Niger State were classified into three groups. (20) The first group had a 0-10 percent (low) burden, consisting of four

LGAs from two geopolitical zones (Niger South and Niger East Senatorial Districts). The second group had an 11-20 percent (medium) burden, comprising 18 LGAs from three zones (Niger South, Niger East, and Niger North). The final group included three LGAs from two zones (Niger East and Niger North), with an above 20 percent (high) burden. Finally, seven LGAs were selected from the three groups and, within each group, from each of the geopolitical zones. In cases where there was more than one LGA within each group/geopolitical zone subcategory, the LGA was randomly chosen. Details of the selected LGAs are provided in Table 1.

The selected LGAs were similar and different at the same time. While they had in common a mixture of primary and secondary health facilities and a perennial malaria transmission pattern, they were different in terms of spoken languages and ways of life. For example, among the medium-burden group, Bosso was inhabited by a Gbagyis-speaking tribe, Kontagora by a Hausas-speaking tribe, and Lapai by a Nupes-speaking tribe, representing the three major tribes in the state. No two LGAs in the study were completely alike, and it is argued that, together, the seven LGAs were representative of the state.

The sample was composed of all facilities with complete data (N = 150) in the seven LGAs. It included 99 facilities that received support from the program and consequently did not charge user fees for antimalarial services ("intervention facilities," henceforth) as well as 51 facilities that did not receive such support ("non-intervention facilities," henceforth). For each facility, 37 months of data (March 2010-March 2013) were collected, amounting to a panel of 5550 facility-month observations. The sample constituted 29 percent of facilities with complete monthly reports; only 523 out of 1585 government health facilities in Niger State reported data on a regular basis. Given the variation in terms of location, size, and operational level of facilities included, it is argued that the sample was reasonably representative of facilities at the state level.

2.3 | Empirical approach

The outcomes of interest include the number of febrile illness cases and the number of confirmed malaria cases, treated at a government health facility in a month. The first outcome captures the extent to which antimalarial services were accessible. It does not necessarily imply actual utilization, but rather addresses how the Niger population acted upon (as opposed to ignored) their suspicion of malaria. The second outcome indicates actual utilization of antimalarial services.

Consistent with existing studies, (13,21,22) the evaluation method for this study is difference-in-differences (D)[iD]. Based on mixed-effects negative binomial modeling, (10) the preferred specification is given by:

[mathematical expression not reproducible] (1)

Subscripts i, j, and t represent facility, LGA, and month, respectively. [Y.sub.jt] denotes the outcome of interest for facility i in LGA j at month t. [INT.sub.ij] is equal to 1 if facility i was an intervention facility, and 0 otherwise. [POST.sub.t] is equal to 1 in the months where the interventions were implemented, and 0 otherwise. [INT.sub.ij]. *[POST.sub.t] is the interaction term. [X.sub.ijt] represents a vector of facility characteristics. [LGA.sub.j]. denotes a vector of indicator variables representing each of the LGAs. TREND, is a linear time trend that starts from 1 at the beginning of the data-collection period and increases by 1 with each passing month until the end of the data-collection period (37). [MTH.sub.t] is a vector of indicator variables representing each of the 12 months in the calendar year. [PEAK.sub.t] is an indicator variable that takes the value of 1 for July-November entries and 0 otherwise. The definition is based on the 2013 report of the Niger State Ministry of Health, (18) which suggests that malaria transmission in the area has a seasonal peak in July-November. [U.sub.ij] represents facility-level random intercepts and [[epsilon].sub.ijt] the idiosyncratic error term. [beta], [gamma], and [alpha] are the parameters to be estimated. [[beta].sub.3] captures the DD (program) effect.

Different analyses can be performed. The data-collection period can be sectioned into the following:

* Period 1 (March-July 2010) where none of the facilities provided antimalarial services for free;

* Period 2 (August 2010-June 2012) where ACTs (MCM), LLINs, and IPTs were provided free-of-charge at intervention facilities; and

* Period 3 (July 2012-March 2013) where RDTs were added to the package of free antimalarial interventions at intervention facilities.

The panel of 37 months is sufficiently long to capture impacts of (a) the free provision of ACTs, LLINs, and IPTs, comparing Period 2 against Period 1, and (b) the free provision of the entire package of antimalarial interventions (inclusive of ACTs, LLINs, IPTs, and RDTs), comparing Period 3 against Period 1. Altogether, four DD estimates are derived in the analyses below, two for each of the outcome variables.

The preferred specification is deliberate. The inclusion of the first three variables in Equation 1 ([INT.sub.ij], [POST.sub.t], and [INT.sub.ij] *[POST.sub.t]) follows the standard DD approach. The other explanatory variables need further explanation. Vector X.., represents time-specific facility characteristics. It includes (a) whether facility i provided primary or secondary care, (b) the number of beds, (c) the number of skilled health personnel, (d) the ratio of female skilled health personnel, (e) the number of support staff, (f) distance from the facility to the farthest community served, and (g) distance from the facility to the nearest free-flowing water source. These variables are intentionally sourced for three reasons. First, they represent confounding factors that can be used to account for baseline differences between intervention and non-intervention facilities and for the non-random assignment of the intervention status. Second, as the unit of analysis is facility, it is important to view the outcome variables as outputs of a firm and to specify the regression to reflect the firm's production function. Here, the specification includes proxies of capital and labor inputs, as well as measures of technical capacity, representing factors of production. Finally, [X.sub.ijt] is needed to encapsulate the multidimensionality of access to health care. (4) In light of the fact that the intervention status (INT.) captures the affordability dimension of access, [X.sub.ijt], addresses the dimensions of social acceptability and physical accessibility through the following variables respectively: the ratio of female skilled health personnel and distance to the farthest community served.

The inclusion of the LGA fixed effects (LGA.) is consistent with existing studies. (23-25) They are used to capture unobserved, time-invariant area characteristics and used as the offset for incidence rates, accounting for the varying size of the target population across LGAs. (10,26)

The variables [TREND.sub.t], [MTH.sub.t], and [PEAK.sub.t], as well as the facility-level random intercepts ([u.sub.ij]), are intended to reflect the repeated measurement of the outcome variables across time. TRENDt accounts for non-stationarity of the outcome variables. MTHt and PEAK, address the seasonality of malaria transmission that is specific to Niger State. Finally, [u.sub.ij] tackles the problem of autocorrelation, allowing for temporal variations of the outcomes at the facility level. The inclusion of a linear trend (24,26-28) and month fixed effects, (23,26,29 ) and the use of mixed-effects modeling (10,26-28,30) are observed in other policy-evaluation studies.

Altogether, the DD specification in this study improves the validity and the precision of the estimated program effects. The addition of [X.sub.ijt], and LGA fixed effects minimizes estimation bias and offers a contribution over existing studies using facility-level data, which do not always account for confounding factors. (8,21) The incorporation of [TREND.sub.t], [MTH.sub.i], [PEAK.sub.t], and [u.sub.ij], ensures that appropriate statistical inferences can be made, addressing non-stationarity, seasonality, and autocorrelation that have been largely ignored in the literature. (16)

2.4 | Justification of the econometric model

The negative binomial model is carefully selected. It is compared against alternative count-data models, and proves to be most appropriate for this study. The negative binomial model has been shown to outperform log-linear OLS (7,16,31) and regular Poisson models in cases of over-dispersion; (29,32) Table 2 (explained below) shows that, for both outcomes of interest, the variances (SD-squared) are much larger than the means. It is also favored over zero-inflated models, following the fact that zero observations make up <5 percent of the sample. Finally, the negative binomial model is preferred to the overdispersed Poisson model. Deviance and AIC statistics of the two models, re-estimated as generalized linear models, were calculated, based on the preferred specification (without the hierarchical structure of the data). For the overdispersed Poisson model, standard errors were scaled based on the square root of Pearson chi-square-based dispersion. For the negative binomial model, standard errors were scaled based on the estimated alpha. The diagnostic statistics (unreported) were consistently lower for the negative binomial model. The exercise suggests that the negative binomial model provides a better fit.

2.5 | Justification of the evaluation method

This study employs the difference-in-differences (D)[iD] design, which is compared against two alternative evaluation methods. The first method is interrupted time series (ITS), consistent with studies that evaluate national policy changes using routine facility data. (29,31,33) In contrast with DD, the ITS design typically considers only the intervention group, using the pre-intervention period as the reference. The program effect is measured as a shift in the outcomes among the intervention group that is distinct from the pre-intervention trend. Nevertheless, as shown in Appendix SI, in this paper, the regression is run separately on both the intervention and the non-intervention groups to strengthen the ITS analysis, consistent with Zuckerman et al (33) and suggestions made by Lagarde. (16) The idea is to compare whether a change in the outcome variables can be more evidently observed among the intervention than the non-intervention group. The purpose of exploring the ITS design in addition to DD is to provide robustness checks. The DD design has indeed been considered the more rigorous approach, as it explicitly incorporates the nonintervention group, thereby having better degrees of freedom and more generalizability. (30)

The second method is comparative/controlled interrupted time series (CITS). The method has been proved to be valid and precise, (24,27,28,30) and is considered more flexible than DD, (25,27,30) rendering it superior under certain conditions. More specifically, while the DD method assumes that outcome trends are similar, that is, "common," between the intervention and the non-intervention groups, the CITS method relaxes the assumption and allows those trends to diverge. (27,30) The CITS desi gn is considered more appropriate if trends of the outcomes of interest statistically differ between the intervention and the non-intervention groups. The DD design is preferred otherwise.

Results from the ITS and the CITS methods are shown in Tables A.l and A.2 in Appendix SI, respectively. Incidence rate ratios (IRR) are reported based on mixed-effects negative binomial modeling. Table A.l reveals that the ITS results are qualitatively similar to the DD results discussed below, demonstrating robustness of the preferred model. Table A.2 shows that DD is preferred to CITS; the diverging trend hypothesis does not hold statistically, leading to overfitting under CITS. (30) An explanation why CITS performs less well here compared to other studies (24,27,28,30) is perhaps the fact that the number of pre-intervention data points in this study is too small (five data points: March-July 2010) to properly establish pre-intervention trends and observe trend divergence. (16,25,27)

2.6 | Justification of the methods to improve statistical inference

This study improves the statistical inference of the DD estimates, addressing seasonality, non-stationarity, and autocorrelation through the use of month fixed effects, a linear time trend, and mixed-effects modeling, respectively. While the method used to incorporate seasonality here is consistent with existing studies, (10,26,29) the literature differs on how to deal with non-stationarity and autocorrelation. To capture non-stationarity, time fixed effects have alternatively been used (21,23,25) and, in certain instances, they have been shown to outperform a linear trend in terms of model fit. (21,25) Also, to correct for the presence of first-order autocorrelation, clustered Huber-White standard errors have been employed, instead of mixed-effects modeling. (15,21,23,33) Since it is not clear how these methods should be compared in practice, (25) Table A.3 in Appendix SI offers results from negative binomial DD regressions, using time fixed effects and standard errors clustered at the LGA level. (15,21,23) The results reveal a pattern that is similar to the preferred estimates below, and show that using time fixed effects does not statistically lead to a higher log-likelihood value, compared to a linear trend. Consistent with the ITS and CITS exercises, the use of alternative methods to account for non-stationarity and autocorrelation reinforces the conclusions of this paper, illustrating their robustness. (25,27)


3.1 | Descriptive statistics

Table 2 presents descriptive statistics of the outcome variables: febrile illness cases and confirmed malaria cases. The table is divided into two panels. Panel A shows the averages and standard deviations of the outcome variables in the intervention and the non-intervention groups in four periods: the entire data-collection period of 37 months, as well as the above-defined Periods 1, 2, and 3. Within each group, mean differences between (a) Period 2 and Period 1 and (b) Period 3 and Period 1 are provided and tested using t tests with an equal-variance assumption. Panel B presents differences of the mean differences between the intervention and the non-intervention facilities. They are equivalent to difference-indifferences estimates, with covariates unaccounted for. Statistical significance of these estimates is based on equal-variance t tests.

According to Table 2, the antimalarial interventions seemed to be effective in increasing the number of febrile illness cases and confirmed malaria cases. The introduction of free ACTs, LLINs, and IPTs was associated with an average increase of 29.739 and 27.354 cases of febrile illness and malaria per facility per month, respectively. The addition of free RDTs led to a larger increase of 51.264 and 39.307 cases respectively. Nevertheless, these estimates are only preliminary, as they do not account for facility and LGA characteristics.

Table 3 shows descriptive statistics of facility characteristics. The intervention and the non-intervention groups are compared, using t tests. On average, the proportion of secondary-care facilities was 6.8 percent lower among the intervention group. The numbers of beds, skilled health personnel, and support staff were also lower, while the ratio of female skilled health personnel and distance to the nearest source of water were larger among the intervention group. Altogether, the table suggests that the intervention facilities provided less advanced care, were smaller in size, were further away from a mosquito-breeding site, and had a larger percentage of female health personnel. These differences are significant at the 1-5 percent level.

3.2 | Difference-in-differences evaluation

Based on the preferred specification, the results are provided in Table 4. The table is split into two panels, with Panels A and B representing each of the outcome variables. Column I refers to the comparison between Periods 2 and 1, capturing the impacts of the first phase of the program. Column II refers to the comparison between Periods 3 and 1, showing the impacts of the second phase. The bottom part of the table explicitly discusses the empirical approach. The DD estimates in Table 4 are based on the interaction term in Equation 1 (INT;j.* POST,). The incidence rate ratios (IRR) (provided in the same fashion as previous studies (10,26) ) and the marginal effects calculated against the predicted number of events, conditional on the random intercepts, are shown. All regression equations pass the Wald test, and the likelihood ratio (LR) tests justify the random-intercepts model. Overall, Table 4 offers support for the interventions. The DD impacts are positive and statistically significant; with the removal of user charges, facilities were able to provide care for more patients with symptoms of malaria. Focusing on the marginal effects, the introduction of free ACTs, LLINs, and IPTs was associated with an increase, at the 1 percent level, of 20.876 and 22.835 cases of febrile illness and malaria per month, respectively. The expansion to include RDTs led to an increase, at the 5 percent level, of 19.007 and 19.681 cases of febrile illness and malaria per month, respectively.


This study evaluates the impact of free antimalarial interventions, namely ACTs, LLINs, and IPTs provided in August 2010-June 2012 (ie, the first phase of the program) and the addition of RDTs from July 2012 onwards (ie, the second phase) in Niger State, Nigeria. The outcomes of interest include the number of febrile illness cases and confirmed malaria cases. Based on 37 months of data from 99 intervention and 51 non-intervention facilities, this study uses the difference-in-differences (D)[iD] method estimated with a negative binomial model with facility-level random intercepts. The preferred specification accounts not only for facility and LGA characteristics to minimize bias, but also for non-stationarity, seasonality, and autocorrelation to improve statistical inferences. The results suggest that the interventions were effective in stimulating health care utilization among malaria-potential (febrile) and malaria-confirmed patients. Compared to the pre-intervention period, the first phase of the program was associated with an increase of 20.876 and 22.835 cases, and the subsequent addition of RDTs led to an increase of 19.007 and 19.681 cases of febrile illness and malaria per month, respectively.

The fact that the overall program led to increased utilization of antimalarial services is consistent with the demand theory. Considering that the probability of obtaining free services among febrile patients (who may or may not have had malaria) was higher and that the expected cost was lower at an intervention than a nonintervention facility, it is unsurprising that the number of febrile illness and confirmed malaria cases was higher among intervention facilities.

The fact that the impacts were less strong in the second phase requires further explanation. The demand theory suggests that the impacts should be stronger in the second phase. The addition of free RDTs should increase utilization of RDTs, through the own-price effect, and that of ACTs, LLINs, and IPTs, through the cross-price effect (given the complementarity of these commodities), leading to a greater number of febrile illness and confirmed malaria cases. Nevertheless, the demand theory makes the ceteris paribus assumption: that all things remain constant. The assumption does not hold in dynamic settings. Empirical evidence has indicated that, if implemented properly for a sufficiently long period, antimalarial interventions can interrupt malaria transmission, reducing the incidence rate and, consequently, the number of malaria-related facility visits over time. (8,11,13) The introduction of free RDTs, in particular, has been associated with a lower rate of clinical services utilization, with non-malaria cases being more effectively screened out. (10) The decline in the utilization rate at a later stage can also be attributed to a lower level of enthusiasm among potential patients, with social mobilization efforts dwindling down as the program progresses. (26) It is argued that the impacts observed in the second phase here were a likely result of the above counteracting effects combined.

There are limitations to the study. First, the study may still be subjected to estimation bias. There may have been observed and unobserved qualities of the selected facilities--unrelated to the interventions--that drove the results. Observed characteristics among the intervention and the non-intervention groups clearly differed, as demonstrated by Tables 2 and 3. Unobserved differences may have also existed. The literature suggests that there are time-varying determinants of antimalarial services utilization that are difficult to account for. Examples include quality of care provided at health facilities, (9) ' (13,26) as well as the level of protective immunity (11,12) and exposure to malaria vectors (11) among different populations in different areas, all of which may change after the introduction of a large-scale intervention program and may impact the program effects.

However, an attempt has been made to reduce estimation bias in this paper. In contrast with existing studies, (21,31) the preferred specification here includes explanatory variables that reflect the selection criteria and control for (observed) baseline differences between intervention and non-intervention facilities. The inclusion of facility characteristics and LGA fixed effects (Table 4) not only improves the fit, but also suppresses the estimates (Table 2), which may be taken as a sign that bias weakens. (14) It is worth noting also that an imperfect study design is common among evaluation studies; (9,14) it is often infeasible to undertake a study that randomly assigns interventions into some sites (thereby circumventing selection bias) while ignoring others that may also need them.

Also, the data used may be subjected to sample selection bias. As mentioned earlier, only 523 out of 1585 health facilities reported utilization data consistently. There may have been characteristics that systematically differentiated reporting from non-reporting facilities. Moreover, there was likely some reporting bias. Among facilities that reported utilization data, there may have been a non-monetary incentive to misreport or be selective in what was reported. These issues may potentially bias the estimates upward.

Nevertheless, it should be emphasized that most data are imperfect. This can be generalized to individual-level data, which are typically considered superior in terms of sample size and breadth of information contained therein. Specifically, individual-level data are susceptible to at least two types of bias: (a) recall bias, as the information is extracted from subjectively answered questions, (22) and (b) response bias, as the survey may not be well understood by the respondents. Albeit also imperfect, facility-level data can be used to complement individual-level data. An advantage of facility-level data is that they are collected through standard-format reports, making the information acquisition process more objective and perhaps less prone to measurement error.

Despite its limitations, the study has many contributions. First, this study adds to the literature of user-fee removal. It carefully encompasses several dimensions of health care access in the analyses, for example, the number of skilled health personnel and geographical distance between the facility and the farthest community served, so that the affordability dimension can be more effectively investigated. The results suggest that the removal of facility-levied fees can lead to increased utilization of antimalarial services, even after controlling for other dimensions of access to health care.

The study also contributes methodologically. As discussed in the Methods section and shown partly in Appendix SI, this study explores an array of evaluation methods (DD, ITS, and CITS) and regression models (overdispersed Poisson and negative binomial, with time trends vs time fixed effects, with random intercepts vs clustered standard errors). It illustrates that, despite quantitative differences, the results are consistent. In so doing, this study motivates future studies to cautiously select their methodology and double-check whether their conclusions are robust across methodological options. Compared to existing studies using facility data that typically have a smaller sample size, this study also collects a long panel of data (N = 5550), which allows for a more thorough investigation of temporal changes. Finally, this study provides results that are grounded on a robust empirical analysis, accounting for facility characteristics and area fixed effects to reduce estimation bias and addressing inference concerns associated with longitudinal data, namely non-stationarity, seasonality, and autocorrelation. The methodological contributions are deliberately undertaken to make up for the imperfect study design.

The quantification of more accurate estimates would require a more comprehensive dataset that includes also non-reporting facilities and a prospective evaluation design that allows for a cleaner, more randomized distinction between the intervention and control groups. This will need to be left with future studies.


Joint Acknowledgment/Disclosure Statement: This research was partially funded through Grants for Development of New Faculty Staff, Ratchadaphiseksomphot Endowment Fund, Chulalongkorn University. The funding source had no involvement in the research process nor the research outcomes. We would like to thank the Ministry of Health under the Niger State Government of Nigeria and the Niger State Malaria Elimination Program for providing and facilitating access to the data. We would also like to thank the anonymous reviewers for constructive comments that helped to immensely improve the paper.

Disclosures: None.


Nopphol Witvorapong [iD]


1. World Health Organization (WHO). World Malaria Report 2015. Geneva: World Health Organization; 2015.

2. Barofsky J, Anekwe TD, Chase C. Malaria eradication and economic outcomes in Sub-Saharan Africa: evidence from Uganda. J Health Econ. 2015;44:118-136.

3. Awoleye OJ, Thron C. Improving access to malaria rapid diagnostic test in Niger State, Nigeria: an assessment of implementation up to 2013. Malar Res Treat. 2016:2016:7436265.

4. Kizito J, Kayendeke M, Nabirye C, et al. Improving access to health care for malaria in Africa: a review of literature on what attracts patients. Malaria J. 2012;11:55.

5. Chuma J, Okungu V, Molyneux C. Barriers to prompt and effective malaria treatment among the poorest population in Kenya. Malaria J. 2010;9:144.

6. Ponsar F, Van Herp M, Zachariah R, et al. Abolishing user fees for children and pregnant women trebled uptake of malaria-related interventions in Kangaba, Mali. Health Policy Plana. 2011;26(2):ii72 -ii83.

7. Watson SI, Wroe EB, Dunbar EL, et al. The impact of user fees on health services utilization and infectious disease diagnoses in Neno District, Malawi: a longitudinal, quasi-experimental study. BMC Health Serv Res. 2016;16:595.

8. Aregawi M, Lynch M, Bekele W, et al. Time series analysis of trends in malaria cases and deaths at hospitals and the effect of antimalarial interventions, 2001-2011, Ethiopia. PLoS ONE. 2014;9(11):e106359.

9. Lagarde M, Palmer N. The impact of user fees on health service utilization in low-and middle-income countries: how strong is the evidence? 6 World Health Organ. 2008;86(ll):839-848.

10. Larsen DA, Bennett A, Silumbe K, et al. Population-wide malaria testing and treatment with rapid diagnostic tests and artemether-lumefantrine in Southern Zambia: a community randomized step-wedge control trial design. Am J Trop Med Hyg. 2015;92(5):913-921.

11. Trape JF, Tall A, Diagne N, et al. Malaria morbidity and pyrethroid resistance after the introduction of insecticide-treated bednets and artemisinin-based combination therapies: a longitudinal study. Lancet Infect Dis. 2011;11:925-932.

12. Tiffany A, Moundekeno FP, Traore A, et al. Encouraging impact following 2.5 years of reinforced malaria control interventions in a hy-perendemic region of the Republic of Guinea. Malaria J. 2016;15:298.

13. Munos M, Guiella G, Roberton T, et al. Independent evaluation of the rapid scale-up program to reduce under-five mortality in Burkina Faso. Am J Trop Med Hyg. 2016;94(3):584-595.

14. Ng M, Colson KE, Fullman N, et al. Assessing the contribution of malaria vector control and other maternal and child health interventions in reducing all-cause under-five mortality in Zambia, 1990-2010. Am J Trop Med Hyg. 2017;97(suppl 3):58-64.

15. Bertrand M, Duflo E, Mullainathan S. How much should we trust differences-in-differences estimates? Q J Econ. 2004;191(1):249-275.

16. Lagarde M. How to do (or not to do)... Assessing the impact of a policy change with routine longitudinal data. Health Policy Plann. 2012;27(1):76-83.

17. Federal Ministry of Health (FMOH), Nigeria. National Malaria Elimination Policy. Abuja: Federal Ministry of Health; 2013.

18. Ministry of Health (MOH), Niger State, Nigeria. Niger state annual operational plans for malaria. Niger State: S. M. C. Program, Minna, State Ministry of Health; 2013.

19. Department of Health Planning, Research and Statistics (DPRS), Nigeria. Annual report, 2010. Nigeria: Federal Ministry of Health; 2010.

20. State Malaria Elimination Programme (SMEP), Niger State, Nigeria. Annual report, 2013. Niger State: Niger State Ministry of Health; 2013.

21. Boudreaux C, Chanthala P, Lindelow M. Assessing the elimination of user fees for delivery services in Laos. PLoS ONE. 2014;9(3):e89784.

22. McKinnon B, Harper S, Kaufman JS, et al. Removing user fees for facility-based delivery services: a difference-in-differences evaluation from ten Sub-Saharan African countries. Health Policy Plann. 2015;30(4):432-441.

23. Bitler MP, Carpenter CS. Effects of state cervical cancer insurance mandates on Pap test rates. Health Serv Res. 2017;52(1):156-175.

24. St.Clair T, Cook TD, Hallberg K. Examining the internal validity and statistical precision of the comparative interrupted time series design by comparison with a randomized experiment. Am J Eval. 2014;35(3):311-327.

25. Wing C, Simon K, Bello-Gomez RA. Designing difference in difference studies: best practices for public health policy research. Anna Rev Pub/ Health. 2018;39:453-469.

26. Zombre D, De Allegri M, Ridde V. Immediate and sustained effects of user fee exemption on healthcare utilization among children under five in Burkina Faso: a controlled interrupted time-series analysis. Soc Sri Med. 2017;179:27-35.

27. Somers MA, Zhu P, Jacob R, et al. The Validity and Precision of the Comparative Interrupted time Series Design and the Difference-indifferences Design in Educational Evaluation. New York City, NY: MDRQ2013.

28. Jacob R, Somers MA, Zhu P, et al. The validity of the comparative interrupted time series design for evaluating the effect of school-level interventions. Eval Rev. 2016;40(3):167-198.

29. Bernal JL, Cummins S, Gasparrini A. Interrupted time series regression for the evaluation of public health interventions: a tutorial. Int J Epidemiol. 2017;46(1):348-355.

30. St.Clair T, Hallberg K, Cook TD. The validity and precision of the comparative interrupted time-series design: three within-study comparisons. J Educ Behav Stat. 2016;41(3):269-299.

31. Lagarde M, Barroy H, Palmer N. Assessing the effects of removing user fees in Zambia and Niger. J Health Serv Res Po. 2012;17(1):30-36.

32. O'hara RB, Kotze DJ. Do not log transform count data. Methods Ecol Evol. 2010;1(2):118-122.

33. Zuckerman RB, Sheingold SH, Orav EJ, et al. Readmissions, observation, and the hospital readmissions reduction program. New Engl J Med. 2016;374:1543-1551.


Additional supporting information may be found online in the Supporting Information section at the end of the article.

How to cite this article: Witvorapong N, Yakubu KY. Effectiveness of antimalarial interventions in Nigeria: Evidence from facility-level longitudinal data. Health Serv Res. 2019;54:669-677.

Nopphol Witvorapong PhD (1) [iD] | Kolo Yaro Yakubu MSc (2)

(1) Center for Health Economics, Faculty of Economics, Chulalongkorn University, Pathumwan, Bangkok, Thailand

(2) Strengthening Accountability and Quality Improvement for Maternal, Newborn and Child Health Project, Pact Nigeria, Gombe, Nigeria


Nopphol Witvorapong, PhD, Center for Health Economics, Faculty of Economics, Chulalongkorn University, Pathumwan, Bangkok, Thailand.


Funding information

Ratchadaphiseksomphot Endowment Fund, Chulalongkorn University

DOI: 10.1111/1475-6773.13122
TABLE 1 Selected local government areas in the sample

           Burden of febrile
LGA        illnesses          Geopolitical zone

Bida       Low                South
Shiroro    Low                East
Lapai      Medium             South
Bosso      Medium             East
Kontagora  Medium             North
Agwara     High               North
Paikoro    High               East

           Facility types
LGA        Intervention    Non-intervention  Total

Bida       11               0                 11
Shiroro    15              13                 28
Lapai      15               5                 20
Bosso      15               3                 18
Kontagora  15              10                 25
Agwara     13              11                 24
Paikoro    15               9                 24
N          99              51                150

TABLE 2 Descriptive statistics of outcome variables and
difference-in-differences (DD) estimates without covariates

                            Number of monthly febrile illness cases
Facility type/period        Intervention facilities  Non-intervention
Panel A: descriptive
statistics of outcomes
of interest across periods
Entire sample period         116.362                   69.860
(37 months)                 (126.267)                (121.786)
Period 1: January-July        66.162                   50.616
2010 (Pre-intervention       (84.308)                 (91.385)
Period 2: August             116.426                   71.142
2010-June 2012              (119.579)                (127.840)
Phase 1)
Period 3: July 2012-         144.086                   77.277
June 2013 (Post-            (151.384)                (119.824)
intervention: Phase 2)
Difference: period 2 vs       50.265 (***)             20.526 (***)
Period 1                      [5.658]                  [8.440]
Difference: period 3 vs       77.925 (***)             26.661 (***)
Period 1                      [7.368]                  [8.632]
Panel B: mean
comparison between
facility types
Difference-in-                29.739 (***)
differences: period 2         [1.746]
vs Period 1
Difference-in-                51.264 (***)
differences: period 3         [2.902]
vs Period 1

                            Number of monthly malaria cases
Facility type/period        Intervention facilities  Non-intervention

Panel A: descriptive
statistics of outcomes
of interest across periods
Entire sample period          87.491                   52.974
(37 months)                  (97.348)                 (90.479)
Period 1: January-July        45.701                   37.749
2010 (Pre-intervention       (51.601)                 (68.702)
Period 2: August              88.458                   53.152
2010-June 2012               (92.612)                 (90.880)
Phase 1)
Period 3: July 2012-         108.237                   60.978
June 2013 (Post-            (119.064)                 (98.820)
intervention: Phase 2)
Difference: period 2 vs       42.757 (***)             15.403 (***)
Period 1                      [4.301]                  [6.035]
Difference: period 3 vs       62.536 (***)             23.229 (***)
Period 1                      [5.624]                  [6.971]
Panel B: mean
comparison between
facility types
Difference-in-                27.354 (***)
differences: period 2         [1.482]
vs Period 1
Difference-in-                39.307 (***)
differences: period 3         [2.391]
vs Period 1

Notes: Standard deviations are in parentheses and standard errors in
square brackets. Mean differences in Panel A are based on t tests,
assuming equal variances. The DD estimates in Panel B are based on t
tests, assuming equal variances, and do not account for any covariates.
(*) 10% level of significance, (**) 5% level of significance, (***)
1% level of significance.

TABLE 3 Descriptive statistics of facility-level characteristics

Variables                       Nature of variables

Primary care = 1 [Excluded      Time-invariant
Secondary care = 1              Time-invariant

Number of beds                  Time-invariant

Number of skilled health        Time-varying
personnel                       (monthly)
Ratio of female skilled health  Time-varying
personnel                       (monthly)
Number of support staff         Time-varying
Distance to farthest community  Time-invariant
served in km
Distance to the nearest source  Time-invariant
of water in km

Variables                       Intervention facilities

Primary care = 1 [Excluded          0.990
category]                          (0.100)
Secondary care = 1                  0.010
Number of beds                      3.611
Number of skilled health            5.372
personnel                          (7.646)
Ratio of female skilled health      0.548
personnel                          (0.369)
Number of support staff             3.201
Distance to farthest community      7.545
served in km                       (6.612)
Distance to the nearest source      4.091
of water in km                     (4.239)
N                                3663

Variables                       Non-intervention facilities

Primary care = 1 [Excluded           0.922
category]                           (0.269)
Secondary care = 1                   0.078
Number of beds                       5.212
Number of skilled health             5.873
personnel                          (13.482)
Ratio of female skilled health       0.492
personnel                           (0.390)
Number of support staff              6.541
Distance to farthest community       7.497
served in km                        (4.082)
Distance to the nearest source       3.764
of water in km                      (4.256)
N                                 1887

Variables                       Mean difference

Primary care = 1 [Excluded       0.068 (***)
category]                       [0.005]
Secondary care = 1              -0.068 (***)
Number of beds                  -1.601 (***)
Number of skilled health        -0.501 (**)
personnel                       [0.284]
Ratio of female skilled health   0.057 (***)
personnel                       [0.011]
Number of support staff         -3.340 (***)
Distance to farthest community   0.048
served in km                    [0.166]
Distance to the nearest source   0.327 (**)
of water in km                  [0.120]

Notes: Standard deviations are in parentheses and standard errors in
square brackets. Mean differences are based on t tests, assuming equal
(*) 10% level of significance, (**) 5 % level of significance,
(***) 1% level of significance.

TABLE 4 Effects of antimalarial interventions:
difference-in-differences (DD) estimates from mixed-effects negative
binomial regressions

                           First phase of intervention (Period 2 vs
                           period 1)
DD Estimates               (1)

Panel A: Outcome variable  = Number of febrile illness cases
IRR[95%CI]                 1.222 (***) [1.079,1.383]
Marginal effects           20.876 (***)
Log likelihood             -21 397.153
LR test                    1965.550 (***)
Panel B: Outcome variable  | Number of confirmed malaria cases
IRR [95% CI]               1.265 (***) [1.102,1.451]
Marginal effects           22.835 (***)
Log likelihood             -20 131.286
LR test                    2260.570 (***)
N                          4200
Bias reduction             Facility characteristics and LGA fixed
Seasonality                Month fixed effects and seasonal peak
Non-stationarity           Time trend
Autocorrelation            Facility-level random intercepts

                           Second phase of intervention (Period 3 vs
                           period 1)
DD Estimates               (II)

Panel A: Outcome variable
IRR[95%CI]                    1.184 (**) [1.016,1.379]
Marginal effects             19.007 (**)
Log likelihood              -10 865.192
LR test                     750.150 (***)
Panel B: Outcome variable
IRR [95% CI]                  1.228 (**) [1.042,1.446]
Marginal effects             19.681 (**)
Log likelihood              -10 247.998
LR test                     805.290 (***)
N                          2100
Bias reduction

Notes: The 95% confidence intervals of the IRRs are shown in square
brackets in the same row. Standard errors of the marginal effects are
in square brackets in the row below. Marginal effects are calculated
against the predict counts (number of febrile illness and confirmed
malaria cases), conditional on the random intercepts. LR test refers
to the likelihood ratio test against a regular negative binomial model
of the same specification without facility-level random intercepts.
(*) 10% level of significance, (**) 5% level of significance,
(***) 1% level of significance.
COPYRIGHT 2019 Health Research and Educational Trust
No portion of this article can be reproduced without the express written permission from the copyright holder.
Copyright 2019 Gale, Cengage Learning. All rights reserved.

Article Details
Printer friendly Cite/link Email Feedback
Author:Witvorapong, Nopphol; Yakubu, Kolo Yaro
Publication:Health Services Research
Geographic Code:7IRAN
Date:Jun 1, 2019
Previous Article:Tailored interventions by community pharmacists and general practitioners improve adherence to statins in a Spanish randomized controlled trial.
Next Article:An observational study of adolescent health outcomes associated with school-based health service utilization: A causal analysis.

Terms of use | Privacy policy | Copyright © 2020 Farlex, Inc. | Feedback | For webmasters