Effectiveness of antimalarial interventions in Nigeria: Evidence from facility-level longitudinal data.
With the number of global malaria-attributable deaths estimated at 438 000 in 2015, malaria is a major public health concern. (1) The disease imposes a burden on the household and the economy, leading, inter alia, to a higher probability of catastrophic expenditures and a lower level of human capital. (2) Despite the fact that malaria is treatable and related services are available at a low cost, antimalarial services remain generally underutilized. (1)
In resource-poor settings, several barriers exist. Examples include stock outages of antimalarial commodities, (3) prohibitively costly distance to the nearest provider, (4) the relative shortage of health personnel in malaria control, (3) and social insensitivities in the delivery of antimalarial services. (5) One of the most important barriers remains financial; (6,7) poorer households are unable to obtain antimalarial services, despite their availability, physical accessibility, and social acceptability.
To improve access to antimalarial services, many governments have lifted financial barriers and removed user fees. While most studies find a statistically significant association between user-fee removal and increased utilization of antimalarial services, (3,6-8) effectiveness of the policy is still regarded as inconclusive. (9)
Given that the overwhelming methodology of choice is difference-in-differences (D)[iD] with routine facility data, (7,9-11 ) shortcomings of the recent literature may be identified. They call into question the credibility of previously estimated results. First, the period within which user-fee removal is evaluated in existing studies has been considered too short. The limited data length implies that only immediate, but not longer-term, impacts are captured, leading to an imperfectly informed policy decision. (9,1112) Second, a relatively weak research design has been used. Difference-in-differences estimates in existing studies are likely to be biased, given the baseline non-comparability of intervention vis-a-vis control groups. (9,11,13,14) Finally, statistical analyses performed in earlier studies have been too simplistic. Given the increased use of longitudinal data, it has been demonstrated that existing studies have inadequately tackled the issues of non-stationarity, autocorrelation, and seasonality, which could lead to incorrect statistical inferences. (9,15,16)
This study evaluates a large-scale program that lifted user fees for antimalarial services at selected health facilities in 2010-2013 in Niger State, Nigeria--a country that has been harshly affected by the disease. The program was split into two phases, with the second phase introducing an additional intervention that was complementary to the first phase. Addressing the above shortcomings explicitly, the evaluation is performed separately for both phases. It is hypothesized that (a) the program was conducive to increased utilization of antimalarial services overall, and (b) the impacts were more pronounced in the second phase, given the complementarity of the interventions.
2 | METHODS
2.1 | Study settings
With 95 percent of the general population at risk and 50 percent of the adult population undergoing at least one episode annually, Nigeria carries the heaviest malaria burden in the world. (17) Situated in the North Central region and considered the largest state in the country, Niger State is no exception. Malaria accounts for 65 percent of all outpatient visits and constitutes a leading cause of child and maternal mortality in the state. (18)
The Niger State government has implemented numerous measures to battle malaria. Through nationwide health care reforms, malaria treatment in public facilities has been subsidized since 2010. (18)
In August 2010-March 2016, with assistance from international donors, the Niger State government was able to adopt the WHO's guidelines for malaria control, (3) and implement a program of antimalarial interventions, providing free antimalarial services and commodities at selected health facilities.
During the first phase in August 2010-June 2012, two types of interventions were provided. (18) The first intervention was malaria case management (MCM), which entailed a prompt provision of artemisinin-based combination therapies (ACTs) to febrile patients. The second intervention was the provision of long-lasting insecticidal bed nets (LLINs) and intermittent preventive treatments of malaria (IPTs) for pregnant women. With a view to preventing (rather than treating) malaria, pregnant women who visited a supported facility would be given an LLIN during their first antenatal visit as well as the first dose of sulfadoxine-pyrimethamine at 16 weeks and the second dose at 22 weeks.
In July 2012, the program expanded and included an additional intervention: parasite-based rapid diagnostic tests (RDTs). The purpose was to screen out non-malarial febrile illnesses, confirming treatment needs. This intervention and the first two interventions were implemented together from July 2012 to March 2016.
A total of 375--out of 1585--government health facilities across all 25 local government areas (LGAs) were included in the program. (18) The State Malaria Control Program and the Ministry for Local Government jointly undertook the selection. Examples of selection criteria included the fact that the facility had to (a) have antenatal services, (b) have an efficient system for drug storage, and (c) never have had prior interventions. (19) The implementation was orchestrated at the state level. Selected facilities were provided with technical assistance and fully subsidized antimalarial commodities, which were delivered directly to them. (18) They were not offered monetary incentives to participate in the program.
2.2 | Data
Data were drawn from three independent sources. Facility characteristics were retrieved from the Niger State Primary Health Care Development Agency and Local Government Malaria Control units. Utilization data were taken from monthly reports at the Niger State Malaria Elimination Program (SMEP), which, during the data-collection period, were available only in hard copy. (They were later digitized in 2016, yet the digital version has incomplete records and contains data from 2013 onward only--after the interventions had been introduced-making it unsuitable for pre-post evaluations.) Under the supervision of SMEP staff, data were manually inputted, with facility names de-identified prior to analyses.
The final sample was determined, based on data completeness and a sampling of representative LGAs. Following the SMEP's definition of LGA-specific burden of febrile illnesses, that is, the percentage of febrile patients (ie, those whose body temperature exceeds 37.8 degrees Celsius for at least 3 days) to the population within 1 year, the LGAs in Niger State were classified into three groups. (20) The first group had a 0-10 percent (low) burden, consisting of four
LGAs from two geopolitical zones (Niger South and Niger East Senatorial Districts). The second group had an 11-20 percent (medium) burden, comprising 18 LGAs from three zones (Niger South, Niger East, and Niger North). The final group included three LGAs from two zones (Niger East and Niger North), with an above 20 percent (high) burden. Finally, seven LGAs were selected from the three groups and, within each group, from each of the geopolitical zones. In cases where there was more than one LGA within each group/geopolitical zone subcategory, the LGA was randomly chosen. Details of the selected LGAs are provided in Table 1.
The selected LGAs were similar and different at the same time. While they had in common a mixture of primary and secondary health facilities and a perennial malaria transmission pattern, they were different in terms of spoken languages and ways of life. For example, among the medium-burden group, Bosso was inhabited by a Gbagyis-speaking tribe, Kontagora by a Hausas-speaking tribe, and Lapai by a Nupes-speaking tribe, representing the three major tribes in the state. No two LGAs in the study were completely alike, and it is argued that, together, the seven LGAs were representative of the state.
The sample was composed of all facilities with complete data (N = 150) in the seven LGAs. It included 99 facilities that received support from the program and consequently did not charge user fees for antimalarial services ("intervention facilities," henceforth) as well as 51 facilities that did not receive such support ("non-intervention facilities," henceforth). For each facility, 37 months of data (March 2010-March 2013) were collected, amounting to a panel of 5550 facility-month observations. The sample constituted 29 percent of facilities with complete monthly reports; only 523 out of 1585 government health facilities in Niger State reported data on a regular basis. Given the variation in terms of location, size, and operational level of facilities included, it is argued that the sample was reasonably representative of facilities at the state level.
2.3 | Empirical approach
The outcomes of interest include the number of febrile illness cases and the number of confirmed malaria cases, treated at a government health facility in a month. The first outcome captures the extent to which antimalarial services were accessible. It does not necessarily imply actual utilization, but rather addresses how the Niger population acted upon (as opposed to ignored) their suspicion of malaria. The second outcome indicates actual utilization of antimalarial services.
Consistent with existing studies, (13,21,22) the evaluation method for this study is difference-in-differences (D)[iD]. Based on mixed-effects negative binomial modeling, (10) the preferred specification is given by:
[mathematical expression not reproducible] (1)
Subscripts i, j, and t represent facility, LGA, and month, respectively. [Y.sub.jt] denotes the outcome of interest for facility i in LGA j at month t. [INT.sub.ij] is equal to 1 if facility i was an intervention facility, and 0 otherwise. [POST.sub.t] is equal to 1 in the months where the interventions were implemented, and 0 otherwise. [INT.sub.ij]. *[POST.sub.t] is the interaction term. [X.sub.ijt] represents a vector of facility characteristics. [LGA.sub.j]. denotes a vector of indicator variables representing each of the LGAs. TREND, is a linear time trend that starts from 1 at the beginning of the data-collection period and increases by 1 with each passing month until the end of the data-collection period (37). [MTH.sub.t] is a vector of indicator variables representing each of the 12 months in the calendar year. [PEAK.sub.t] is an indicator variable that takes the value of 1 for July-November entries and 0 otherwise. The definition is based on the 2013 report of the Niger State Ministry of Health, (18) which suggests that malaria transmission in the area has a seasonal peak in July-November. [U.sub.ij] represents facility-level random intercepts and [[epsilon].sub.ijt] the idiosyncratic error term. [beta], [gamma], and [alpha] are the parameters to be estimated. [[beta].sub.3] captures the DD (program) effect.
Different analyses can be performed. The data-collection period can be sectioned into the following:
* Period 1 (March-July 2010) where none of the facilities provided antimalarial services for free;
* Period 2 (August 2010-June 2012) where ACTs (MCM), LLINs, and IPTs were provided free-of-charge at intervention facilities; and
* Period 3 (July 2012-March 2013) where RDTs were added to the package of free antimalarial interventions at intervention facilities.
The panel of 37 months is sufficiently long to capture impacts of (a) the free provision of ACTs, LLINs, and IPTs, comparing Period 2 against Period 1, and (b) the free provision of the entire package of antimalarial interventions (inclusive of ACTs, LLINs, IPTs, and RDTs), comparing Period 3 against Period 1. Altogether, four DD estimates are derived in the analyses below, two for each of the outcome variables.
The preferred specification is deliberate. The inclusion of the first three variables in Equation 1 ([INT.sub.ij], [POST.sub.t], and [INT.sub.ij] *[POST.sub.t]) follows the standard DD approach. The other explanatory variables need further explanation. Vector X.., represents time-specific facility characteristics. It includes (a) whether facility i provided primary or secondary care, (b) the number of beds, (c) the number of skilled health personnel, (d) the ratio of female skilled health personnel, (e) the number of support staff, (f) distance from the facility to the farthest community served, and (g) distance from the facility to the nearest free-flowing water source. These variables are intentionally sourced for three reasons. First, they represent confounding factors that can be used to account for baseline differences between intervention and non-intervention facilities and for the non-random assignment of the intervention status. Second, as the unit of analysis is facility, it is important to view the outcome variables as outputs of a firm and to specify the regression to reflect the firm's production function. Here, the specification includes proxies of capital and labor inputs, as well as measures of technical capacity, representing factors of production. Finally, [X.sub.ijt] is needed to encapsulate the multidimensionality of access to health care. (4) In light of the fact that the intervention status (INT.) captures the affordability dimension of access, [X.sub.ijt], addresses the dimensions of social acceptability and physical accessibility through the following variables respectively: the ratio of female skilled health personnel and distance to the farthest community served.
The inclusion of the LGA fixed effects (LGA.) is consistent with existing studies. (23-25) They are used to capture unobserved, time-invariant area characteristics and used as the offset for incidence rates, accounting for the varying size of the target population across LGAs. (10,26)
The variables [TREND.sub.t], [MTH.sub.t], and [PEAK.sub.t], as well as the facility-level random intercepts ([u.sub.ij]), are intended to reflect the repeated measurement of the outcome variables across time. TRENDt accounts for non-stationarity of the outcome variables. MTHt and PEAK, address the seasonality of malaria transmission that is specific to Niger State. Finally, [u.sub.ij] tackles the problem of autocorrelation, allowing for temporal variations of the outcomes at the facility level. The inclusion of a linear trend (24,26-28) and month fixed effects, (23,26,29 ) and the use of mixed-effects modeling (10,26-28,30) are observed in other policy-evaluation studies.
Altogether, the DD specification in this study improves the validity and the precision of the estimated program effects. The addition of [X.sub.ijt], and LGA fixed effects minimizes estimation bias and offers a contribution over existing studies using facility-level data, which do not always account for confounding factors. (8,21) The incorporation of [TREND.sub.t], [MTH.sub.i], [PEAK.sub.t], and [u.sub.ij], ensures that appropriate statistical inferences can be made, addressing non-stationarity, seasonality, and autocorrelation that have been largely ignored in the literature. (16)
2.4 | Justification of the econometric model
The negative binomial model is carefully selected. It is compared against alternative count-data models, and proves to be most appropriate for this study. The negative binomial model has been shown to outperform log-linear OLS (7,16,31) and regular Poisson models in cases of over-dispersion; (29,32) Table 2 (explained below) shows that, for both outcomes of interest, the variances (SD-squared) are much larger than the means. It is also favored over zero-inflated models, following the fact that zero observations make up <5 percent of the sample. Finally, the negative binomial model is preferred to the overdispersed Poisson model. Deviance and AIC statistics of the two models, re-estimated as generalized linear models, were calculated, based on the preferred specification (without the hierarchical structure of the data). For the overdispersed Poisson model, standard errors were scaled based on the square root of Pearson chi-square-based dispersion. For the negative binomial model, standard errors were scaled based on the estimated alpha. The diagnostic statistics (unreported) were consistently lower for the negative binomial model. The exercise suggests that the negative binomial model provides a better fit.
2.5 | Justification of the evaluation method
This study employs the difference-in-differences (D)[iD] design, which is compared against two alternative evaluation methods. The first method is interrupted time series (ITS), consistent with studies that evaluate national policy changes using routine facility data. (29,31,33) In contrast with DD, the ITS design typically considers only the intervention group, using the pre-intervention period as the reference. The program effect is measured as a shift in the outcomes among the intervention group that is distinct from the pre-intervention trend. Nevertheless, as shown in Appendix SI, in this paper, the regression is run separately on both the intervention and the non-intervention groups to strengthen the ITS analysis, consistent with Zuckerman et al (33) and suggestions made by Lagarde. (16) The idea is to compare whether a change in the outcome variables can be more evidently observed among the intervention than the non-intervention group. The purpose of exploring the ITS design in addition to DD is to provide robustness checks. The DD design has indeed been considered the more rigorous approach, as it explicitly incorporates the nonintervention group, thereby having better degrees of freedom and more generalizability. (30)
The second method is comparative/controlled interrupted time series (CITS). The method has been proved to be valid and precise, (24,27,28,30) and is considered more flexible than DD, (25,27,30) rendering it superior under certain conditions. More specifically, while the DD method assumes that outcome trends are similar, that is, "common," between the intervention and the non-intervention groups, the CITS method relaxes the assumption and allows those trends to diverge. (27,30) The CITS desi gn is considered more appropriate if trends of the outcomes of interest statistically differ between the intervention and the non-intervention groups. The DD design is preferred otherwise.
Results from the ITS and the CITS methods are shown in Tables A.l and A.2 in Appendix SI, respectively. Incidence rate ratios (IRR) are reported based on mixed-effects negative binomial modeling. Table A.l reveals that the ITS results are qualitatively similar to the DD results discussed below, demonstrating robustness of the preferred model. Table A.2 shows that DD is preferred to CITS; the diverging trend hypothesis does not hold statistically, leading to overfitting under CITS. (30) An explanation why CITS performs less well here compared to other studies (24,27,28,30) is perhaps the fact that the number of pre-intervention data points in this study is too small (five data points: March-July 2010) to properly establish pre-intervention trends and observe trend divergence. (16,25,27)
2.6 | Justification of the methods to improve statistical inference
This study improves the statistical inference of the DD estimates, addressing seasonality, non-stationarity, and autocorrelation through the use of month fixed effects, a linear time trend, and mixed-effects modeling, respectively. While the method used to incorporate seasonality here is consistent with existing studies, (10,26,29) the literature differs on how to deal with non-stationarity and autocorrelation. To capture non-stationarity, time fixed effects have alternatively been used (21,23,25) and, in certain instances, they have been shown to outperform a linear trend in terms of model fit. (21,25) Also, to correct for the presence of first-order autocorrelation, clustered Huber-White standard errors have been employed, instead of mixed-effects modeling. (15,21,23,33) Since it is not clear how these methods should be compared in practice, (25) Table A.3 in Appendix SI offers results from negative binomial DD regressions, using time fixed effects and standard errors clustered at the LGA level. (15,21,23) The results reveal a pattern that is similar to the preferred estimates below, and show that using time fixed effects does not statistically lead to a higher log-likelihood value, compared to a linear trend. Consistent with the ITS and CITS exercises, the use of alternative methods to account for non-stationarity and autocorrelation reinforces the conclusions of this paper, illustrating their robustness. (25,27)
3 | RESULTS
3.1 | Descriptive statistics
Table 2 presents descriptive statistics of the outcome variables: febrile illness cases and confirmed malaria cases. The table is divided into two panels. Panel A shows the averages and standard deviations of the outcome variables in the intervention and the non-intervention groups in four periods: the entire data-collection period of 37 months, as well as the above-defined Periods 1, 2, and 3. Within each group, mean differences between (a) Period 2 and Period 1 and (b) Period 3 and Period 1 are provided and tested using t tests with an equal-variance assumption. Panel B presents differences of the mean differences between the intervention and the non-intervention facilities. They are equivalent to difference-indifferences estimates, with covariates unaccounted for. Statistical significance of these estimates is based on equal-variance t tests.
According to Table 2, the antimalarial interventions seemed to be effective in increasing the number of febrile illness cases and confirmed malaria cases. The introduction of free ACTs, LLINs, and IPTs was associated with an average increase of 29.739 and 27.354 cases of febrile illness and malaria per facility per month, respectively. The addition of free RDTs led to a larger increase of 51.264 and 39.307 cases respectively. Nevertheless, these estimates are only preliminary, as they do not account for facility and LGA characteristics.
Table 3 shows descriptive statistics of facility characteristics. The intervention and the non-intervention groups are compared, using t tests. On average, the proportion of secondary-care facilities was 6.8 percent lower among the intervention group. The numbers of beds, skilled health personnel, and support staff were also lower, while the ratio of female skilled health personnel and distance to the nearest source of water were larger among the intervention group. Altogether, the table suggests that the intervention facilities provided less advanced care, were smaller in size, were further away from a mosquito-breeding site, and had a larger percentage of female health personnel. These differences are significant at the 1-5 percent level.
3.2 | Difference-in-differences evaluation
Based on the preferred specification, the results are provided in Table 4. The table is split into two panels, with Panels A and B representing each of the outcome variables. Column I refers to the comparison between Periods 2 and 1, capturing the impacts of the first phase of the program. Column II refers to the comparison between Periods 3 and 1, showing the impacts of the second phase. The bottom part of the table explicitly discusses the empirical approach. The DD estimates in Table 4 are based on the interaction term in Equation 1 (INT;j.* POST,). The incidence rate ratios (IRR) (provided in the same fashion as previous studies (10,26) ) and the marginal effects calculated against the predicted number of events, conditional on the random intercepts, are shown. All regression equations pass the Wald test, and the likelihood ratio (LR) tests justify the random-intercepts model. Overall, Table 4 offers support for the interventions. The DD impacts are positive and statistically significant; with the removal of user charges, facilities were able to provide care for more patients with symptoms of malaria. Focusing on the marginal effects, the introduction of free ACTs, LLINs, and IPTs was associated with an increase, at the 1 percent level, of 20.876 and 22.835 cases of febrile illness and malaria per month, respectively. The expansion to include RDTs led to an increase, at the 5 percent level, of 19.007 and 19.681 cases of febrile illness and malaria per month, respectively.
4 | CONCLUSIONS AND DISCUSSION
This study evaluates the impact of free antimalarial interventions, namely ACTs, LLINs, and IPTs provided in August 2010-June 2012 (ie, the first phase of the program) and the addition of RDTs from July 2012 onwards (ie, the second phase) in Niger State, Nigeria. The outcomes of interest include the number of febrile illness cases and confirmed malaria cases. Based on 37 months of data from 99 intervention and 51 non-intervention facilities, this study uses the difference-in-differences (D)[iD] method estimated with a negative binomial model with facility-level random intercepts. The preferred specification accounts not only for facility and LGA characteristics to minimize bias, but also for non-stationarity, seasonality, and autocorrelation to improve statistical inferences. The results suggest that the interventions were effective in stimulating health care utilization among malaria-potential (febrile) and malaria-confirmed patients. Compared to the pre-intervention period, the first phase of the program was associated with an increase of 20.876 and 22.835 cases, and the subsequent addition of RDTs led to an increase of 19.007 and 19.681 cases of febrile illness and malaria per month, respectively.
The fact that the overall program led to increased utilization of antimalarial services is consistent with the demand theory. Considering that the probability of obtaining free services among febrile patients (who may or may not have had malaria) was higher and that the expected cost was lower at an intervention than a nonintervention facility, it is unsurprising that the number of febrile illness and confirmed malaria cases was higher among intervention facilities.
The fact that the impacts were less strong in the second phase requires further explanation. The demand theory suggests that the impacts should be stronger in the second phase. The addition of free RDTs should increase utilization of RDTs, through the own-price effect, and that of ACTs, LLINs, and IPTs, through the cross-price effect (given the complementarity of these commodities), leading to a greater number of febrile illness and confirmed malaria cases. Nevertheless, the demand theory makes the ceteris paribus assumption: that all things remain constant. The assumption does not hold in dynamic settings. Empirical evidence has indicated that, if implemented properly for a sufficiently long period, antimalarial interventions can interrupt malaria transmission, reducing the incidence rate and, consequently, the number of malaria-related facility visits over time. (8,11,13) The introduction of free RDTs, in particular, has been associated with a lower rate of clinical services utilization, with non-malaria cases being more effectively screened out. (10) The decline in the utilization rate at a later stage can also be attributed to a lower level of enthusiasm among potential patients, with social mobilization efforts dwindling down as the program progresses. (26) It is argued that the impacts observed in the second phase here were a likely result of the above counteracting effects combined.
There are limitations to the study. First, the study may still be subjected to estimation bias. There may have been observed and unobserved qualities of the selected facilities--unrelated to the interventions--that drove the results. Observed characteristics among the intervention and the non-intervention groups clearly differed, as demonstrated by Tables 2 and 3. Unobserved differences may have also existed. The literature suggests that there are time-varying determinants of antimalarial services utilization that are difficult to account for. Examples include quality of care provided at health facilities, (9) ' (13,26) as well as the level of protective immunity (11,12) and exposure to malaria vectors (11) among different populations in different areas, all of which may change after the introduction of a large-scale intervention program and may impact the program effects.
However, an attempt has been made to reduce estimation bias in this paper. In contrast with existing studies, (21,31) the preferred specification here includes explanatory variables that reflect the selection criteria and control for (observed) baseline differences between intervention and non-intervention facilities. The inclusion of facility characteristics and LGA fixed effects (Table 4) not only improves the fit, but also suppresses the estimates (Table 2), which may be taken as a sign that bias weakens. (14) It is worth noting also that an imperfect study design is common among evaluation studies; (9,14) it is often infeasible to undertake a study that randomly assigns interventions into some sites (thereby circumventing selection bias) while ignoring others that may also need them.
Also, the data used may be subjected to sample selection bias. As mentioned earlier, only 523 out of 1585 health facilities reported utilization data consistently. There may have been characteristics that systematically differentiated reporting from non-reporting facilities. Moreover, there was likely some reporting bias. Among facilities that reported utilization data, there may have been a non-monetary incentive to misreport or be selective in what was reported. These issues may potentially bias the estimates upward.
Nevertheless, it should be emphasized that most data are imperfect. This can be generalized to individual-level data, which are typically considered superior in terms of sample size and breadth of information contained therein. Specifically, individual-level data are susceptible to at least two types of bias: (a) recall bias, as the information is extracted from subjectively answered questions, (22) and (b) response bias, as the survey may not be well understood by the respondents. Albeit also imperfect, facility-level data can be used to complement individual-level data. An advantage of facility-level data is that they are collected through standard-format reports, making the information acquisition process more objective and perhaps less prone to measurement error.
Despite its limitations, the study has many contributions. First, this study adds to the literature of user-fee removal. It carefully encompasses several dimensions of health care access in the analyses, for example, the number of skilled health personnel and geographical distance between the facility and the farthest community served, so that the affordability dimension can be more effectively investigated. The results suggest that the removal of facility-levied fees can lead to increased utilization of antimalarial services, even after controlling for other dimensions of access to health care.
The study also contributes methodologically. As discussed in the Methods section and shown partly in Appendix SI, this study explores an array of evaluation methods (DD, ITS, and CITS) and regression models (overdispersed Poisson and negative binomial, with time trends vs time fixed effects, with random intercepts vs clustered standard errors). It illustrates that, despite quantitative differences, the results are consistent. In so doing, this study motivates future studies to cautiously select their methodology and double-check whether their conclusions are robust across methodological options. Compared to existing studies using facility data that typically have a smaller sample size, this study also collects a long panel of data (N = 5550), which allows for a more thorough investigation of temporal changes. Finally, this study provides results that are grounded on a robust empirical analysis, accounting for facility characteristics and area fixed effects to reduce estimation bias and addressing inference concerns associated with longitudinal data, namely non-stationarity, seasonality, and autocorrelation. The methodological contributions are deliberately undertaken to make up for the imperfect study design.
The quantification of more accurate estimates would require a more comprehensive dataset that includes also non-reporting facilities and a prospective evaluation design that allows for a cleaner, more randomized distinction between the intervention and control groups. This will need to be left with future studies.
Joint Acknowledgment/Disclosure Statement: This research was partially funded through Grants for Development of New Faculty Staff, Ratchadaphiseksomphot Endowment Fund, Chulalongkorn University. The funding source had no involvement in the research process nor the research outcomes. We would like to thank the Ministry of Health under the Niger State Government of Nigeria and the Niger State Malaria Elimination Program for providing and facilitating access to the data. We would also like to thank the anonymous reviewers for constructive comments that helped to immensely improve the paper.
Nopphol Witvorapong [iD] https://orcid.org/0000-0002-7926-381X
1. World Health Organization (WHO). World Malaria Report 2015. Geneva: World Health Organization; 2015.
2. Barofsky J, Anekwe TD, Chase C. Malaria eradication and economic outcomes in Sub-Saharan Africa: evidence from Uganda. J Health Econ. 2015;44:118-136.
3. Awoleye OJ, Thron C. Improving access to malaria rapid diagnostic test in Niger State, Nigeria: an assessment of implementation up to 2013. Malar Res Treat. 2016:2016:7436265.
4. Kizito J, Kayendeke M, Nabirye C, et al. Improving access to health care for malaria in Africa: a review of literature on what attracts patients. Malaria J. 2012;11:55.
5. Chuma J, Okungu V, Molyneux C. Barriers to prompt and effective malaria treatment among the poorest population in Kenya. Malaria J. 2010;9:144.
6. Ponsar F, Van Herp M, Zachariah R, et al. Abolishing user fees for children and pregnant women trebled uptake of malaria-related interventions in Kangaba, Mali. Health Policy Plana. 2011;26(2):ii72 -ii83.
7. Watson SI, Wroe EB, Dunbar EL, et al. The impact of user fees on health services utilization and infectious disease diagnoses in Neno District, Malawi: a longitudinal, quasi-experimental study. BMC Health Serv Res. 2016;16:595.
8. Aregawi M, Lynch M, Bekele W, et al. Time series analysis of trends in malaria cases and deaths at hospitals and the effect of antimalarial interventions, 2001-2011, Ethiopia. PLoS ONE. 2014;9(11):e106359.
9. Lagarde M, Palmer N. The impact of user fees on health service utilization in low-and middle-income countries: how strong is the evidence? 6 World Health Organ. 2008;86(ll):839-848.
10. Larsen DA, Bennett A, Silumbe K, et al. Population-wide malaria testing and treatment with rapid diagnostic tests and artemether-lumefantrine in Southern Zambia: a community randomized step-wedge control trial design. Am J Trop Med Hyg. 2015;92(5):913-921.
11. Trape JF, Tall A, Diagne N, et al. Malaria morbidity and pyrethroid resistance after the introduction of insecticide-treated bednets and artemisinin-based combination therapies: a longitudinal study. Lancet Infect Dis. 2011;11:925-932.
12. Tiffany A, Moundekeno FP, Traore A, et al. Encouraging impact following 2.5 years of reinforced malaria control interventions in a hy-perendemic region of the Republic of Guinea. Malaria J. 2016;15:298.
13. Munos M, Guiella G, Roberton T, et al. Independent evaluation of the rapid scale-up program to reduce under-five mortality in Burkina Faso. Am J Trop Med Hyg. 2016;94(3):584-595.
14. Ng M, Colson KE, Fullman N, et al. Assessing the contribution of malaria vector control and other maternal and child health interventions in reducing all-cause under-five mortality in Zambia, 1990-2010. Am J Trop Med Hyg. 2017;97(suppl 3):58-64.
15. Bertrand M, Duflo E, Mullainathan S. How much should we trust differences-in-differences estimates? Q J Econ. 2004;191(1):249-275.
16. Lagarde M. How to do (or not to do)... Assessing the impact of a policy change with routine longitudinal data. Health Policy Plann. 2012;27(1):76-83.
17. Federal Ministry of Health (FMOH), Nigeria. National Malaria Elimination Policy. Abuja: Federal Ministry of Health; 2013.
18. Ministry of Health (MOH), Niger State, Nigeria. Niger state annual operational plans for malaria. Niger State: S. M. C. Program, Minna, State Ministry of Health; 2013.
19. Department of Health Planning, Research and Statistics (DPRS), Nigeria. Annual report, 2010. Nigeria: Federal Ministry of Health; 2010.
20. State Malaria Elimination Programme (SMEP), Niger State, Nigeria. Annual report, 2013. Niger State: Niger State Ministry of Health; 2013.
21. Boudreaux C, Chanthala P, Lindelow M. Assessing the elimination of user fees for delivery services in Laos. PLoS ONE. 2014;9(3):e89784.
22. McKinnon B, Harper S, Kaufman JS, et al. Removing user fees for facility-based delivery services: a difference-in-differences evaluation from ten Sub-Saharan African countries. Health Policy Plann. 2015;30(4):432-441.
23. Bitler MP, Carpenter CS. Effects of state cervical cancer insurance mandates on Pap test rates. Health Serv Res. 2017;52(1):156-175.
24. St.Clair T, Cook TD, Hallberg K. Examining the internal validity and statistical precision of the comparative interrupted time series design by comparison with a randomized experiment. Am J Eval. 2014;35(3):311-327.
25. Wing C, Simon K, Bello-Gomez RA. Designing difference in difference studies: best practices for public health policy research. Anna Rev Pub/ Health. 2018;39:453-469.
26. Zombre D, De Allegri M, Ridde V. Immediate and sustained effects of user fee exemption on healthcare utilization among children under five in Burkina Faso: a controlled interrupted time-series analysis. Soc Sri Med. 2017;179:27-35.
27. Somers MA, Zhu P, Jacob R, et al. The Validity and Precision of the Comparative Interrupted time Series Design and the Difference-indifferences Design in Educational Evaluation. New York City, NY: MDRQ2013.
28. Jacob R, Somers MA, Zhu P, et al. The validity of the comparative interrupted time series design for evaluating the effect of school-level interventions. Eval Rev. 2016;40(3):167-198.
29. Bernal JL, Cummins S, Gasparrini A. Interrupted time series regression for the evaluation of public health interventions: a tutorial. Int J Epidemiol. 2017;46(1):348-355.
30. St.Clair T, Hallberg K, Cook TD. The validity and precision of the comparative interrupted time-series design: three within-study comparisons. J Educ Behav Stat. 2016;41(3):269-299.
31. Lagarde M, Barroy H, Palmer N. Assessing the effects of removing user fees in Zambia and Niger. J Health Serv Res Po. 2012;17(1):30-36.
32. O'hara RB, Kotze DJ. Do not log transform count data. Methods Ecol Evol. 2010;1(2):118-122.
33. Zuckerman RB, Sheingold SH, Orav EJ, et al. Readmissions, observation, and the hospital readmissions reduction program. New Engl J Med. 2016;374:1543-1551.
Additional supporting information may be found online in the Supporting Information section at the end of the article.
How to cite this article: Witvorapong N, Yakubu KY. Effectiveness of antimalarial interventions in Nigeria: Evidence from facility-level longitudinal data. Health Serv Res. 2019;54:669-677. https://doi.org/10.llll/1475-6773.13122
Nopphol Witvorapong PhD (1) [iD] | Kolo Yaro Yakubu MSc (2)
(1) Center for Health Economics, Faculty of Economics, Chulalongkorn University, Pathumwan, Bangkok, Thailand
(2) Strengthening Accountability and Quality Improvement for Maternal, Newborn and Child Health Project, Pact Nigeria, Gombe, Nigeria
Nopphol Witvorapong, PhD, Center for Health Economics, Faculty of Economics, Chulalongkorn University, Pathumwan, Bangkok, Thailand.
Ratchadaphiseksomphot Endowment Fund, Chulalongkorn University
TABLE 1 Selected local government areas in the sample Burden of febrile LGA illnesses Geopolitical zone Bida Low South Shiroro Low East Lapai Medium South Bosso Medium East Kontagora Medium North Agwara High North Paikoro High East N Facility types LGA Intervention Non-intervention Total Bida 11 0 11 Shiroro 15 13 28 Lapai 15 5 20 Bosso 15 3 18 Kontagora 15 10 25 Agwara 13 11 24 Paikoro 15 9 24 N 99 51 150 TABLE 2 Descriptive statistics of outcome variables and difference-in-differences (DD) estimates without covariates Number of monthly febrile illness cases Facility type/period Intervention facilities Non-intervention facilities Panel A: descriptive statistics of outcomes of interest across periods Entire sample period 116.362 69.860 (37 months) (126.267) (121.786) Period 1: January-July 66.162 50.616 2010 (Pre-intervention (84.308) (91.385) period) Period 2: August 116.426 71.142 2010-June 2012 (119.579) (127.840) (Post-intervention: Phase 1) Period 3: July 2012- 144.086 77.277 June 2013 (Post- (151.384) (119.824) intervention: Phase 2) Difference: period 2 vs 50.265 (***) 20.526 (***) Period 1 [5.658] [8.440] Difference: period 3 vs 77.925 (***) 26.661 (***) Period 1 [7.368] [8.632] Panel B: mean comparison between facility types Difference-in- 29.739 (***) differences: period 2 [1.746] vs Period 1 Difference-in- 51.264 (***) differences: period 3 [2.902] vs Period 1 Number of monthly malaria cases Facility type/period Intervention facilities Non-intervention facilities Panel A: descriptive statistics of outcomes of interest across periods Entire sample period 87.491 52.974 (37 months) (97.348) (90.479) Period 1: January-July 45.701 37.749 2010 (Pre-intervention (51.601) (68.702) period) Period 2: August 88.458 53.152 2010-June 2012 (92.612) (90.880) (Post-intervention: Phase 1) Period 3: July 2012- 108.237 60.978 June 2013 (Post- (119.064) (98.820) intervention: Phase 2) Difference: period 2 vs 42.757 (***) 15.403 (***) Period 1 [4.301] [6.035] Difference: period 3 vs 62.536 (***) 23.229 (***) Period 1 [5.624] [6.971] Panel B: mean comparison between facility types Difference-in- 27.354 (***) differences: period 2 [1.482] vs Period 1 Difference-in- 39.307 (***) differences: period 3 [2.391] vs Period 1 Notes: Standard deviations are in parentheses and standard errors in square brackets. Mean differences in Panel A are based on t tests, assuming equal variances. The DD estimates in Panel B are based on t tests, assuming equal variances, and do not account for any covariates. (*) 10% level of significance, (**) 5% level of significance, (***) 1% level of significance. TABLE 3 Descriptive statistics of facility-level characteristics Variables Nature of variables Primary care = 1 [Excluded Time-invariant category] Secondary care = 1 Time-invariant Number of beds Time-invariant Number of skilled health Time-varying personnel (monthly) Ratio of female skilled health Time-varying personnel (monthly) Number of support staff Time-varying (monthly) Distance to farthest community Time-invariant served in km Distance to the nearest source Time-invariant of water in km N Variables Intervention facilities Primary care = 1 [Excluded 0.990 category] (0.100) Secondary care = 1 0.010 (0.100) Number of beds 3.611 (6.172) Number of skilled health 5.372 personnel (7.646) Ratio of female skilled health 0.548 personnel (0.369) Number of support staff 3.201 (4.023) Distance to farthest community 7.545 served in km (6.612) Distance to the nearest source 4.091 of water in km (4.239) N 3663 Variables Non-intervention facilities Primary care = 1 [Excluded 0.922 category] (0.269) Secondary care = 1 0.078 (0.269) Number of beds 5.212 (11.869) Number of skilled health 5.873 personnel (13.482) Ratio of female skilled health 0.492 personnel (0.390) Number of support staff 6.541 (23.084) Distance to farthest community 7.497 served in km (4.082) Distance to the nearest source 3.764 of water in km (4.256) N 1887 Variables Mean difference Primary care = 1 [Excluded 0.068 (***) category] [0.005] Secondary care = 1 -0.068 (***) [0.005] Number of beds -1.601 (***) [0.242] Number of skilled health -0.501 (**) personnel [0.284] Ratio of female skilled health 0.057 (***) personnel [0.011] Number of support staff -3.340 (***) [0.392] Distance to farthest community 0.048 served in km [0.166] Distance to the nearest source 0.327 (**) of water in km [0.120] N Notes: Standard deviations are in parentheses and standard errors in square brackets. Mean differences are based on t tests, assuming equal variances. (*) 10% level of significance, (**) 5 % level of significance, (***) 1% level of significance. TABLE 4 Effects of antimalarial interventions: difference-in-differences (DD) estimates from mixed-effects negative binomial regressions First phase of intervention (Period 2 vs period 1) DD Estimates (1) Panel A: Outcome variable = Number of febrile illness cases IRR[95%CI] 1.222 (***) [1.079,1.383] Marginal effects 20.876 (***) [6.856] Log likelihood -21 397.153 LR test 1965.550 (***) Panel B: Outcome variable | Number of confirmed malaria cases IRR [95% CI] 1.265 (***) [1.102,1.451] Marginal effects 22.835 (***) [7.523] Log likelihood -20 131.286 LR test 2260.570 (***) N 4200 Bias reduction Facility characteristics and LGA fixed effects Seasonality Month fixed effects and seasonal peak indicator Non-stationarity Time trend Autocorrelation Facility-level random intercepts Second phase of intervention (Period 3 vs period 1) DD Estimates (II) Panel A: Outcome variable IRR[95%CI] 1.184 (**) [1.016,1.379] Marginal effects 19.007 (**) [8.926] Log likelihood -10 865.192 LR test 750.150 (***) Panel B: Outcome variable IRR [95% CI] 1.228 (**) [1.042,1.446] Marginal effects 19.681 (**) [8.417] Log likelihood -10 247.998 LR test 805.290 (***) N 2100 Bias reduction Seasonality Non-stationarity Autocorrelation Notes: The 95% confidence intervals of the IRRs are shown in square brackets in the same row. Standard errors of the marginal effects are in square brackets in the row below. Marginal effects are calculated against the predict counts (number of febrile illness and confirmed malaria cases), conditional on the random intercepts. LR test refers to the likelihood ratio test against a regular negative binomial model of the same specification without facility-level random intercepts. (*) 10% level of significance, (**) 5% level of significance, (***) 1% level of significance.
|Printer friendly Cite/link Email Feedback|
|Title Annotation:||RESEARCH ARTICLE|
|Author:||Witvorapong, Nopphol; Yakubu, Kolo Yaro|
|Publication:||Health Services Research|
|Date:||Jun 1, 2019|
|Previous Article:||Tailored interventions by community pharmacists and general practitioners improve adherence to statins in a Spanish randomized controlled trial.|
|Next Article:||An observational study of adolescent health outcomes associated with school-based health service utilization: A causal analysis.|