Why Some Well-Planned and Community-Based ICTD Interventions Fail.
Since World War II the world has invested $2.3 trillion in development projects to improve health, alleviate poverty, educate, and provide other services unavailable before that. Aside from development funds thousands of academics and practitioners have devoted their careers to building schools, improving agriculture, controlling disease, and educating children and adults in the developing world. While there has been progress, the number of absolute poor remains the same: 2 billion (Taylor, Taylor, & Taylor, 2012). In addition, the introduction of information and communication technologies (ICTs) into international development activities since the late 1990s added a new layer of challenges (Gomez & Pather, 2012). ICTs continue to be rapidly integrated into poverty alleviation and social development programs across the developing world in the sustained hope that computer literacy, Internet access, and mobile phone use will transform the lives and livelihoods of the world's disadvantaged. Billions of dollars are spent each year in the "development projects such as telecentres, village phone schemes, e-health and e-education projects, e-government kiosks, etc." (Heeks & Molla, 2009, p. 1) without significant improvements in the lives of the world's most disadvantaged populations.
Researchers lack a model to explain the failure of conventional development aid to produce expected results (Cohen & Easterly, 2010). One approach to improving outcomes has been inclusion of the affected community. Chambers (2013), a champion of community-based approaches, reviews the field and discusses several approaches in his classic work, Ideas for Development. Despite following a community-based approach, ICTDs (ICTs for development) interventions continue to fall short of expected outcomes. Concern about economic inequality continues (Piketty, 2015) as well as inquiries on how development projects interact with local histories, perceptions, and practices (Beck, 2017).
This article examines several examples of well-planned, community-based projects in a search for a common thread behind anomalous outcomes. The authors of this article noted anomalous outcomes in three diverse ICTD studies in which they personally participated. (1) A thread that combines these case studies is the anomalies that can be traced to a lack of partnership with the proper community members in some crucial aspect of the intervention. Even small rural communities can exhibit significant heterogeneity of function and opinion. Knowing which community members are the correct ones to involve is generally not a well-posed question and therefore does not possess a unique answer. In one of our case studies, the community leaders were consulted and promptly agreed to participate in an ICTD system development. After a team of outsiders designed and implemented a system, it became apparent that the ICTD system was not used because the consulted community was not the same community for which the system was intended. In another case, anomalies appeared in the evaluation phase. For example, responses to a questionnaire that was adopted from a verified study of experts was culturally inappropriate for the community it was applied to and it did not produce actionable results. In the third study the problem seemed to arise in the assessment phase. Predictions of qualitative analysis of interview data did not match the results of an evaluation of the effect of the intervention some years after the intervention was completed. The anomaly resulted because the intervention designed was based on the responses of community leaders who were not especially interested in the intervention, rather than involving those who proved to be avid system users. The difference from the first case study is that the second case study encompassed two different layers of the same community, not two different communities.
These case studies are from three countries: Nepal, Rwanda, and Peru. The Nepali project involved design, implementation, and evaluation of an ICTD system meant to provide a link between hospitals and schools in a provincial capital (Ilam) and a rural community (Namsaling), which was unconnected to the capital by either passable road or communication link. Researchers from an American university implemented the project after approval by community elders and members of an NGO involved in community water projects. The Rwandan project employed community health workers (CHWs) to collect health data that could be directly stored as electronic health records (EHRs). The project was conceived by foreign researchers in collaboration with Rwandan government officials. It was implemented by researchers from an American university. The CHWs acted only as data collectors because of the trust they enjoyed from the local community. In the Peruvian study Voice over Internet Protocol (VoIP) phones intended for adult education were installed in each of four schools. Project planning and design were done in consultation with the community in question; however, U.S. university students carried out the project implementation. Since community members were not principals in the implementation, there was no additional community input during the implementation stage. This approach resulted in an intervention that successfully solved a serious problem, but not the targeted problem. Authors of this article were active participants in each of the above studies. One author was an investigator in the Rwandan case study, and the other author was an investigator for both the Nepali and the Peruvian studies.
The next section presents related literature discussing community involvement and reasons for failures. Subsequently, the three problematic case studies are outlined, along with a discussion of their successes and issues. In the discussion section the authors offer some suggestions on how problems could have been ameliorated. The concluding section makes several recommendations that could conceivably help in a study design for the future ICTD interventions.
That community involvement is critical to the success of ICTD interventions is not a new idea. In his 2013 book, Ideas for Development, Chambers reviews the evolutionary shift from top-down approaches to bottom-up approaches in the light of case studies of development project implementations. Chambers (2013) offers an array of community-based approach acronyms accessible to the practitioners and presents their historical influx. In these community-based approaches, participatory monitoring and evaluation, in which local people monitor and evaluate programs, became compulsory (Estrella et al., 2000). A support for this approach also comes from Cohen and Easterly (2010), who give multiple examples of successful community-based projects (e.g., Svensson & Bjorkman, 2007).
In their book, Empowerment on an Unstable Planet: From Seeds of Human Energy to a Scale of Global Change, Taylor et al. (2012) explain the process of transformation to realistically grounded community-based change. The authors give numerous examples of "grassroots (or bottom-up) development," which they call SEED-SCALE. In addition, the ICTD literature is replete with empirical evidence that ICT interventions often fail because they are initiated from the outside without involvement by the relevant community (Heeks, 2002).
Geldof, Grimshaw, Kleine and Unwin (2011) provide a comprehensive review based on 53 ICTD studies. Among the key lessons learned from such studies the authors list strong partnerships and knowledge of local context. In his book, ICT4D: Information and Communication Technology for Development, Unwin (2009) has pointed out that two important aspects relevant to understanding local context are an understanding of local needs and user needs, best established by engaging users and communities in the process. In addition, Unwin (2009) also stresses that a response to local demand is essential for local capacity building.
Previously, studies presented successful community ICT projects whose achievements depended on the use of a collaborative methodology involving universities, governments, and communities (Marshall & Taylor, 2005). They argued that community involvement is necessary in all phases of the project; however, community participation may be far more complex than expected (Bailur, 2007). In this article a fine point regarding community participation is noted: Even a nearly imperceptible deviation from full inclusion of all relevant parties in all project aspects can result in large deviations from the expected outcomes.
It is not common that in their studies academics and practitioners describe failed initiatives along with a scrutiny of the possible reason(s) for failure of some aspect(s) of the project. However, Dodson, Sterling, and Bennett (2012) claim that "while there is no lack of documentation on failures, ICTD researchers and practitioners still do not appear to take full advantage of the fruits of our failures" (Dodson et al., 2012, p. 20). In that work the authors build on 40 research articles, written between 2003 and 2010, to identify commonalities among projects that failed to meet some or all of their development objectives and, in that process, they debate the effectiveness of ICTD initiatives (Dodson et al., 2012). The authors conclude that top-down, technology-centric, goal-diffuse approaches to ICTD contribute to deficient development results.
Ten years ago Heeks (2008), in his seminal work on ICTD 2.0, recognizes the main reasons for the failures of ICTD 1.0. He groups the reasons and provides three watchwords: sustainability, scalability, and evaluation. Heeks (2010) questions if ICT contributes to development by identifying the need to address motivational and structural issues around impact assessment and by asking policy makers to be more aware of ICTs' transformational potential.
There is a specific analysis of predicaments by Chib and Zhao (2009), who delve into financial and social sustainability in Chinese and Indian ICT interventions. More recently Aker, Ghosh, and Burrell (2016) consider the pitfalls of ICT for agriculture initiatives, while Gunawong and Gao (2017) investigate e-government failure in Thailand. However, the number of critical articles on ICTD interventions seems to be abating. The authors of this article would like to rekindle the debate around that topic.
The contribution of this article is to go deeper into the overlooked issues. We postulate that communitybased interventions carried out in conjunction with local partners who understand the local context are not necessarily enough to assure success. The appropriate community members must be involved in every step of project planning, implementation, and evaluation.
Three Case Studies with Anomalous Outcomes
Study 1: Nepali Wi-Fi Link from Ilam to Namsaling
Collaborators: Student chapter of Engineers without Borders (EWB) of the University of Colorado Boulder (UCB), Namsaling Community Development Centre (NCDC), Engineering for Developing Communities (EDC) Program (now the Mortenson Center) of UCB, Nepali Wireless Networking Project (NWNP)
Focus: To enable electronic communication among the schools and hospitals in the provincial capital of Ilam Province and in Namsaling, a moderately sized rural community on the east side of the Mai Khola River
Location: Ilam and Namsaling communities of Ilam Province in the Mechi Zone of Nepal
Community Participants: Local health workers, teachers, school administrators, elected town council members
Study Design: Designed by advisers of EWB and EDC in collaboration with NWNP
Method: An 802.11g link was constructed and tested to connect school and hospital
Anomalous Results: After the link was tested and in working order, no one used it
The ICTD project in Namsaling, Ilam District, Nepal had its origins in water projects that were joint efforts of the NCDC in Ilam, Nepal and the student chapter of EWB of the UCB.
Namsaling is a rural community that lies adjacent to the community of Ilam, the capital of the Ilam Province in eastern Nepal. Although the terrain is rugged, the population density in the 2011 census was 170/[km.sup.2], greater than the population density of New York State and only slightly less than the population density of Delaware. By comparison, the Peruvian Amazon has a population of roughly 1/[km.sup.2]. The Namsaling community was hard to reach by road in the dry season (requiring a Jeep) and unreachable in the wet season, requiring a 7km trek from Ilam. Although Namsaling possessed a health post as well as primary and secondary schools, materials were lacking at both. Highly trained personnel from other parts of Nepal were hard to hire in an area without roads or any form of communication. The high population density, however, had already attracted the cellphone companies for a pilot installation in 2008, in the period between the design and implementation phases of this project.
One author of the present work first was in Namsaling in February 2008 for a planning visit along with a group that included an EWB advisor and a volunteer emigre from Nepal with relatives in Kathmandu and detailed technical knowledge of Namsaling. The same author was a part of the group that later implemented the installation.
February 2008 was still a turbulent period in Nepal, a tail-end of a prolonged civil war. The war started in 1996 with a Maoist insurgency, resulting in their control of the country's rural areas (most of the country) and lasted for ten years. The palace massacre of 2001 in which members of the royal family almost annihilated each other contributed to an uneasy atmosphere in addition to the reign of a new and inexperienced king. Massive antiwar demonstrations started in 2006 and culminated in May 2008 when the Assembly declared Nepal a federal democratic republic and abolished the monarchy, removing the Shah dynasty from power.
The political instability made the start of the project difficult for numerous reasons, including transportation problems (strikes, roadblocks, demonstrations). However, the project team managed to reach Namsaling. Prior to arrival in Namsaling the group spent one day in Kathmandu, one day in Birtamod (Terai region), a village at the cut-off to Ilam, and two days in Ilam with members of the NCDC discussing the upcoming visit to Namsaling. These meetings were important as the political situation was changing by the hour during this period. The meetings with NCDC continued for two more days in Birtamod following the visit to Namsaling. During our 36 hours in Namsaling, group members interviewed the Namsaling town council members and documented the facilities in the health center, government buildings, and schools.
The technical solution to the ICTD project was straightforward. After project approval had been obtained from the Namsaling town council in February 2008, an NWNP technician and a UCB engineering faculty member designed an IEEE 802.11g network. Arrangements were made with NWNP to deliver the equipment to Namsaling before July 2008. After the equipment was delivered in July 2008, the NWNP technician met the volunteer students from UCB for the build-out (implementation) portion of the project.
During the appraisal visit in February 2008 a pilot cellphone system had been tested in Namsaling. The pilot was successful, and a fledgling cellphone system was operational in Namsaling in July 2008 when the Wi-Fi network build-out took place. The existing pilot was a rudimentary 2G network. The network was analog and could not transmit data. This analog cellphone network lacked the same functionality as the Wi-Fi Internet link. However, the fact that there was some form of communication that had not been in place previously may have changed the perception of data needs within the community.
The peaceful election in April 2008 led to the formation of a government, and travel became easier. A major segment of the network plan detailed how another group would provide content for the Wi-Fi link. This group was supposed to install and test content on the link servers as soon as the link was secured. This part of the project never materialized because the affected community did not provide input regarding their interests and needs for content.
That the ICTD system was built to the required specification in the allotted time and with all the required permissions represented the success of the project. The major issue with the study was the lack of content to put on the system. During the study execution, the stakeholders were not consulted about the content they might want. Therefore, no content was ever installed on the system. Although the Namsaling council approved the installation, the community leaders did not have a clear idea what the installation could mean for the community and what content could be offered to the community.
Study 2: Carnegie Mellon University Development of Rwandan Electronic Health Records System
Collaborators: Carnegie Mellon University Rwanda (currently called CMU-Africa), Ricoh Corporation, Rwandan Ministry of Health (MoH), Kibagabaga hospital in Rwanda's Gasabo district
Focus: A pilot project testing the use of smartphones to track children's health and development and to store the data in electronic health records (EHRs). A prime motivation was to reduce persistent child malnutrition in Rwanda
Location: Remera, an urban community in Kigali, and Bumbogo, a rural community near Kigali
Community Participants: Community health workers (CHWs) and parents of children under age five
Study Design: Carried out by CMU-Africa researchers with input from Rwandan MoH
Method: CHWs recorded longitudinal measurements of children's weight for nine months and surveyed parents using an internationally verified survey on children's diet diversity
Anomalous Results: Survey data from parents regarding diversity of children's diet
The Rwandan case study was performed in 2014 and was designed to test the feasibility of EHRs and their applicability to the Rwandan health sector. In this case study, CHWs monitor child growth and development using technical tools to establish the EHR. Research by the World Bank (Bundervoet, 2013) found that in Rwanda, which has been battling child malnutrition for decades, 44% of children under five years of age suffer from stunting. (2) An ancillary goal of the study was to collect data that would inform researchers on the causes of stunting in Rwanda and to help ameliorate the problem.
The case study was performed in cooperation with the Rwandan MoH and Kibagabaga hospital in Gasabo district. The liaison with the hospital was a coordinator of CHWs, and 24 participants were randomly selected from CHWs in her jurisdiction. CHWs that participated in the study were typical in education, age, and sex of CHWs in Gasabo district, one of the most populated districts in Rwanda, with approximately a half-million people. (3)
Each CHW was given a smartphone with a mobile application to monitor children's growth and development. The evaluation was carried out in both an urban and a rural location, with 12 CHWs in each. The mobile application was designed to be tolerant of delays in the mobile network and optimized for low-resource settings.
In addition to collecting children's growth indicators, CHWs performed a nutritional survey by asking parents of young children about the food their children typically consume. The results from the nutrition survey were intended to give a clue as to why stunting remains a significant issue in Rwanda. This nutritional survey was adapted from an earlier study (Arimond & Ruel, 2004), with questions developed by accredited nutritionists and verified in other countries. This nutritional survey is meant to test the diversity of diet, as it is known that a diverse diet reduces malnutrition (Brinkman, de Pee, Sanogo, Subran, & Bloem, 2010; Thorne-Lyman et al., 2009). It was tacitly assumed that this survey would be applicable to the Rwandan context.
The study was carried out from March to December of 2014. Each group of CHWs was separately trained in smartphone and mobile application use, and the training sessions were held in community health locations in Gasabo district.
CHWs collected two data sets for this project. Typically, a CHW visits a family once a month and measures each child's weight to track his or her growth and development. For purposes of this project, CHWs recorded this information, along with the age of the child and parent's information, using a smartphone application, which synchronized with the database implemented for this study.
Overall, the mobile application for collecting health indicators was well received by the CHWs. We list here the successes from nine months of working with the 24 CHWs:
* The CHWs were comfortable with the part of the application that replicates their previous work in collecting weight on children under five.
* Many CHWs had never used a smartphone, but learned the needed skills in two training sessions. The phone settings were in English (a language they were not fluent in) and they adjusted to that.
* Data tracking a child's weight and age electronically was considerably more accurate, consistent, and accessible than handwritten records. In addition, with electronic data collection, the researchers were able to improve the correlation of children's weight with age over paper records by 50% and reduce measurement error by 40%.
The second type of data from the nutritional survey was incorporated into the mobile application. At the beginning of the project, CHWs asked parents questions about the types of food they feed their children and the frequency. The survey was organized so that the parents had to answer yes/no, select answers to multiplechoice questions, or use a drop-down menu. Food was divided into seven food groups important for children's growth and development. After the survey, each child received a nutrition score ranging from 0-7, depending on how many food groups (4) they consumed.
The CHWs had many problems during the administration of the nutrition portion of the survey. The parents' answers were unrealistic; for example, many parents answered several food survey questions by checking boxes that indicated their four-month-old baby was consuming multiple food groups such as meat and cheese, along with nursing, and the CHWs did not know how to deal with this issue. After several meetings with the CHWs where those problems were discussed, the CHWs were able to collect more reasonable data by explaining to the parents that their answers did not make sense and by spending more time articulating the meaning of the food groups. However, survey reliability continues to be questionable because a discussion of food is culturally shunned in Rwanda.
The nutritional survey for this study was adopted from an earlier study (Arimond & Ruel, 2004), which used the same dietary groups to evaluate the nutritional status of children under age five in 11 developing countries, among them Rwanda. In the case study carried out in 2014, CHWs recorded that the food diversity of Rwandan children under age five was 4.1 food groups on average, while a previous study from Arimond and Ruel (2004) reports an average of 2.9. That discrepancy was one of the reasons researchers questioned the validity of nutritional data collected by the CHWs; the 2.9 diversity value is consistent with the current level of malnutrition in Rwanda, while a value of 4.1 is not.
There were two problems with the data collection in the Rwandan case study from 2104:
1. The researchers were unaware of local cultural norms, for example, discussing food is a cultural taboo in Rwanda. Neither parents nor CHWs were involved in the survey design. In addition, the food groups in the standard nutritional survey did not align with Rwandan terminology.
2. The CHWs did not understand the purpose of the nutritional survey, mainly because they had not been involved in the design and development of the survey questions. They could not respond to queries of the confused parents who needed more guidance regarding the seven food groups because the CHWs themselves did not understand the food groups.
Study 3: Distance Education on the Peruvian NapoNet
Collaborators: University of Colorado at Boulder (UCB), Grupo de Telecomunicaciones Rurales (GTR) of the Catholic University of Peru (PUCP), Centro de Salud de Santa Clotilde (CSSC)
Focus: A pilot study of continuing education by distance learning so that adults in communities without high schools could earn high school degrees. The four communities involved in this study were a subset of 15 communities linked by an earlier high-speed Wi-Fi network that supplied each of the community health posts with high-speed Internet and VoIP. The goal of the continuing education project was to distribute more equitably the copious underused bandwidth
Location: Four communities along the Napo River of Amazon, Peru between Iquitos and Cabo Pantoja, on the border with Ecuador
Participants: Principals and teachers of primary and secondary schools and adult learners in the above locations. The participating schools were a high school in Santa Clotilde and three primary schools in Copal Urco, San Rafael, and Rumi Tumi
Study Design: Authorized by the CSSC, the entity in charge of network traffic at the time. Authorization was based on the results of extensive interviews carried out with numerous stakeholders of the network, health posts, schools, and villages. Hardware designed in a collaboration of GTR and the UCB researchers
Method: Placing VoIP phones in one central location in each of the four schools to be used for adult education. Teachers and principals were trained in how to use and maintain the phones
Anomalous Results: The program did not function as planned based on the original interviews with stakeholders. Once the system was operational, teachers and principals used the phones for what they perceived as the more important task of communicating with each other and the school ministry rather than for the intended distance education. School officials had been interviewed during the assessment and were overwhelmingly in favor of an educational intervention. They were not involved in the design details of the distance education system, although they approved of the system before and during installation.
The Global Fund initiated a multimillion-dollar program in 2002 to combat malaria, and an outgrowth of that program was a 455km Wi-Fi-based long-distance (WiLD) network that linked the 15 health posts along the Napo River. The network could transmit bandwidths of roughly 10 Mbps over the network. The Global Fund stipulated that the network would be used for health first. However, transmission of health data between the health posts required only infrequent short bursts of kbps of bandwidth. The VoIP for calls to and from the hospital required approximately 32 kbps during the time of the calls, on average, an hour a day. The rest of the time the health workers--to the exclusion of all others in the villages-- had access to the Internet. This arrangement led to resentment. A group of government medical workers, medical faculty from the Universidad Nacional de la Amazonia Peruana (UNAP), and some government officials suggested to researchers from UCB that they should attempt a more equitable sharing of the network bandwidth. The researchers approached CSSC personnel, who at the time administered the network, and the result was the NapoNet project.
NapoNet was a joint effort of UCB, PUCP, UNAP, CSSC, the public health service of Peru, and the local governments in which the health posts resided. The project's first step was to carry out an extensive set of interviews (with International Board Review approvals from UCB as well as local approvals) with stakeholders along the Napo River. Interviews carried out in 2009 included teachers and students in Santa Clotilde schools, town meetings with eight Napo villages, meetings with indigenous religious leaders, mayors, and health personnel. Follow-up interviews were carried out in 2012. The goal of the interviews was to determine what services were most desirable to the communities. The idea was to implement new services until the bandwidth of the network was fully used in a community-approved manner.
UCB and GTR, in consultation with CSSC, designed the pilot after an analysis of the 2009 interview data. The pilot involved construction of a subnetwork of NapoNet intended for distance education. Santa Clotilde, the largest secondary school in the central Napo region, had already set up a distance program with several the villages. The initial pilot was to first link two of the smaller villages with Santa Clotilde, and then to include others. The interviews were completed in July 2010.
Construction of the subnetwork was initiated in the winter of 2009-2010. Before the subnetwork was erected NapoNet consisted of a series of towers, each high above the 35-meter jungle canopy to allow for line-of-sight connections of the transceiving antennas. The drop points in each village then consisted of a directional antenna pointed from the top of the tower to a directional antenna on the health post. A single battery fed by two solar panels powered the transceivers and antennas in the towers.
The expected outcomes of the initial pilot were fourfold:
1. The educational subnet would augment the existing traffic on the backbone.
2. More people would work on their secondary education, and the time required to graduate from secondary school would shorten.
3. There would be greater acceptance of the Napo Network as a community asset.
4. The Napo Network would provide motivation to continue the build-out of the educational network. A motivation for the distance-learning network was to speed up distribution of learning materials as well as to serve as a means for the instructors to answer questions from distance students. The main base station of the educational network in Santa Clotilde and the first two villages was outfitted in the winter of 2009-2010. A second build-out in the winter of 2010-2011 installed the monitoring network used to log all calls (no caller information other than the village of origin and reception) to monitor network activity. The network activity was logged locally but transmitted daily over the Internet connection so that records were kept for subsequent analysis at one of the participating universities.
There were three unexpected outcomes:
1. There was no growth in network traffic or in the subnet traffic that involved the three villages.
2. There were no graduations from the distance education program during the pilot period.
3. There was no awareness among the population that the pilot was underway. Based on these observations, researchers saw no reason to continue the project until there was a better understanding of the results.
To uncover more information about the unexpected results, extensive interviews were carried out (with an amended IRB approval) in all the affected communities in the summer of 2012.
The results were as surprising as the monitoring results:
1. The subnet was used primarily by the instructors and school principals to speak with each other.
2. The system had not been used for distance education at all; students had not been allowed to use the system (VoIP telephone) in any documented case.
3. The distance education enrollment had dwindled during the period, which was unrelated to the existence of the subnet as no one was aware of it.
4. The identified unintended uses were greatly beneficial to the educational system of the Napo.
Lessons from failures are a common theme in many research fields, for example, the impact of social capital in development projects and failures in rural development (Michelini, 2013). In gender studies, researchers present examples from agricultural development and analyze failure to improve outcomes for women (Johnson, Kovarik, Meinzen-Dick, Njuki, & Quisumbing, 2016). In human-computer interaction (HCI) investigators consider social inequality (Ekbia & Nardi, 2016) to broaden the perception of the impact of their field on the developing world. In a comparable attempt, the present analysis seeks to advance the understanding of the effectiveness of ICTD initiatives by exploring the reasons for failure, an approach that has become more prevalent in the ICTD field (Hussain & Brown, 2018; Ismail, Heeks, Nicholson, & Aman, 2018).
Common to each case presented in this article is that the participants who executed the program were not involved in the program design. However, not all was a failure because each study was successful in executing most of its goals. The Nepali study followed the basic tenet of community-based studies: that it is important to consult community leaders before taking action. However, the fundamental problem with that consultation was that if you ask people if they want something for free, they will usually accept it, although they may not genuinely understand what they are being offered. In the Nepali study, there was no community buy-in for the project. A compounding factor was the political turmoil at the time that affected every aspect of Nepali life. The coming elections would determine the form of government and were transformational in rural as well as in urban areas. Whether the Maoist forces that had occupied much of the countryside during the nighttime hours for years were to prevail over the conventional democratic parties was a question that dominated politics from the local to the national levels. The type of information to be transmitted over a community link depended on the form of the government and, hence, community organization in the near future, which in this case was uncertain.
Equally important is to be aware that while working with tribal communities (i.e., hierarchically organized), consultation with community leaders regarding the community's need may not always reveal the true interests of the majority of the population. This possibility poses the question of how best to weigh the input from the various stakeholders in a meaningful and respectful way.
In addition, this study suffered from a scoping issue. The partners on the project planned to build the community link without considering which content should be transmitted on that link. This type of narrow scoping is common in ICTD projects (Diaz Andrade & Urquhart, 2012; Prakash & De', 2007), particularly those with well-defined timeframes, budgets, and research goals. Narrow-scoped developmental projects have a low probability of making an impact and as such could be a waste of resources.
The Rwandan study was successful in demonstrating that using modern electronic tools for health data collection allows better tracking of health indicators (Brown & McSharry, 2016). However, the nutritional survey of children's parents produced anomalous data as the researchers were unaware of the local cultural norm that discussing food was a taboo in Rwanda. In addition, food groups from a standard nutritional survey did not fit Rwandan dietary practice. Rwandan parents did not understand the concept of food groups such as carbohydrate, protein, dairy, etc. The main lesson from this project's shortcomings was that local nutrition experts, along with the CHWs who implemented the study, should have been involved in the planning, appraisal, and evaluation phase.
In the Peruvian study, the initial fully approved assessment of the villages employed extensive interviews with individuals, stakeholders, and groups. However, the decision to choose distance education students as the primary user group was made by the villagers involved in the health sector who controlled network access at the behest of university researchers from UCB and GTR and not by those villagers in charge of the villages' educational resources. Those involved in the schools actively supported directing the resources toward education, but not specifically to distance education. When the resources for distance education became available, school personnel used the resources for what they perceived as the more important task of communicating with each other, with their superiors, and with the Ministry of Education, primarily on matters of curriculum and curricular implementation, and not for the intended distance education.
Unlike the finding from Dodson et al. (2012), none of the three studied were top-down, technology-centric approaches. Instead, they involved the community at some level through questions; however, we go even further and hypothesize that community participation in all aspects of research design is necessary for the collection of high-quality field data. In each case study reviewed, an inclusion of the implementation personnel at an earlier stage of the project would have alleviated if not eliminated the anomalies. In Rwanda, parents and CHWs would have noted that the questions were inappropriate for the Rwandan setting. Had the educational personnel of the Amazon been involved in analyzing the data from the villages, they may have pointed out a more pressing need for better long-distance communication to share curriculum and its implementation.
Prior to the Nepali study, discussions should have been held with the community regarding the community's communication needs. The roles for this community could have been better defined if local health, education, and government practitioners and officials had come forward to lead efforts to improve communication between Namsaling and Ilam. In that case it could have been determined if a WiLD link was appropriate or if communication needs could simply have been covered over the existing channels available at that time such as the newly installed cell network. If WiLD was to be the solution, then the initial milestone should have been defined; for example, a consultation between a doctor in the Ilam hospital and a CHW treating a patient in Namsaling.
The Rwandan study involved training CHWs to collect health data using smartphones. One way to have grounded the study in the community would have been to enlist participating CHWs as equal partners from the inception as well as to include a cross-section of local children's parents in the questionnaire design. Group and individual meetings with the health workers and the parents could have supplied invaluable information that could have resulted in a better survey design. Community discussion (at the appraisal stage) before survey development would have influenced the questions. These added steps would have increased the time necessary to implement the study, but could have increased the relevance of the results greatly.
The Peruvian study employed a participatory approach for appraisal, but moved further away from that approach as the process continued. The design was based on data taken from town meetings and interviews, and the implementation form was discussed with stakeholders during the design stage. However, the stakeholders and the community were not the same entity. One community member was employed in the implementation, although only to help with the subnet construction, but was not taught about the system capabilities or the basic technology. This community member was simply given instructions while under the supervision of a crew of UCB students. The design phase should have included the same cross-section of community members who were involved in the town meetings and interviews. The number of people involved may have been unwieldy, but this step was necessary. Interested community members should have been trained and then paid to carry out the network implementation. At each point the community members, paid or volunteers, should have been equal partners in the decision-making process. This might not necessarily have lengthened the timespan of the study although it would have required longer periods in the Amazon for the researchers.
In summary, all three studies were only in their "first loop of learning," or linear learning stage as defined in Mulgan (2017). This first loop of learning is data collection within the existing framework. Neither study went into the second loop of learning, which is creating new categories and models; nor the third loop of learning, which is rethinking how to think (Mulgan, 2017). Linear learning is adequate for stable environments (Mulgan, 2017), and none of the above studies took place in a stable environment. However, one approach to achieving a fundamental social change would be going into higher loops of learning, which are often not possible within short-term or pilot projects lasting a couple years. The obstacle to accessing higher loops of learning is that they require more time and more stakeholder involvement, implying that funding must be longer term and project sustainable.
Executives and managers who will carry out development activities need to consider accountability, which requires data coming from monitoring and evaluation. Organizations that carry out development interventions, however, are seldom if ever a part of the community in which the intervention is taking place. Management of one organization tends to deal with the management of another organization or at least with stakeholders who are commensurate with the management. Even if the stakeholders from management implement an intervention, it is unlikely they will monitor it competently.
We have reviewed three case studies in this article in which the people intended to monitor did not monitor. In the case of health care in Rwanda, questions developed to monitor the interventions did not involve the people supplying the data. The survey questions were either irrelevant or culturally insensitive and failed to elicit realistic answers. The problem is deeper than simply enlisting experts on the local culture. The best experts on the population under study are members of the population under study. The members of this population must be fully aware that the study is taking place (a failing of the Peru study) and be encouraged to participate from the beginning. By constructing implementations that are fully understood and accepted by a community, measures of progress can be constructed in collaboration between the experts and the community concerned. Using an entire community in a study is a tedious approach, but it might be a necessity for studies that engage communities. If development efforts are to bring a lasting change to the community and reach higher loops of learning, such an approach might be essential.
Suzana Brown, Assistant Professor, Department of Technology and Society at SUNY Korea, Incheon, South Korea. email@example.com
Alan Mickelson, Associate Professor of Electrical Engineering, Electrical, Computer, and Energy Engineering, University of Colorado Boulder, CO, USA. firstname.lastname@example.org
Aker, J. C., Ghosh, I., & Burrell, J. (2016). The promise (and pitfalls) of ICT for agriculture initiatives. Agricultural Economics, 47(S1), 35-48.
Arimond, M., & Ruel, M. T (2004). Dietary diversity is associated with child nutritional status: Evidence from 11 demographic and health surveys. The Journal of Nutrition, 134(10), 2579-2585.
Bailur, S. (2007, 28-30 May). The complexities of community participation in ICT for development projects: The case of "Our Voices." Proceedings of 9th International Conference on Social Implications of Computers in Developing Countries (pp. 1-17).
Beck, E. (2017). How development projects persist: Everyday negotiations with Guatemalan NGOs. Durham, NC: Duke University Press.
Brinkman, H. J., de Pee, S., Sanogo, I., Subran, L., & Bloem, M. W. (2009). High food prices and the global financial crisis have reduced access to nutritious food and worsened nutritional status and health. The Journal of Nutrition, 140(1), 153S-161S.
Brown, S., & McSharry, P. (2016). Improving accuracy and usability of growth charts: Case study in Rwanda. BMJ Open, 6(1). e009046. 10.1136/bmjopen-2015-009046
Bundervoet, T. (2013) "Malnutrition chapter" by World Bank, Unpublished document, based on The 2010 DHS and the 2012 Comprehensive Food Security and Vulnerability Analysis and Nutrition Survey (CFSVA).
Chambers, R. (2013). Ideas for development. Abingdon-on-Thames, UK: Routledge.
Chib, A., & Zhao, J. (2009). Sustainability of ICT interventions: Lessons from rural projects in China and India. L.M. Harter, M. Dutta, & C. Cole (Eds.), Communicating for social impact: Engaging communication theory, research, and pedagogy (pp. 145-159). Cresskill, NJ: Hampton Press.
Cohen, J., & Easterly, W. (Eds.). (2010). What works in development?: Thinking big and thinking small. Washington, DC: Brookings Institution Press.
Diaz Andrade, A., & Urquhart, C. (2012). Unveiling the modernity bias: A critical examination of the politics of ICT4D. Information Technology for Development, 18(4), 281-292.
Dodson, L., Sterling, S., & Bennett, J. K. (2012, 12-15 March). Considering failure: Eight years of ITID research. Proceedings of the Fifth International Conference on Information and Communication Technologies and Development (pp. 56-64). Atlanta, GA: ACM.
Ekbia, H., & Nardi, B. (2016, 7-12 May). Social inequality and HCI: The view from political economy. Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (pp. 4997-5002). San Jose, CA: ACM.
Estrella, M., Blauert, J., Gonsalves, J., Campilan, D., Gaventa, J., Guijt, I., ... & Ricafort, R. (Eds.). (2000). Learning from change: Issues and experiences in participatory monitoring and evaluation. Ottawa, ON: International Development Research Centre.
Geldof, M., Grimshaw, D. J., Kleine, D., & Unwin, T. (2011). What are the key lessons of ICT4D partnerships for poverty reduction? Systematic Review Report, 1-89. Retrieved from https://www.gov.uk/dfidresearch-outputs/ what-are-the-key-lessons-of-ict4d-partnerships-for-poverty-reduction-systematic- reviewreport
Gomez, R., & Pather, S. (2012). ICT evaluation: Are we asking the right questions? The Electronic Journal of Information Systems in Developing Countries, 50(1), 1-14.
Gunawong, P, & Gao, P (2017). Understanding e-government failure in the developing country context:
A process-oriented study. Information Technology for Development, 23(1), 153- 178.
Heeks, R. (2002). Information systems and developing countries: Failure, success, and local improvisations. The Information Society 18(2), 101-112.
Heeks, R. (2008). ICT4D 2.0: The next phase of applying ICT for international development. Computer, 41(6), 26-33.
Heeks, R. (2010). Do information and communication technologies (ICTs) contribute to development? Journal of International Development, 22(5), 625-640.
Heeks, R., & Molla, A. (2009). Impact assessment of ICT-for-development projects: A compendium of approaches (Development Informatics Working Paper No. 36). Manchester, UK: IDPM.
Hussain, F., & Brown, S. (2018). Developing a comparative framework of ICT4D initiatives in the Global South: A critical realist approach. The Electronic Journal of Information Systems in Developing Countries, 84(6), e12055.
Ismail, S. A., Heeks, R., Nicholson, B., & Aman, A. (2018). Analyzing conflict and its management within ICT4D partnerships: An institutional logics perspective. Information Technology for Development, 24(1), 165-187.
Johnson, N. L., Kovarik, C., Meinzen-Dick, R., Njuki, J., & Quisumbing, A. (2016). Gender, assets, and agricultural development: Lessons from eight projects. World Development, 83, 295-311.
Marshall, S., & Taylor, W. (2005). Facilitating the use of ICT for community development through collaborative partnerships between universities, governments and communities. International Journal of Education and Development Using ICT, 1(1), 5-12.
Michelini, J. J. (2013). Small farmers and social capital in development projects: Lessons from failures in Argentina's rural periphery. Journal of Rural Studies, 30, 99-109.
Mulgan, G. (2017). Big mind: How collective intelligence can change our world. Princeton, NJ: Princeton University Press.
Piketty, T. (2015). About capital in the twenty-first century. American Economic Review, 105(5), 48-53.
Prakash, A., & De', R. (2007). Importance of development context in ICT4D projects: A study of computerization of land records in India. Information Technology & People, 20(3), 262-281.
Svensson, J., & Bjorkman, M. (2007). Power to the people: Evidence from a randomized held experiment of a community-based monitoring project in Uganda. World Bank Group. Retrieved from https://doi.org/10.1596/1813-9450-4268
Taylor, D. C., Taylor, C. E., & Taylor, J. O. (2012). Empowerment on an unstable planet: From seeds of human energy to a scale of global change. Oxford, UK: Oxford University Press.
Thorne-Lyman, A. L., Valpiani, N., Sun, K., Semba, R. D., Klotz, C. L., Kraemer, K.,... & Bloem, M. W. (2009). Household dietary diversity and food expenditures are closely linked in rural Bangladesh, increasing the risk of malnutrition due to the financial crisis. The Journal of Nutrition, 140(1), 182S-188S.
Unwin, T. (2009). The technologies: Identifying appropriate solutions for development needs. In Unwin, P.T.H., & Unwin, T. (Eds.), ICT4D: Information and communication technology for development (pp. 76119). Cambridge, UK: Cambridge University Press.
State University of New York at Korea (SUNY Korea), South Korea
University of Colorado Boulder, USA
(1.) The analyzed studies were not chosen in a random and representative manner. The main justification is that it would otherwise be impossible to access problematic data unless directly involved with the project as such findings are seldom, if ever, published.
(2.) Stunting is defined as height for age below two standard deviations of the WHO Child Growth Standards median. It often reflects the cumulative effects of undernutrition or chronic malnutrition.
(3.) http://statistics.gov.rw/publications/2012-population-and-housing-census- provisional-results
(4.) The seven food groups are protein, carbohydrates, vitamin A-rich food, vegetables, fruit, oil and fat, and milk and milk products.
|Printer friendly Cite/link Email Feedback|
|Title Annotation:||Research Article|
|Author:||Brown, Suzana; Mickelson, Alan|
|Publication:||Information Technologies & International Development|
|Date:||Jan 1, 2019|
|Previous Article:||What Are the Drivers of ICT Diffusion? Evidence from Latin American Firms.|
|Next Article:||Empowering Rural Youth for Socioeconomic Development: A Skill Development Approach in Sarawak, Malaysia.|