Improving aid effectiveness: two studies suggest solutions.

A year after unrelenting natural disasters, aid fatigue is being reported. The unprecedented giving during the tsunami and hurricane "Katrina" disasters was accompanied by calls for greater evidence of aid effectiveness. Yet, this is an area about which little is known and written. How do you measure effectiveness in a reactive situation? How do you create benchmarks and metrics, when every disaster occurs in an entirely different context?--when the terrain, resources, access and laws are entirely different?

[ILLUSTRATION OMITTED]

Aid agencies regularly conduct evaluations of their own performance, much of which is done through interviews with aid providers on the front line and upstream through national and regional offices to the headquarters; sometimes donors participate as well. However, two gaps remain with this approach. First, these evaluations are rarely consolidated and a sector-wide perspective is hard to obtain. As a result, it is hard to institutionalize the lessons learned across organizations. The Tsunami Evaluation Coalition is working to address this gap by consolidating the non-confidential and voluntary performance assessments of agencies that participated in the tsunami relief operations.

A second gap is the inability to connect the aid provided with the aid received. Was it timely, appropriate, adequate? Was it provided with a process that respected the dignity of those who received it? One idea to address this gap, which emerged at the June 2004 Humanitarian Impact Conference, hosted by Fritz Institute and supported by the Hewlett and Gates Foundations, was to ask aid recipients about their perceptions. It was believed that regardless of the upstream constraints, the perspective of the beneficiary would provide valuable information that has the potential to inform and improve the provision of services by the humanitarian sector.

This approach was operationalized after the December 2004 tsunami in two large-scale quantitative studies conducted by Fritz Institute and the research carried out by TNS India, which has affiliate offices in the tsunami-affected countries. The first study, conducted 60 days after the disaster, surveyed 1,406 affected families in 197 villages in India and Sri Lanka, who were interviewed about their recollections of the rescue and relief services they received in the first 48 hours and 60 days after the tsunami. The second, conducted nine months following the disaster, involved talking with 1,800 affected families from 191 villages. All studies were conducted by trained researchers in the local languages, Tamil and Sinhalese.

[GRAPHIC OMITTED]

In India, the major providers of rescue and relief services during the first 48 hours were the local village community (47%), followed by the Government (23%). This was in sharp contrast in Sri Lanka, where individuals from outside the community provided the majority of the initial services (72%), followed by the relatively weak presence of the Government (8%). When asked to rate the rescue services received within the first 48 hours on a five-point scale, from very poor (1) to very good (5), respondents in India provided an average satisfaction score of 3.8, compared to 2.9 in Sri Lanka. Interestingly, the satisfaction levels were highest in the worst-affected areas of India and lowest in the worst-affected areas of Sri Lanka.

[GRAPHIC OMITTED]

While some of the above variations could be linked to the severity of the impact, which was greater in Sri Lanka, it also highlights the importance of local preparedness and the Government's effectiveness. The Indian Government had made significant changes after the 2001 Gujarat earthquake, and several coastal communities had some exposure to disaster preparedness programmes as they lay in areas susceptible to cyclones. However, as the country had not previously experienced a major natural disaster, the Sri Lankan Government was slower to respond and the level of preparedness was admittedly low.

Surveyed families in both countries reported relatively robust supplies of commodity aid, such as food, water and clothing, during the first two days following the tsunami. In India, 91 per cent reported having received food supplies, 85 per cent received drinking water and 66 per cent received clothing. In Sri Lanka, 78 per cent received food over the first 48 hours, 70 per cent received fresh drinking water and 52 per cent received clothing.

However, despite the strong provision of commodity relief, other critical services were in shorter supply. Less than half of Indian respondents reported search and rescue efforts (48%) following the disaster and only 15 per cent cited debris clearance, while Sri Lankan victims reported even lower levels of service provision, with 31 per cent citing search and rescue efforts and only 8 per cent reporting debris clearance. Some felt that the absence of such services led to further loss of life. One survivor commented that "we feel at least some would have survived if they were searched for in the first day itself". Medical care, while relatively well distributed in India (66%), reached only a third of respondents in Sri Lanka over the two days following the tsunami.

The above findings may be relevant to Governments as they assess the type of services necessary in areas particularly vulnerable to disasters. While basic services like food, water and shelter may be provided by local or national communities, it is critical that specialized services, such as searching for the dead, clearing debris and medical help, be planned for in a systematic way. Survey results strongly indicate that relief (distribution) processes and content (timeliness and substantive adequacy of assistance) are of high importance to aid beneficiaries.

While most surveyed families showed strong appreciation for the services provided by humanitarian agencies, many stated that they were humiliated by the distribution process, which made them feel like beggars, and by the poor quality or inappropriateness of food and clothing. Instances of aid rejection reportedly occurred when the distribution mechanisms or the character of assistance undermined the dignity of recipients. For example, the clothing distribution was considered by many to be humiliating: 55 per cent of surveyed families in India stated that clothing assistance reduced their dignity, as did 33 per cent in Sri Lanka. Many were upset that the clothing was used or in poor condition; others could not find the clothes appropriate to local cultural norms. One Indian respondent noted, "we scrambled the whole heap and still could not get a saree; there were only churidars [trousers] for women". Some felt that the available clothing was simply inappropriate to the local conditions, and one beneficiary in Sri Lanka reported that "clothes given from abroad (were) not suitable for our climate condition".

Surveyed families also responded strongly to the issue of fairness, angrily citing instances where aid was released along political, religious or class lines. While we cannot estimate the scale of inequity in distribution, respondents reported instances in which local elites distorted or seized control over the distribution of relief materials. This finding raises issues about the distribution processes, the type of aid and the consistency of aid approaches by multiple providers. Further discussion and inquiry about the distribution processes are also likely to be useful as the sector seeks to improve its efficiency and effectiveness.

The tsunami had a significant and substantial impact on the livelihoods of those in its path. In Sri Lanka, 59 per cent of respondents saw a reduction of over 50 per cent of family income, with 30 per cent reporting between 76 and 100 per cent reduction. In India, 47 per cent of respondents reported reductions of over 50 per cent, with 17 per cent reporting reductions of between 76 and 100 per cent. Nine months after the disaster struck, we asked the affected families about their satisfaction with the efforts of relief organizations to restore livelihoods, and it was interesting to see the wide variation in satisfaction by area and type of relief provider.

Overall, 24 per cent of respondents in Sri Lanka are satisfied with the efforts of the international non-governmental organizations and 20 per cent with local NGOs, and a much smaller percentage (8%) is satisfied with the livelihood restoration services provided by the Government. While 16 per cent of the affected families in Hambantota and Batticaloa are appreciative of the Government's efforts, over 90 per cent in Ampara, Trincomalee and Jaffna are dissatisfied. On the other hand, over one third of the families surveyed in Trincomalee and Batticaloa reported to be satisfied with the international NGOs services, while 91 per cent in Ampara and 74 per cent in Hambantota are dissatisfied. Similarly, a very high proportion of the affected families in Ampara, Matara and Hambantota were dissatisfied with the efforts by local NGOs towards restoring livelihood.

In India, the results were opposite. In a majority of the districts, government efforts in restoring the livelihood are perceived as more satisfactory than those of international and local NGOs. However, this finding varies by district.

These results provide the foundation for more in-depth analysis of the levels of satisfaction and allow the exploration of relatively different approaches to livelihood restoration and the levels of satisfaction. Aid recipients can be one source of valuable information about the relative effectiveness of aid efforts in the short and long term after a disaster. When combined with other information, data from recipients can be used to identify the relative success of some approaches to relief and development.

Anisya S. Thomas is Managing Director of Fritz Institute, which addresses complex operational challenges in the delivery of humanitarian aid worldwide. She was an associate professor at Florida International University and has written numerous articles on competitive strategy, international entrepreneurship and cross-cultural management.

[ILLUSTRATION OMITTED]