Printer Friendly


Byline: Ejaz Ashraf, Saima Sadaf, Muhammad Yaseen and Hafiz Khurram Shurjeel

ABSTRACT: Publicly funded agencies in all over the world are expected to demonstrate the impacts of their programs, projects, and policies. Evaluation is a management tool that involves both progress of the programs or projects as well as end results or outcomes. Process evaluation deals with necessary measures for delivery of the services and for improvement of the program. On the other hand, outcome evaluation allows program leaders to be more effective in making decisions about future of the programs based on the results whether the program has met its goals and objectives for which it was initially designed and undertaken. Moreover, outcome evaluation also identifies cause-and-effect relationship between activities and outcomes of the program and to fix responsibilities of different stakeholders for success or failures. The continuation or closure of the program depends on outcome evaluation.

It is imperative for every organization to follow internal or external evaluation for improvement of programs, to make decisions about future of the programs, and to identify certain reasons between actions taken during program for delivery of services and the outcomes of program.

Key words: program, evaluation, outcome, improvement, stakeholders


Evaluation is both an art and a science. The art of the evaluation involves identifying purpose and stakeholders, appropriate designs for actions and delivery of services, and interpreting goals and objectives of program or project. The science of evaluation involves systematically gathering and analysing evidences about process and impact of program. Evaluation is determining the worth or merit of whatever is being evaluated [16]. The object can be a program, a project, a product, a policy, or a one-off event. Many different uses can be made of those value judgements from assessing the financial or social impact of any program, to improving program designs and plans for new programs. Evaluation has many aspects and will be discussed in subsequent sections. Formative and summative evaluation Reference [15] was the first evaluator to write about the differences between formative and summative evaluation.

Since then the terms have become almost universally accepted in the field of evaluation. The distinction between formative and summative evaluation is concerned with ultimate use of value judgement. Formative evaluation is conducted to provide necessary judgements for implementation of planned actions and useful improvements in the program. Summative evaluation is generally conducted after completion of the program (or when a program has sustainability) and for benefits of different stakeholders or decision-makers. The findings from summative evaluation could be used to decide whether to continue a program or not, or to justify program money. The main difference is that the aim of a summative evaluation is to report on the program, whereas a formative evaluation keeps eye on program weaknesses and measures to overcome them [16].

1.2 Formative or Process evaluation focuses on the process, actions or services delivered during program. It involves concepts, ideas, goals, objectives and actions for delivery of services during a program. Process evaluation is generally formative in nature. A process evaluation highlights on actual development and implementation of any specific program. It makes sure whether goals and objectives are implemented as planned. In other words, it documents the process of implementation of any program. Formative or process evaluation also helps stakeholders to see how program gives long term impact on respondents for whom this program was initially designed.

1.3 Why formative evaluation is important in extension programs?

A focus on outcome evaluation is understandable in the context of budget restrictions and an increasing need to be accountable for public money. There are several arguments that highlight the reasons for focusing exclusively on outcome (summative) evaluation. One argument is; why stakeholders wait till the end of the program to conduct summative evaluation. Indeed the need is to give priority to conduct formative evaluation in agricultural extension programs to pinpoint the weaknesses and to take necessary measures before all the resources and public money consumed. The term "process-orientation' is used to describe programs that do not have rigidly defined goals at the onset of the program but have a defined purpose and a set of potential goals.

Programs with a "process' orientation are difficult to evaluate as they are carried out in variable, unpredictable situations; they produce outputs that are hard to measure objectively and often have permeable boundaries and less-than-direct relationships between inputs and outputs [6]. The current resurgence in outcomes-oriented evaluation is clearly not a new concept for program evaluation. In 1932 Tyler set up an eight-year study in which he evaluated 30 schools on the basis of objectives that were identified by the teachers themselves. This has been cited as the first comprehensive study of program evaluation [9]. The concept of program appraisal was a new perspective introduced by Tyler. Program evaluation can be described as multi- professional as it shares certain attributes with other professions.

1.4 Relevance of evaluation to Agricultural Extension system

Extension system refers to the conscious use of communication and information to help people to form sound opinion and make good decisions [18]. The evaluation of agricultural extension programs implies the systematic collection of information about activities, characteristics, and outcomes to make judgments related to programs, improve its effectiveness, and/or inform decisions about future programming. There are several factors making evaluation an important issue in agricultural extension programs today. The changing face of agricultural extension, budget cuts, and the environment of accountability for program money all contribute to anticipating role of evaluation that need to play.

1.5 Purpose of evaluation

The novel purpose of evaluation is to know the value or significance of something. Evaluators have to decide whether desired goals and objectives of the program have been achieved.

Discuss the purpose of the evaluation in explicit terms. Evaluation may be used to:

- Provide guide lines to spend public money during the program

- Communicate the evaluation process and results to different stakeholders of the program.

- Identify strengths and weaknesses of a program

- Determine the outcome of a program.

1.6 Steps in conducting program evaluation

1. Identify purpose, reasons for conducting the evaluation and scope of the evaluation.

2. Review program goals as stated and anticipate the measurement techniques

3. Identify the stakeholders of the program those have special interest in evaluation.

4. Obtain input/feedback of the stakeholders and see how they perceive the program, project or policy.

5. Based on discussions with stakeholders, review the purpose of the evaluation.

6. Decide whether stakeholders need internal or external evaluation.

7. Decide the budget for evaluation.

8. Determine data-collection methods. Decide on data- collection procedures to answer your evaluation questions.

9. Create data-collection instrument. Construct or adapt existing data-collection instrument(s) (such as surveys or interviews).

10. Test the draft instrument for validity and reliability before administering it.

11. Test data-collection instrument by conducting a pilot study with a small group of respondents to see how the designed instrument work in the field before conducting full-scale study.

12. Collect evaluation data.

13. Summarize and analyze the data.

14. Prepare reports for stakeholders.

15. Communicate the results of the evaluation to stakeholders [2].

1.7 Performance indicators used in Evaluation

A performance indicator is a simple statistic recorded over time, to inform managers of the success of some aspect of program management. Examples of indicators in agricultural extension might be the number of farmers contacted by extension agents per year, or the number of members participating in discussion groups. Reference [13] stresses the importance of ensuring that evaluators use a full range of data collection and analysis techniques. Reference [7] listed five main categories of indicators:

- Effectiveness - the extent to which a program is satisfying the purpose for which it was established.

- Social justice - the social impact of a program in terms of equity, equality, access and participation.

- Operational efficiency -the relationship between inputs and outputs. This includes both productivity and aspects of service delivery

- Outcome efficiency - the relationship between outputs and outcomes.

- Standards of service - the quality of the service to clients. Reference [17] described that although there is negligible conceptual difference in all existing monitoring and evaluation approaches, yet there is wide open chance to work for common standards for conducting monitoring and evaluation for any project or program to agree upon common indicators for measuring key factors or variables.

Reference [11] pointed out that given the increasing tendency for governments to look for ways of decreasing public money on agricultural extension services, there is a growing need in Australia to conduct evaluation which provides evidence as to whether public sector involvement in agricultural extension is justified for overall expenditure on agriculture. Great economic and social benefits are obtained by the use of public funds on agricultural extension.

The relative importance of agriculture in the economic growth of industrialised countries has declined. Added to this, increased use of externally-purchased inputs has changed the nature of publicly funded extension services and lead to questions of the rationale for the delivery of such services by governments [4].

Public funding for research and development (R and D) is shrinking and agricultural programs must meet increasingly high standards of accountability and quality. Well-designed evaluation approached or methods are highly desirable for planning and implementing high quality agricultural extension programs, and hence assuring public benefits from R and D expenditure.

In 1997, the Research Branch of the British Columbia Ministry of Forests conducted a formative evaluation of biodiversity publications that had been released during the previous several years by the Royal British Columbia Museum and the provincial ministries of forests and environment. The evaluation was designed to reveal how clients prefer to get biodiversity publications and to measure client satisfaction with the existing distribution system. The agencies hoped to use the evaluation results to improve exposure of future publications.

According to reference [19] although the root of evaluation development lies in the US, in the 1960s evaluation began to surface in Australia and later in Europe. The Australasian Evaluation Society was the first non-American evaluation society to emerge. It has been remarked that the development of program evaluation in Australia differed somewhat from that in the US, in that internal evaluation was always more prevalent than in the US. This can perhaps be explained by the attitude in the US to evaluation as an academic discipline and the existence of the numerous evaluations higher degree courses there. In the USA, evaluation tends to be seen as something that is done by trained external evaluator qualified with Ph.D. whereas in Australia there tends to be a greater focus on internal evaluation

According to reference [1] there is now considerable agricultural R and D activity that was instigated on the basis of claims that farmer participation is critical to the generation of technologies that are relevant to farmers

According to references [12], [5], and [8] in Australia, systems based models of R and D have been recommended or applied in some regions although the approaches vary widely. The Hawkesbury model of agricultural systems development is based on experiential learning [10], which encapsulates the concept of learning through experience. Most such systems approaches incorporate a strong element of iterative evaluation. Overall, program evaluation is now more relevant to agricultural extension in its many forms than it has ever been. This trend has been gaining momentum over recent years. One would therefore expect extension professionals to have developed interest and expertise in program evaluation in response to these clear signals. Apparently this has not generally been the case.

Reference [14] describes three primary uses of evaluation such as to:

- judge merit or worth of program (such is the purpose of a "summative evaluation"),

- improve programs (such as the purpose of a "formative evaluation"), or to

- generate knowledge.

A summative evaluation is conducted at the end of a program to help decision makers to decide about future of any program. A formative evaluation is conducted during the life of a program to identify its strengths or weaknesses and to enhance its quality and effectiveness. An evaluation conducted to generate knowledge is usually carried out by academics examining trends or causal links across programs. Program evaluation is related to policy evaluation; however "programs" typically include human resources and infrastructure while "policies" are more likely to be regulations or standards with or without infrastructure.


Research and formal evaluation are both modes of inquiry. While employed to examine different phenomena for different purposes, they are both subject to the same standards of excellence if their findings, conclusions and recommendations are to be taken seriously.

3.1 Need for in-service training in evaluation for professionals

As a result of the changing nature of agricultural extension at the macro level, there is a strong demand from Government departments, R and D Corporations and other institutions for personnel who are well-trained in evaluation to take on a wide range of evaluation tasks in rural industries. Existing information on evaluation is largely out of date and inaccessible to many extension agents. Perhaps this can be explained by the nature of the aim of agricultural extension itself, and by physical science-oriented training of most extension agents. Agricultural extension aims to change behaviour through the use of communication. Training to measure behaviour change should be conducted, especially in quantitative terms. As most agricultural extension workers have little training in social sciences. They are likely to be unfamiliar with the range of qualitative data collection methods in evaluation programmes.

There is a dire need of training opportunities in evaluation approaches for extension practitioners and program developers. Evaluation should be high on the agenda of extension organisations.

3.2 The new culture of agricultural extension

In today's world, the emphasis is on adult learning, understanding existing farming system, practices of extension worker as facilitator and hence invites practitioners to conduct evaluation of all these activities. To cater this change in extension culture, new evaluation approaches have to b adapted. Further, policy makers and program implementers need to establish some mechanism for evaluation for providing valuable information to achieve project aims.. The formative evaluation results should be utilised continuously in improving the management of the project and to guide others in establishing extension programs.

The policy makers and implementers should ensure no physical, emotional or other harm as a consequence of participation in the evaluation of programs. It is imperative for every organization to follow internal or external evaluation for improvement of programs, to make decisions about future of the programs, and to identify certain reasons between actions taken during program for delivery of services and the outcomes of program.


Following are few recommendations for policy and decision makers in the field of monitoring and evaluation:

- Need to develop understanding for having internal or external plans for evaluation in all government or non-government organizations in the country.

- It is highly recommended that sustainable self- assessment policy must be existed in every organization.

- Monitoring and evaluation plans need to be communicated to all stakeholders of the projects and programs

- All stakeholders must uphold the outcomes of the evaluation for quick, time-bound and merit based decisions


[1] Anderson, F.M. and Dillon, J. L., "Farming systems research in the IARCs and other international groups." In J.V. Remenyi (ed.), Agricultural Systems Research for Developing Countries, ACIAR Proceedings 11, 141-147 (1985).

[2] Bennett, C.F., "Analyzing impacts of extension programs." US Department of Agriculture, Washington, D.C. (1976)

[3] Bebbington, A. J., Merrill-Sands, D. and Farrington, J., "Farmer and Community Organisations in Agricultural Research and Extension: functions, impacts and questions." ODI Agricultural Administration (Research and Extension) Network paper 47 (1994).

[4] Cary, J. W., "Changing Foundations for Government Support of Agricultural Extension in Economically Developed Countries." Sociologia Ruralis, 18, 336- 347(1993).

[5] Clark, R.A., Bourne, G.F., Cheffins, R.C., Esdale, C.E., Filet, P.G., Gillespie, R.C. and Graham, T.W.G., "The Sustainable Beef Production Systems Project: Beyond Awareness to Continuous Improvement." Queensland Department of Primary Industries, Brisbane, Project Report Series Q 96002 (1996)

[6] Farrington, J. and Nelson, J., "Using Log frames to Monitor and Review Farmer Participatory Research." Overseas Development Institute, London, Network Paper No. 73, (1997).

[7] Fish pool, K. I. (1993) Quality Assurance Evaluations. An Outline of Principles/Best Practice for Program Evaluators in NSW Agriculture, NSW Agriculture.

[8] Foale, M.A., "Management of dry land farming systems in North Eastern Australia. How can scientists impact on farmer's decision making?" Agricultural Science 10, 34-37(1997).

[9] Gredler, M. E., "Program Evaluation," Prentice- Hall, inc., Englewood Cliffs, New Jersey (1996)

[10] Kolb, D.A. "Experiential Learning: Experience as a Source of Learning and development." Englewood Cliffs, NJ, USA (1984).

[11] Lees, J. W., "More than accountability." The Rural Development Centre, University of New England, Armidale (1991).

[12] Mc Cown, R.L., "Research in a farming systems framework. In V.R. Squires and P.G. Tow (eds),Dry land Farming: A Systems Approach. An Analysis of Dry land Agriculture in Australia." Oxford University Press, Melbourne, 242-249 (1991) .

[13] Owen, J. M., "Program Evaluation, Forms and Approaches" Allen and Unwin, St Leonards, NSW, Australia, (1993).

[14] Patton, M. Q., "Utilization Focused Evaluation." Sage, Thousand Oaks, USA, (1997).

[15] Scriven, M., "The methodology of evaluation." In Curriculum evaluation. Edited by Stake. Chicago: Rand McNally (1967).

[16] Scriven, M., (1991) "Evaluation Thesaurus" (4th Edition) Newbury Park: Sage

[17] Stem, C., Margoluis, R., Salafsky, N., and Brown, M., "Monitoring and evaluation in conservation: A review of trends and approaches." Conservation Biology,19(2) 295-309 (2005).

[18] Van den Ban, A. W. and Hawkins, H. S. "Agricultural Extension," Blackwell Science, Berlin, Germany (1996).

[19] Winston, J., "Personal communication. Royal" Melbourne Institute of Technology, Bundoora Campus, Melbourne (1997).
COPYRIGHT 2016 Asianet-Pakistan
No portion of this article can be reproduced without the express written permission from the copyright holder.
Copyright 2016 Gale, Cengage Learning. All rights reserved.

Article Details
Printer friendly Cite/link Email Feedback
Publication:Science International
Article Type:Report
Geographic Code:9PAKI
Date:Aug 31, 2016

Terms of use | Privacy policy | Copyright © 2020 Farlex, Inc. | Feedback | For webmasters