Printer Friendly

Camp was great, but the water was too cold.

An of us would like to work in the "perfect camp." We want the campers to have self esteem-building experiences, the counselors to perform their duties enthusiastically and appropriately, the promotional materials to shout "come to camp," the budget to balance, and the score on the ACA standards visit to be 100. We don't ever want parents to complain or to doubt our abilities to own and/or operate a camp. We want to know if the water is just right or too cold.

In reality, however, our camps don't usually run perfectly all the time. We need to continually strive to improve our camps and to make "better camping for all" a reality each year. While none of us will ever run the perfect camp, we can use the techniques of evaluation in our strive toward perfection. This article will describe the what, why, where, who, when, and how of conducting evaluations in camp.

The underlying premise is that evaluation consists of three dimensions: criteria, data, and judgment. Criteria refer to the standards or the ideals by which we evaluate things. Data are the systematic pieces of information or facts related to the criteria that are collected. Judgment is the determination of the value or worth of whatever was evaluated, based on the data collected and the previously determined criteria. Each of these components fits together into the equation evaluation = criteria + data + judgment. We will show how that equation can be used to enhance the accountability and decision-making undertaken in camps.

Evaluation as the Framework

for the Camp Program

Evaluation includes all strategies and technologies that are used to determine the value and worth of programs, facilities, administrative procedures, and staff within organizations. It refers to formal procedures like end-of-the-session evaluations and ACA standards visits, as well as informal, day-to-day, intuitive assessments of how the camp is operating.

Another term often associated with evaluation is assessment or needs assessment. The needs assessment process provides the initial basis for planning and decision-making. It should be noted that interactions occur between the needs assessment and the actual camp program, the needs assessment and the formal evaluation, and the camp program and the formal evaluation. The many forms of evaluations and needs assessments that exist help to assess where we are, where we want to be, and how we can reach our desired goals.

While evaluation based on informal intuition has been valuable for camp personnel, we contend that in this age of accountability and rapid change, formal systems of evaluation beyond having an ACA standards visit every three years are needed. Developing a system for evaluation, gathering resources, and conducting periodic formal evaluations may be the basis for more efficient operations, staff, and camp programs. The effort will result in increased benefits in camp for children and adults. The focus of evaluation is on improving effectiveness and efficiency in the conduct of all aspects of camp programs.

Evaluations are Critical

The two major reasons for evaluation are accountability and decision-making. Some camp directors are afraid of evaluation because they may not want to learn what they can do better. If everything appears to be going all right, evaluation seems like more work on an already overburdened schedule just to find that everything isn't 100 percent right in camp. The adage, "don't fix it if it ain't broke" comes to mind.

But evaluation is not meant for crisis situations when changes have to be made. Evaluation and assessment done systematically can help a camp director avoid crisis situations. That's one reason for the ACA Standards program. The program's primary purposes are to help camps conduct a self-evaluation and make changes and improvements to comply with minimum expectations or established guidelines, and to have visitors confirm that the effort has been maintained.

Other people disregard evaluations because their prior evaluation experiences didn't tell much more than they already knew. Other camp administrators have done evaluations, but then have not used the data, so the evaluations were seen as simply a "waste of time." While all of these excuses are good, we suspect the major reasons evaluations are not systematically conducted in camps are because camp personnel do not know how to set up an effective evaluation process, how to analyze the data, and/or how to interpret the data in useful ways to assist in accountability and decision-making.

Evaluation is always going on, as we are continually taking in information and making judgments about the world around us. For example, we look at someone's clothing and we think to ourselves how nice the shirt or blouse looks, or perhaps, that another color might look better on that person. If we sit in our office and are uncomfortable with the temperature, we will either turn up the heat or put on a sweater. These examples of evaluations lead us to determine the worth or value of something. We are continually evaluating our immediate situation and this evaluation is ongoing and necessary.

Likewise, a need exists for formal systematic evaluations done with predetermined criteria, reliable and valid data, and an open perspective on how programs, facilities, staff, and administrative procedures in camp can be improved. Good evaluations can lead to improved accountability and informed decision-making.

Developing a Plan

for Evaluation

Ideally each camp director should have a formal evaluation plan for addressing four primary aspects of camp: staff, program, facilities, and administration. The director should determine how each of these areas can be evaluated based on an examination of objectives, outcomes, and standards.

Evaluation by objectives involves testing to see whether the camp's objectives have been met. Reviewing outcomes involves measuring just exactly what happened to campers as a result of their camp experiences. It includes assessing and determining attitudes, new knowledge gained, and personal growth and development. Standards refers to evaluating the camp using the criteria set forth in the ACA Standards program.

We might think about a framework to determine how one begins to know what needs to be evaluated, in what system, and the best way to do it. The figure below provides a matrix that can be used in making these determinations.

The primary aspects of camp evaluations are fairly easy to see. Obviously, staff evaluations are important. The benefits of staff evaluation include improved job performance and provision of feedback for the personal development of staff who are often young and in paid positions of responsibility for the first time. Since evaluation is so important in the personnel process, mid-season or formative evaluations as well as end-of-year or summative evaluations are frequently used.

The formalities of these personnel evaluations are up to die camp director. Generally, staff personnel files are kept which provide documented evidence of the performance of staff. Staff performance evaluations are usually based on a combination of the goals and objectives as found in the job descriptions, and on the performance outcomes that result from doing the assigned jobs.

Ideally, the evaluation should consist of an examination of the relationship between the criteria as stated in the job description and the performance of the staff member. Thus, a well written, accurate job description is the basis for the staff evaluation. The staff member ought to know from the very beginning on what criteria he or she will be evaluated. The staff member must also receive feedback concerning the judgment that is made by the evaluator. The feedback to staff ideally occurs on an everyday informal basis, as well as during the formal process that is scheduled during and after the camp season.

Administrative aspects that might be evaluated include the camp's methods of organization and operation. A great diversity will exist among camps, but many of the administrative aspects can be evaluated best by ACA standards. For example, the self-help guide used in preparation for an accreditation visit can be used by camp directors as they assess the effectiveness and efficiency of their fiscal operations, their business practices, and other professional practices. ACA's administrative standards used in the accreditation process help directors to examine their administration procedures in relation to the minimum expected practices as established for the camping profession.

ACA standards are also most helpful in evaluating risk management and safety concerns in the facilities, equipment, and landscape of a camp. Routine checks of facilities and equipment in the form of "walk-throughs" and scheduled maintenance procedures can serve as a formal system of evaluation as well.

The major focus of most camp evaluations is the camp program, which includes an the activities that occur at camp and the ways in which they are conducted. Outcomes from the camp program also include the perspective of camper growth and development as well as the affective outcomes related to the enjoyment and excitement of camp experiences. Therefore, having goals for the camp program and camper experiences is important in measuring how good the program is, as well as what needs to be improved.

One way to determine some of the interests and desires of the camp participants is to conduct a needs assessment with campers and parents. Written materials to determine criteria and set a baseline for evaluations can be found in Camping Magazine, and also in ACA educational sessions offered at conferences and section meetings. The ability to assess the needs and evaluate the outcomes of the actual camp program from the standpoint of campers, staff, parents, board members, and whoever has knowledge of outcomes is essential.

When, Where and Who

Camp directors could spend their entire summer trying to evaluate all aspects of the camp. This time and resource commitment, however, is not necessary if a system for evaluation is established. The four aspects of staff, program, facilities, and administration are the basis for developing a plan to address when, where, and who should be evaluated.

A system based on the proposed matrix shown in the figure on page 32 may need to be developed to assist in the evaluation process. Different elements of the camp program may be addressed on a rotating yearly basis. Evaluation may also occur at different times during the camp sessions.

Two types of evaluation, formative and summative, must be considered. These types determine whether evaluation is done as a process or as a product (Lundegren & Farrell, 1986).

Formative evaluation occurs while a program or administrative procedure is in progress and is used to examine the process as it is occurring within the camp. "In process" evaluations are often useful even if done informally, and can result in adjustments that make programs and procedures work smoother. Summative evaluation occurs at the end of a program or at the end of a staff member's contract; this type of evaluation addresses the product or the results of the efforts undertaken.

Most people connect formal evaluation with summative evaluation. While staff generally ought to be evaluated every year, not all aspects of the camp operation need to be evaluated every year. Various dimensions of the camp program may be done in cycles over several years. For instance, the evaluation by ACA standards is mandatory every three years for those camps that seek accreditation. After determining what is to be evaluated, the schedule for yearly, biannual, or triennial evaluations should be established and can be organized using the matrix shown in the figure on page 32.

Another consideration for the process is to determine who should be involved in the evaluation. As suggested previously, parents, campers, staff, board members, or outside evaluators may be used. Different ways exist for involving individuals in data gathering to make judgments. Who is involved may depend on what is asked.

What to Ask

Once you have determined what it is you want to learn from an evaluation, have established criteria, and have made a plan for whom to involve, you must determine what and how to ask the questions. What to ask depends on the camp and its particular situation; each camp is going to have different goals and objectives and varied ways to reach them.

Questions are usually asked to determine attitudes, beliefs, knowledge, behaviors, and characteristics (Henderson, 1988). Thus, "How would you rate the food at camp" would be a knowledge question. "How many times did you choose to go to free swim?" would be a behavior question, and "What cabin group were you in?" is a demographic characteristic question. Determining what to ask may depend upon what it is you wish to measure or learn about campers or staff.

Basically two types of data may be collected, quantitative and qualitative, depending on the kinds of questions asked. Quantitative data refer to counted or numeric information. For example, you might ask a "yes/no" question that you tally with percentages to quantify the responses. The data collected by the ACA Standards program are a good example of quantitative data.

Likert scales are another frequently used method of collecting quantitative data. For example, you might ask, "How would you rate the food? 1 = terrible, 2 = fair, 3 = good, 4 = excellent. From these data, the camp director can develop averages to see how various features of the camp relate to other features, or she can determine the percentage of campers who thought the food was terrible, good, and so on.

Qualitative data allow the individual to describe through words the experience that he or she has at camp. Open-ended questions that allow for elaboration on the part of the camper, staff, parent or whoever is filling out the questionnaire result in qualitative data. An example of possible open-ended questions would be: "Tell me about the best thing at camp" or, "Describe what you dislike about the evening sing-alongs and campfires." These questions are easy to ask but are not always as easy to analyze because of the wide range of responses that may be obtained. These data are usually more useful in providing in-depth understanding about issues that may be impossible to acquire through quantitative procedures. They also may be better asked in an oral interview rather than a written questionnaire.

Depending upon what criteria are being measured, either quantitative or qualitative data, or both, may be useful. While qualitative data may be most useful in evaluating camp programs, they may be used in any area of camp evaluation. The camp director will need to determine how best to collect the data to assist in decision making.

In evaluating camp programs, the doors are wide open for determining what to ask and how to ask evaluation questions. Many aspects can be evaluated within the actual program: the food service, the facilities, the leadership, the evening activities, specific camp activities, the overnight trips, and so on. Learning about another camp's evaluation system or seeing examples of other camps' evaluation forms are often useful. But camp directors must tailor their evaluations to their unique setting and to the criteria that they wish to measure for use in their camp.

Further, while we typically think of evaluations as pertaining to questionnaires, face-to-face interviews as well as participant observations may be useful. In the event that interviews or observations are used, these procedures must be documented in some way with written notes, if they are to be reliable. Thus, systematic evaluation almost always requires some type of written materials.

Throughout the evaluation process of camps, camp directors must be realistic. For example, to ask campers if they would be more likely to return to camp next summer if horseback riding was included may not be appropriate unless serious plans exist to incorporate that program option into camp. Another realistic approach to evaluation is to focus on particular activities over the course of two or three years rather than to evaluate everything every year.

It is essential to only collect data that are going to be used. To collect evaluations from 300 campers and then not analyze and use the data is a waste of both the campers' and director's time. It may not be necessary to collect data from every camper or every parent either. From a public relations standpoint, you may want to have contact with every participant; but from a statistical standpoint, a percentage of the total population may be adequate in the sampling process.

A random sample may be more manageable than having everyone do an evaluation. Random means that everyone has a chance of being chosen. Thus, if you have 10 cabin or tent groups, you would literally put their names in a hat and draw out two to get the random sample. You may want to consider focusing on certain age groups or select certain activities in alternate years.

It may be important to know if different age groups or first-year verses second-year campers experience camp in different ways. In such cases, one also needs to ask questions or code questionnaires in such a way that you can compare groups.

Appropriate wording of written questionnaires and verbal evaluations is essential for accurate and valid data collection. To ask children how camp has contributed to their social skills obviously will not yield the kind of results that we'd like. Asking about new friends, however, may be more understandable and still provide data useful in evaluating outcomes of camp from the camper perspective.

Who Cares?

The best evaluation criteria and data collection will be useless unless it is used in some way for decision-making and/or accountability. Once the data are collected for any of the areas of camp, they must be analyzed. To analyze qualitative data involves summarizing the main points of the comments collected. Analyzing quantitative data involves tabulating percentages, averages, and other numerical summaries. With the widespread availability of microcomputer statistical packages today, more sophisticated statistical calculations can be done if desired.

No matter how the data are analyzed, however, judgments will need to be made. If 60 percent of campers think the food is delicious and 40 percent think it needs improvement, does that mean that the camp is doing all right, or does it mean that it needs to improve? These questions are not easy to answer. But if no decisions about changes are going to be made, or if no one is ever going to see the results of the evaluation, then evaluations should not be done.

We already mentioned that staff must receive feedback if they are to improve and learn from their work experiences. We also suggested that having ACA standards visitors come to camp for accreditation is an important form of feedback for the camp director and staff concerning all aspects of the camp program. For the camp program and activities, however, each year the aspects that are evaluated should be carefully scrutinized and the results of the program evaluation should be reported. A formal report may be written or bits and pieces of the analysis may simply be systematically incorporated into the camp program. Regardless of the means used for reporting, some system needs to be in place for using the evaluations.

The Bottom Line

We teach a course entitled "Evaluation of Leisure Services" in the University of North Carolina's Curriculum in Leisure Studies and Recreation Administration. It is probably perceived as the most unpopular majors course we offer. Yet, over the years, students have come back to tell us how important that course has been for their professional careers. While evaluation often sounds like a boring topic, no one can debate its value in the 1990s when resources are dwindling and we are trying to provide more and better services with fewer resources. Evaluation can help us to be more effective and efficient if it is used properly.

Evaluation is not something you can learn by reading this article, going to a three-hour workshop, or reading a book. Having the basics, however, can help you to begin a process of learning. Each time you do an evaluation you will learn something that will be helpful for next time. Thus, it is a matter of practice, practice, practice and tuning into the unique needs of your camp.

To summarize the key points presented in this article we offer the following considerations:

1. Evaluation includes both informal and systematic formal processes carried out in the camp.

2. Evaluation occurs at its best when criteria, data, and judgment are used together to improve the camp.

3. The purpose of evaluation is to gather data that can be used to make the camp run more effectively and/or efficiently.

4. Many aspects of camp (staff, facilities, administration, and program) can be evaluated; the director must develop a system for evaluating each periodically.

5. Data collected can be qualitative or quantitative; the director must determine for his or her camp which are the appropriate ways to collect data.

6. Evaluation is our friend, not our enemy. None of us is perfect so we should not be afraid to learn that we always have room for improvement.

7. Developing good evaluation systems is a career-long undertaking. Because so many options exist for evaluation and because we live in such a changing world, the opportunities for evaluation will be numerous and frequent.

We all have a strong commitment to our campers, our program, our staff, and our environment. We want our camps to provide the most enriching experiences possible. Our challenge is to determine how to best use the opportunities for evaluation in our camps so we can enhance our professional efforts. Only with sound evaluation systems can we strive toward truly developing "Better Camping for All.


Henderson, K.A. (1988). Questionnaire development: Fun or frustration? Camping Magazine, 60 (7), 32-33. Lundegren, H.M. & Farrell, P. (1985). Evaluation for leisure service managers. Philadelphia: Saunders College Publishing. American Camping Camping Association, Inc. (1992). Standards for day and resident camps. Martinsville, IN: American Camping Association, Inc.

Karla A. Henderson, Ph.D., is a professor in Curriculum in Leisure Studies and Recreation Administration at the University of North Carolina. She is currently a member of the ACA National Education Council.

M. Deborah Bialeschki, Ph.D., is an associate professor in Leisure Studies and Recreation Administration at the University of North Carolina. She is currently chair of the ACA Standards Board.
COPYRIGHT 1993 American Camping Association
No portion of this article can be reproduced without the express written permission from the copyright holder.
Copyright 1993, Gale Group. All rights reserved. Gale Group is a Thomson Corporation Company.

Article Details
Printer friendly Cite/link Email Feedback
Title Annotation:1993 J. Wendell Howe Golden Quill Awards; conducting camp evaluations
Author:Bialeschki, M. Deborah
Publication:Camping Magazine
Date:May 1, 1993
Previous Article:Environmentally hot or not?
Next Article:Outdoor programming for older adults who are frail.

Related Articles
Cameras in camp: helping campers understand principles of photography.
Environmentally hot or not?
Evaluation: what to measure and why?
Looking ahead by honoring the past.
Using gates to enhance your paddling program.
Year-round schools, children's needs.
ACA Presents National Awards.
Entries Wanted for J. Wendell Howe Golden Lens Awards. (the American Camping Association[R]).
Do you know what your outcomes are? The impact of Oregon 4-H residential camp programs on positive youth development.

Terms of use | Copyright © 2016 Farlex, Inc. | Feedback | For webmasters