Evaluating a good season.

Several months after the ending of the 2002 camp season, a concerned camp director contacted me. As the director of a small nonprofit camp, he had received a bit of critical feedback from the board of directors on his general performance. The director was angry and thought that this was more of a personal attack rather than having anything to do with his actual performance. Through questioning, it became apparent that this man was intelligent and quite knowledgeable about the camp industry. Yet when I asked him to explain how successful the past few seasons had been, his answers indicated quite a bit of naivety. He stated that the past three seasons had gone "very well." When I asked him to define the criteria he used to formulate this conclusion, he looked at me as if l was actually the one in need of coaching.

"I was there the entire summer," he informed me. "I know more than anybody just how well the season went."

The belief that a camp director is the one with the most knowledge regarding the success or lack of success for a camp season is quite prevalent. Indeed, we would expect this to be the case with every organization. The top person--CEO, president, or director--is expected to be the person with the most knowledge about the overall success for his or her organization. However, this is not necessarily true. Having worked as a camp counselor and specialist, I can easily recall information that was never known by the director and the administration, yet still known by the majority of the other staff. We knew how much food was stolen by the kitchen staff. We knew of the sexual assault on a female staff member that was not reported. We knew of the use of the camp vehicle for "joyriding" after hours.

Thus, when a camp director states that he had a "good" season, I proceed to ask what this evaluation is based upon. In most cases, it comes down to his or her general impression. When pushed further, he or she will provide data--such as higher staff retention rates, fewer complaints from parents, and/or fewer children sent home. When asked for exact statistics--Just how many parental complaints did you receive this year? How about last year? What is the comparison in staff retention rates over the past three years?

Deciding whether a season was good or bad is exceedingly more complicated than a director's general impression. Reliance on one's impression as the determining factor is actually a setup for long-term failure. In an example from psychology, researchers have found a consistent pattern in humans of erroneous evaluations for recent events. In one experiment, individuals were instructed to submerge one hand in a bucket of icy water for a minute (this becomes quite a painful experience after only a few seconds). After a brief break, they were given the same instructions except that the length of time was ninety seconds. On the second trial--unbeknownst to the participants--the temperature of the water was raised slightly, though it was still uncomfortably cold. After a seven-minute break, the subjects were called back into the room and informed that they would have to repeat one of the above procedures. They had the choice of which to undergo for a second time. Surprisingly, the majority of the subjects chose the long procedure over the short even though this actually resulted in more discomfort. In tests with aversive sounds and pictures and even uncomfortable medical procedures, participants chose a longer one with a decrease in discomfort at the end over a shorter one without such a pain decrease.

How does this apply to evaluating a good or bad camp season? The quality of an experience at its conclusion has a great effect on evaluations. Thus, camp directors might have an awful season overall, yet a final few good days will skew his or her perspective to a less negative evaluation. Note that the opposite is also true. A bad event in the final few days can mar the evaluation of a fairly good season. Again, an evaluation of a camp's season based on the director's impression is often meaningless without supportive data.

Obstacles to Face in Evaluation

There are other obslacles that directors face in their evaluations. The human brain is a wonderfully complicated organ. It has allowed us to ascend above all other creatures. In order to achieve this position, the brain has had to make use of "shortcuts." One of the tenets of brain research is the brain prefers speed over accuracy. The brain is constantly making decisions every second. These shortcuts are used to process information rapidly. Unfortunately, this speed comes at the price of accuracy.

Reliance on Memory

Many decisions require us to consider information we have learned in the past. A problem with this tactic is that we use the stored information in our memory that is most easily retrieved. A more vivid memory will be recalled even if it is not the most representative example. One of the most commonly cited examples is airline disasters. Statistic after statistic shows that flying is far safer than driving. However, when a person is making a decision to travel, a recent airline tragedy could well sway the decision because of the perceived increased risk of danger.

If a camp director recalls numerous negative events over the season, he or she may then decide that the season was "bad." But was it? There are without doubt other less-negative events that he or she simply cannot recall, but the negative events are more easily retrievable.

Anchoring in the Present

The mood a person is in or the emotion he or she is currently experiencing acts as an immediate anchor and influences the memories he recalls. An individual who is angry will more easily be able to recall memories associated with this particular emotion, even if the memories have nothing to do with the immediate situation. An emotion of satisfaction will likewise elicit other pleasant memories while, at the same time, inhibit negative memories. Thus the current mood of the director influences what he or she recalls. For example, on a "good" day, the director will have excellent recall of all of the good events that occurred during the camp season. This would lead him to evaluate the season as a good one. The director truly cannot recall the negative events that occurred; this does not mean that they did not happen.

Three Solutions

There is no doubt that directors can create long-standing successful camp programs. But how does one determine from season to season if the camp is truly going in the right direction and not just following the director's impression of success? Three general methods can be used to assist a director in his or her evaluations of the success (or lack of success) of a camp season:

Goal Setting

It behooves directors to set goals before the season starts. How many parental complaints are acceptable? What number of staff is expected to turnover during the season? Through goal setting, a director enters the season with guidelines for success. If the director accomplishes--or surpasses--these goals, this is indicative of success. The American Camping Association even has a standard on strategic planning in its operational management section of the Accreditation Standards for Camp Programs and Services. Such a standard focuses a director's attention to both short- and long-term goals.

Satisfaction Surveys

Camp directors may wish to focus on ratings of staff, participant, and parental satisfaction. This entails formal ratings that can be tracked from year to year. In this way, several seasons' worth of data can be compared. The examples of general satisfaction surveys for parents, campers, and staff to voice their thoughts and reactions to a traditional camp program are included (see pages 20 and 21). These are starting points for devising a basic picture of consumer satisfaction and can be used until a more formal protocol has been devised. Satisfaction surveys are different from performance measure evaluations--which are used to evaluate the success of a camp in meeting its mission and stated objectives. These too are another part of a comprehensive camp evaluation.

Involve Other People

All camp employees have valuable information regarding whether a season is successful or not. A director is not omnipotent he or she cannot be everywhere at one time nor see what employees see. Make sure other people have input into the determination of success or failure.

Feedback

Unfortunately, directors cannot be everywhere at one time in a camp. Particularly during the height of the camp season, many directors believe that their major role is to put out the small "fires" that occur on an almost daily basis. Strategic planning for the future is simply not a top priority when a director must deal with campers, their families, and staff, it is therefore necessary for directors to have a sufficient source of quality information.

Fortunately, there are numerous sources of information available to a director on the success or lack of it in a camp. Every staff member has his or her own perspective regarding the camp. Those individuals "in the trenches" may actually have feedback that would he of great benefit to the director. They may offer ideas that would save money, improve programming to bring in more campers, or make the camp more organized on a day-to-day basis. And, possibly the most intimidating, they could have feedback for a director regarding his or her leadership abilities.

Camp directors who believe that they can make important decisions without the input of others are risking very bad judgments. Indeed, psychological research has dearly demonstrated that decision-making is not necessarily a strong point for humans. In 1954, Paul Meehl published a highly influential and controversial book rifled Clinical Versus Statistical Prediction. The book summarized twenty studies comparing the judgments of professionals (in this case, psychiatrists and psychologists) to purely statistical models. With the statistical model, the human judge is removed, and the decision is based on strict relations between appropriate data. As an example of this latter method, life insurance agents obtain data on a person (age, health status, personal habits, etc.) and plug them into already existing tables and charts that have been formulated to estimate life expectancy.

In all studies in Meehl's book, the statistical method was either superior or tied with the clinical method. In no study was a personal judgment superior to the statistical method. In the decades that have followed, this result has been found over and over again. Even "experts" in their fields have rarely been found to make better decisions than statistical methods. Fields that have been studied include medicine, psychology, law, and insurance. And though there has never been a study comparing the judgments of camp directors to statistical methods, there is no doubt that the same result would be found.

Obstacles to Feedback

Several obstacles may prevent employee feedback and participation. Even employees who have wonderful ideas will likely refrain from approaching a leader if the following t;actors exist:

* Fear--Leadership is not a personality characteristic; it is comprised of a set of skills and maturity. However, if leaders use coercion and punishment as the major "motivator" for employees, they will likely be afraid to approach him or her. Leaders who demonstrate a lack of emotional control (e.g., sudden bouts of rage or who are easily angered) will also intimidate employees into silence. No employee, particularly if they are seasonal and therefore temporary, desires a confrontation with such a boss.

* Barriers--Directors may have so many immediate distractions during the summer that they do not have the time to meet with staff. One camp director who reported on his "open door policy" during the staff training quickly created a formal chain-of-command structure during a particularly hectic season. Now staff were required to meet a unit leader, then the boys' camp director, then an assistant director, until finally they were allowed to meet with the director. The more obstacles that stand in the way of an employee, the less likely he or she will exert the energy to overcome them.

* Directors' presentations--Some directors clearly verbalize an interest in input from staff. Others never mention it and make no efforts to seek such input. A camp director who actively seeks input will certainly encounter more than one staff member who portrays a lack of interest.

* Reliance on trusted staff--Directors are only human and, in spite of attempts to remain neutral, will have favorite employees. It is not surprising that these individuals' input is requested. Unfortunately, in the intimacy of a camp setting, a director's chosen favorites become quickly obvious. Other employees may experience jealousy or resentment and avoid offering suggestions. The director, in turn, is reliant on a select group of individuals for information. However, the more varied the input, the more quality information is available to make a proper decision. It is often the employees who a director really doesn't like who have important critical feedback.

Additionally, each individual staff member has an internal life that affects the decision to offer feedback. When a staff member conceives of a suggestion or feedback, he or she does not immediately run to the director and share it. Such a faulty understanding is illustrated in Figure 1.

[FIGURE 1 OMITTED]

Staff usually evaluate their ideas or feedback using secondary interpretations, which either galvanize or prevent an employee in his or her attempt to share the information with the director. Several common interpretations are listed below:

* Is the idea any good?--Before sharing the idea, feedback, or suggestion, the staff member will consider its worth.

* Is it safe to offer this information?--A staff member may have a great idea or pertinent feedback, but if he or she determines that it is not safe to offer it (such as with an easily angered leader), he or she will then refrain from sharing.

* Will the director listen?--If the director has demonstrated a lack of interest in suggestions and feedback, the staff member will not follow through.

* How much effort will I have to go through to offer this suggestion? A camp season is hectic for all staff members; free time is limited. If a staff member decides that too much time and energy is required to "sell" the idea, the motivation to share it will decrease.

A more realistic depiction of the process of offering suggestions or feedback is offered in Figure 2.

[FIGURE 2 OMITTED]

Encouraging Feedback

As Figure 2 illustrates, simply because a staff member has formulated valuable feedback, there is no guarantee that it will ever reach the attention of the director. Characteristics of the camp environment, the director, and individual interpretations will effect the decision to share the ideas or feedback. The following suggestions make it more likely that such information does reach the director.

* Create a policy regarding suggestions and feedback and be consistent in its implementation--Will there be an open door policy? Is there a chain of command to be followed? Is there a suggestion box that is emptied every day? Formulate a specific protocol for staff suggestions, inform staff of it, and consistently follow through in implementing it.

* Seek input from a variety of sources--do not rely on one's own judgments or those of favorite employees. Purposefully attempt to obtain the feedback from quiet, disgruntled, and/or "problematic" employees.

* Simplify the feedback process--the more accessible a director is the more likely staff will offer feedback. Requiring a written request three weeks in advance for a meeting is guaranteed to lessen staff enthusiasm.

Feedback is essential for leadership improvement. Additionally, staff have numerous ideas that can truly assist the camp. Leaders who do not make themselves accessible to feedback and suggestions are wasting valuable resources. A camp director who does not seek feedback and suggestions from staff is one who needs much more work in developing his leadership skills. Furthermore, he or she might well be an obstacle to the growth and development of the camp. In combination with satisfaction surveys and goal setting, ongoing feedback will remove the onus of full responsibility for evaluations of camp success from the well intentioned but likely faulty recall and .judgment of the camp director.

2003 Season Camper Satisfaction Survey

Please circle the appropriate information:

1. How old are you?                   7  8  9  10  11  12  13
2. Are you a male or a female?        Male Female
3. What encampment are you attending?

   First  Second  Third  Fourth

Please give us your opinions in questions 4-7

Scale: 1 = not at all 2 = dislike 3 = neutral 4 = like
       5 = like very much

4. How much did you like camp?              1  2  3  4  5
5. How much did you like your counselors?   1  2  3  4  5
6. How much did you like the camp food?     1  2  3  4  5
7. Would you comeback to camp?              Yes  No

Of the following activities, would you prefer more or less to occur
in camp?

boating                         less no change more
fishing                         less no change more
swimming                        less no change more
arts and crafts                 less no change more
nature hikes                    less no change more
overnight camping trips         less no change more
sports                          less no change more
cabin meetings/bunk meetings    less no change more

In the box below, write two things that you think would make
the camp a better place:

2003 Season Staff Satisfaction Survey

Please circle the appropriate information:

1. How old are your campers?               7  8  9  10  11  12  13
2. Are they male or a female?              Male Female
3. Are you an international staff member?  Yes  No
4. What is your age?                       --
5. Are you a male or female?               Male  Female

Please give us your opinions in questions 6-8

Scale: 1 = not at all 2 = dislike 3 = neutral 4 = like
       5 = like very much

6. How satisfied are you with you camp experience?  1  2  3  4  5
7. How satisfied do you think your campers
   are with their camp experience?                  1  2  3  4  5
9. Would you recommend this camp to a friend
   for employment?                                  Yes  No

Of the following camp experiences, how satisfied are you?

programming                        1  2  3  4  5
camp administration                1  2  3  4  5
availability of supplies           1  2  3  4  5
camper behavior                    1  2  3  4  5
training and ongoing education     1  2  3  4  5
physical environment of the camp   1  2  3  4  5

In the box below, please tell us what you would most like changed
at camp and/or what you find most beneficial in your employment
experience?

2003 Season Parentr Satisfaction Survey

Please circle the appropriate information:

1. How old is your child?                  7  8  9  10  11  12  13
2. What is your child's gender?            Male Female
4. What encampment did he/she attend?

   First  Second  Third  Fourth

Please give us your opinions in questions 4-8

Scale: 1 = very dissatisfied 2 = dissatisfied 3 = neutral 4 = satisfied
       5 = very satisfied

4. How satisfied was your child with camp?                1  2  3  4  5
5. How satisfied are you with his/her camp experience?    1  2  3  4  5
6. How satisfied are you with the application process?    1  2  3  4  5
7. How satisfied are you with the camp staff?             1  2  3  4  5
8. Would you send your child back to camp?                Yes  No

Of the following activities, would you prefer more or less to occur
in camp?

boating                         less no change more
fishing                         less no change more
swimming                        less no change more
arts and crafts                 less no change more
nature hikes                    less no change more
overnight camping trips         less no change more
sports                          less no change more
cabin meetings/bunk meetings    less no change more

In the box below, please tell us what we are doing
especially well or hoe we can improve camp:

Available from the ACA Bookstore

* Coaching the Camp Coach by Michael Shelton MS., C.A.C., C.F.T

* Learning Leadership: How to Develop Outstanding Teen Leadership Training Programs at Camp by Michael Brandwein

To order, visit www.ACAcamps.org/bookstore.

References

Meehl, P.E. (1954). Clinical versus statistical prediction: A theoretical analysis and a review of the evidence. Minnesota: University of Minnesota Press.

Kahneman, D.; Fredrickson, B.L.; Schreiber, C.A.; and Redelmeier, D.A. (1993). When more pain is preferred to less: Adding a better end. Psychological Science, 4, 401-405.

Michael Shelton, M.S., C.A.C., C.F.T., is a consultant, trainer; and the director of Camp William Penn, a camp owned by the City of Philadelphia Department of Recreation. He is the author of Coaching 1he Camp Coach and the soon-to-be-published Secret Encounters: Addressing Sexual Behaviors in Camps.