Case studies reveal camper growth.

Every year camp directors receive letters from campers' parents describing growth experiences that began at camp and have continued to impact their son's or daughter's life in the home and community. Counselors often note similar descriptions of growth in the same campers during program sessions. Most likely, though, camp administrators do not include these testimonials and descriptions of change in evaluations of their camp program. In other words, the data are available, are recognized as important, but probably aren't being effectively utilized.

In these times of increasing pressure to engage in program evaluation and demonstrate the benefits of camp programs, camp directors may well ask themselves: "How do I use these rich, personal experiences of growth as 'data' that will help confirm (or substantiate) the impact of our program?" "Aren't these 'data' valuable?" " Can't they count?"

Take a closer look at the importance of such rich information in your evaluation efforts, and look specifically at the potential use of case studies in providing qualitative data for evaluation efforts. Case studies of youth in two nationwide projects, the National Camp Evaluation Project (NCEP): 1993-96 and the National Inclusive Camp Practices (NICP): 1997-2000, have yielded positive results. These studies have incorporated qualitative data and comparative case analysis to explore the impact of camp programs on individuals with and without disabilities. This method can potentially be used as a component of overall program evaluation.

Importance of Case Study Research and Evaluation

Case studies are often used to describe the real-life context in which programs impact participants and to explore various effects in situations where there may be no clear, single set of outcomes for all participants. Data about the case are collected using multiple methods and multiple sources. When a number of cases are collected, then comparative case analysis can be used to summarize and analyze common and unique experiences and outcomes across individuals. Single and multi-case studies of this kind have been long-used by qualitative researchers.

Case studies provide the big picture

Qualitative data can be used to complement the quantitative data obtained from forced-choice surveys or rating scales that camps often use to evaluate their programs. Collecting qualitative data is time-consuming, and for most camp programs, it is rarely feasible to engage in individual case studies. However, when used as a complement to quantitative methods, collecting a small number of case studies can provide a context-rich, in-depth picture of how program experiences might be related to camper outcomes.

Case studies can help a camp program explore what underlies the quantitative data in terms of the individual's unique experiences that are recorded during or following a camp program. Case studies are also useful for discovering the specific ways a camp experience unfolded for a particular child and how that experience relates to the outcomes, both positive and negative, for the individual. Case studies can help camp program staff understand why a participant had particular outcomes in the context of the overall camp experience.

NCEP and NICP Dynamics

The NCEP was a three-year research project across fourteen states that focused on determining the personal/social and recreation skill dimensions of camper performance through use of both quantitative and qualitative measures. In addition to the objective testing, seventy-three case studies of campers were conducted in residential camp programs specializing in serving campers with disabilities. The NICP study, currently in progress, is being conducted within approximately sixteen states and is focusing on residential camp and outdoor school programs. Its primary focus is on inclusive or mainstream programs serving youth with and without disabilities, and similar to the NCEP, case studies are being used to determine camper/student growth in addition to the objective measures. In both studies, researchers have incorporate case studies to achieve a more comprehensive approach to measure growth and to validate the quantitative data regarding camper/student outcomes.

Method Used in the Case Studies

Although the case-study procedures differed somewhat between the NCEP and NICP research, both studies have used identical interviews of key respondents as a major case study procedure. In each case study, the subject's camp counselor was interviewed on the last day of the camp session, and the camper's parents were interviewed either in person or by phone generally one to three weeks after their child returned home from camp.

Using questionnaires with identical content, trained research assistants interviewed counselors and parents on the growth, if any, the camper or child made in eight developmental areas:

* social interactions

* communication with others

* responsibilities

* self-reliance

* self-esteem 

* participation in recreation

* skill achievement

* self-help 

Examples of positive changes for each of the eight developmental areas were provided to the informants (counselors and parents) during the interview. For a question on self-reliance, positive changes included:

* enjoys self during free time

* attempts tasks on own versus requesting counselor's help

* makes own choices for various activities

* is independent

* is resourceful 

In each developmental area, the counselor or parent was then asked if the participant's behavior had decreased, stayed the same, or improved. Counselors were asked to compare their camper's behavior since the first day of camp to the end of the camp session. Parents were asked to compare their child's behavior before and after the camp session. Whenever a counselor or parent said the participant's behavior had changed (either decreased or increased/improved), they were asked to provide a specific example of the change which was then recorded.

Case study analysis

After the case studies had been collected, respondents' judgements about change in the campers were analyzed, and the descriptions, or examples, of a change in behavior for these subjects were then grouped into categories or types of descriptions. Using procedures recommended by Mayhut and Morhouse (1994), two reviewers, working independently, read the descriptions and categorized them. Then the reviewers compared the categories they had found and, where there was disagreement, they discussed the description's meaning and decided together which category was the best fit for the description.

Final Results from NCEP Case Studies

An analysis of the counselors' and parents' judgement regarding camper growth (i.e., decreased, stayed the same, improved) indicated that the overwhelming majority of these campers improved or stayed the same in all eight areas of personal development. An analysis of all examples received from counselors and parents across the same areas revealed similar results. Overall, the vast majority of examples (97 percent of the descriptions recorded from counselors and 96 percent of the descriptions recorded from parents) described an improvement in camper performance associated with the camp experience. A summative analysis of all interview comments received from counselors and parents revealed that increased independence (self-reliance) was the predominant growth outcome for the case-study campers (see page 23).

Emerging Results from NICP Case Studies

Case-study interviews from the NICP are being based on NCEP case-study findings. In the NCEP research, counselors' and parents' judgements about change in their camper or child were also quantified so that a comparison could be made between the two groups of respondents. It was determined that counselors judged their campers to make the greatest growth in the areas of social interactions (69 percent) and communication (69 percent), while parents judged their children to make the greatest improvement in self-reliance (59 percent) and self-esteem (57 percent).

Based on these findings, the researchers selected the same four areas of development to highlight recent case-study findings from the NICP study. Representative case-study interviews that illustrate changes in camper growth as judged by each camper's counselor and parent are listed in the sidebar "Examples of Perceived Growth for Youth in Camp and Home Environments" on page 25.

Positive Outlook

In general, the NICP research includes both quantitative and qualitative data to achieve a more comprehensive evaluation of camper growth. As we enter the second year of the NICP study, counselor and parent perceptions from the emerging case-study data are exciting. Although preliminary, this qualitative data indicate positive benefits for youth with and without disabilities who participate together in inclusive camp or outdoor/environmental education programs. The following recent case-study data, provided by a parent of a child without a disability and parent whose child is disabled, indicate the significance of this approach and the impact these inclusionary camp programs are having on youth:

* "He talked about having a 'handicapped fellow' (Mom's own wording) in his cabin group. He is willing to talk to us now about how he is able to see that everyone has needs, disabilities (so to speak), and abilities."

* "There is no place she can be a typical child, except at this camp. It is always hard for her to relate to anything in her environment. Yet at camp, she is in an environment where she is a regular kid with a few extra differences, and different sides of her come out (e.g., like dancing at the campfire). When she arrives at camp, she gets out of the car and waves at her parents to leave (something she does nowhere else). She knows that she belongs while she is at camp. This camp is one of the most important things that happens to her all year round."

The Benefits of Case Studies

Case studies can be a strong component in program evaluation. A camp program could use the casestudy method to engage in a more indepth study of a small, randomly selected group of participants and gain a more qualitative, context-rich view of the impact of their program. Because the participants are randomly selected, the camp would obtain an unbiased cross-section of participants' experiences and outcomes.

If a camp asks its staff members and the parents of campers to complete a post-camp evaluation, the case study would be an excellent complement to these quantitative data, helping validate such findings and identifying additional camper outcomes. Adding the case studies to program evaluation might also shed important light on why certain program components and/or populations of participants have positive or negative outcomes. Objective evaluations from all participants and parents can still be used year by year to consider the quality of the whole program and its components, but including qualitative data from case studies would add to the richness of the results.

Qualitative Data vs. Quantitative Data

Quantitative data usually appear in numeric form, tend to focus on the outcome of the evaluation, and mainly deal with statistics and fact gathering. Typical data collection uses paper and pencil tests with preset answers or surveys with yes/no answers.

Qualitative data appear as worded descriptions and are usually discovery based, where evaluators try to understand reasons and causes. Participants respond to questions in their own words.

Analysis of Comments from NCEP Interviews

(N) = Number of counselor and parent comments regarding camper growth

Comments from Counselors (73 campers)

* Increased independence/skills (111)

* More motivated/enjoys participating (114)

* More confident/proud of self (37)

* Improved social interactions (74)

* Improved communication (5)

Comments from Parents (65 campers)

* Increased independence/skills (141)

* More motivated/enjoyed participation (55)

* More confident/proud of self (39)

* Improved social interactions (61)

* Improved communication (46)

Examples of Perceived Growth for Youth in Camp and Home Environments

At the end of the camp session, counselors provided examples of any improvements their campers had made in four areas of personal development since their first day at camp. Two to three weeks after the session, the parents provided examples of any improvements their child had made in the same areas of personal development since attending camp.

Area of Growth: Social Interactions

Camper: Boy, age 15, without a disability, summer camp

Example from Counselor: He was very quiet and reserved on the first day of camp. About the second day and on he began talking to more people. He ended up being willing to talk to about anyone.

Example from Parent: He seems less afraid now to make friends. He is also less timid. Recently, he was very comfortable around different people when our family went on a vacation.

Area of Growth: Self-Esteem

Camper: Boy, age 8, with autism, summer camp

Example from Counselor: He began to enjoy the activities more, and changed over the week from being withdrawn and sitting by himself on the sidelines (e.g., sat on nearby picnic table when boys played soccer together). By the end of camp he started trying new activities when asked and getting more involved.

Example from Parent: His confidence has improved. He wants to try harder levels on his Nintendo games now that he is a "big boy." He is reading the game hints over and over again, and wants to try them even though it says very few people can beat those levels.

Area of Growth: Communication

Camper: Girl, age 13, with moderate/severe mental retardation, summer camp

Example from Counselor: It took awhile for the girls to learn her signs and understand her and for her to feel comfortable around them. The girls improved in their signing, and she improved in her talking to them.

Example from Parent: She wants to talk a lot about camp. At camp, the girls in her cabin put together a photo journal for her. She continually wants to share it with the family and close friends.

Area of Growth: Self-Reliance

Camper: Girl, age 16, with mental retardation and a physical impairment, summer camp

Example from Counselor: She now notices what other girls are doing and tries to keep up with them. She doesn't sit and do nothing as she did earlier. She will sit down and read her book and always seems able to entertain herself.

Example from Parent: She is more willing to help; she now goes to the pantry and clears the table. It seems she has never been outside without a hand to hold onto; now she goes in and out of the house without anyone else. She gets a glass of water by herself (which she didn't before), and is just much more willing to do things on her own.

Steve Brannan, Ed.D., is professor emeritus in the School of Education at Portland State University, Portland, Oregon.

Ann Fullerton, Ph.D., is an associate professor in the School of Education at Portland State University.