Do you know what your outcomes are? The impact of Oregon 4-H residential camp programs on positive youth development.

As camp youth development professionals, we all try to keep up with the camp program research base. Research reported in these studies is vital in planning for the inputs required to reach targeted outcomes of our own camp programs.

**********

We know that camp is comprised of three interrelated attributes: the outdoors, education, and community (Slatter 1984). These attributes combine to create the camp experience. Research on positive youth development emphasizes the importance of a positive, inclusive atmosphere in youth programming (Roth & Brooks-Gunn 2003). When youth participate in programs, they need to feel included and comfortable in order to benefit from the program. At 4-H camp the adult leaders, the camp counselors, who are often older 4-H teens, and the friendliness of other campers combine to play an instrumental role in ensuring a fun and inclusive camp community where youth can flourish.

While environmental education may not be what most people think of when 4-H is mentioned, developing environmental stewards is a targeted content area for 4-H programs nationwide; a great deal of this target is met through resident camp programs. Campers learn about nature and the outdoors, often returning from camp with a wider awareness and appreciation of the natural world (Smith 2001).

In addition to providing a caring community and content knowledge, all 4-H programs are designed to enhance the development of important life skills in youth (Hendricks 1996). There are thirty-five identified life skills from which 4-H staff can select in designing 4-H programs; they are divided into the four quadrants of Head, Heart, Hands, and Health.

In Oregon, we were convinced that 4-H camps were delivering quality programs as the camp literature base suggests. But could we rely on research at other sites to assure that our youth were reaching the outcomes we designed and desired for them? We knew we were doing lots of things well, but were there things we could be doing better? To explore developing a statewide, multiple-site camp evaluation that would document outcomes at Oregon's 4-H camps, state specialist staff presented a seminar at the 2003 spring 4-H staff development conference.

Methods and Procedures

Because one of the most important aspects of conducting program evaluations at multiple sites is buy-in from the site directors (Arnold 2003; Garst & Bruce 2003), the questionnaire to be used was developed with significant input from local camp directors--who were primarily 4-H agents in Oregon. This process began at the staff development conference where we invited participating agent directors to articulate the impact of their camp program on youth.

This session was a critical step in preparing for a meaningful evaluation. We asked the question this way, "How will your campers be different on Saturday when their parents pick them up from when they were dropped off on Sunday?" While each of the thirteen Oregon 4-H camps offer different programs at a variety of sites, the camps shared some common desired outcomes for personal growth, life skill development, and camper satisfaction, which were identified by the agent directors.

Information from this session was drafted into an initial two-page questionnaire. The questionnaire was then reviewed by three 4-H agents with significant camp expertise, as well as by a local 4-H county camp advisory council. Following the reviews, the questionnaire was finalized for use in the study.

The evaluation was conducted using a multi-site evaluation methodology, with each of the thirteen participating camps administering the same questionnaire and following the same evaluation protocol (Arnold 2003; Garst & Bruce 2003). The questionnaire included basic demographic information, eight questions related to the camper's perceived life skill development (Hendricks 1996); six retrospective pre/post questions related to growth during camp (Pratt, McGuigan, & Katzev 2000); and a set of four questions related to the camper's satisfaction with camp.

Each site agreed to designate time at the end of camp during which campers were asked to complete the questionnaire about their camp experience. The completed questionnaires were sent to the state 4-H office for data entry and analysis. By the end of the summer, each camp received a report of the results for its own camp from the state office. The data were then aggregated for use in this statewide analysis--a strategy that allows a more robust understanding of the impact of similar programs that are held at different sites (Arnold 2003; Straw & Herrell 2002).

Eight hundred forty-nine youth campers, entering grades four through nine, completed the questionnaire (100 percent response rate). Five hundred fifty-nine of the campers were girls (66 percent) and 238 were boys (34 percent). Three hundred campers (35 percent) came from urban communities. Five hundred forty-nine (65 percent) came from rural communities. Sixty-one percent (518) of the campers were 4-H members. This was the first time attending 4-H camp for 55 percent (467) of the campers.

Results

Opportunities for Personal Growth

To measure personal growth, campers were asked how they felt about six items related to the camp experience. For example, campers were asked about being away from home, managing their free time, living in nature, and doing skits or presentations in front of others. Using a retrospective pretest and a 1 to 4 scale, campers rated each item based on how they felt before camp and then after camp. A rating of 1 indicated the statement was "not true" and a rating of 4 indicated the statement was "very true." Before-and-after ratings were analyzed using a paired t-test. The analysis revealed a significant, positive change for each item (p < .05). Using Cronbach's Alpha the internal reliabilities for the personal growth scale were calculated at a =.56 for "before" and a =.61 for "after." Table 1 (see page 48) presents the pre- and post-camp mean scores and the results of the paired t-test analysis.

Life Skill Development

Campers were asked how much 4-H camp helped them to develop life skills selected from the Targeting Life Skills Model (Hendricks 1996). Campers rated each of eight skills on a 1 to 4 scale, with a rating of 1 indicating that camp contributed nothing to the development of that skill and a 4 indicating that camp contributed "a lot!" (Cronbach's Alpha =.89). Mean ratings ranged from 3.10 (working through disagreements) to 3.47 (learning new things I like to do). Table 2 (see page 48) presents the score range and mean ratings for each of the life skills.

Camper Satisfaction with Camp

Finally, in an effort to determine the extent of a positive atmosphere at camp, campers were asked about their counselors, friends, and whether camp was fun. Campers were asked to rate each of 4 items on a 1 to 4 scale, with 1 indicating "not true" and 4 indicating "very true" (a = 73). Mean ratings for camp satisfaction ranged from 3.32 (camp was one of the most fun things I have done) to 3.80 (I liked my camp counselors). Table 3 presents the score range and mean ratings for each item.

Conclusion

The results of the Oregon 4-H Residential Camp evaluation adds to the growing body of research in camp-based youth development suggesting that summer camps play an important role in the development of young campers. The results show that camp provides an opportunity for youth to grow socially, to develop important life skills, and experience nature, all in a fun, hands-on setting. At the end of camp, campers reported that camp helped them to feel good about themselves, learn new things, to make friends, and work together in a spirit of cooperation. Perhaps most importantly, the natural setting of camp allowed campers to enjoy learning about the natural world. This information, gleaned specifically at Oregon 4-H Camps, will be valuable to agents in communicating the values of 4-H camp to parents, grant funders, and stakeholders.

Generic "one-size-fits-all" evaluation instruments may seem like the answer to a camp director's dream. However, the commonalities in generic instruments may become so broad as to lose any ability to inform a camp's own on-site youth development practice. An evaluation, based on the camp's targeted outcomes, should identify not only what the camp is doing "right," but where there is room for improvement.

The Youth Development Outcomes of the Camp Experience Study found "that religiously affiliated camps appear most likely of the sponsoring groups to see statistically significant positive change in their most important outcome, which in most cases, was spiritual growth (Henderson & Scanlin 2004)." For programs that have identified this as an important outcome, this is the expected result. Similarly, we should not be surprised that camps that do not target environmental attitudes as an outcome showed "no growth" in this area in the study.

Not only should a camp integrate its most important outcomes into everyday camp life, these outcomes should be reflected on the camp's evaluation. To document the delivery of quality youth development programs, camps must intentionally identify outcomes that can be measured with an evaluation. Do you know what your outcomes are?

References

Arnold, M. E. (2003, December). Using Multi-Site Methodology to Evaluate 4-H Youth Leadership Retreats. Journal of Extension, 41(6). Retrieved October 27, 2004, from www.joe.org/joe/2003december/rb1.shtml.

Garst, B. A. and Bruce, F. A. (2003). Identifying 4-H Camping Outcomes using a Standardized Evaluation Process across multiple 4-H educational centers. Journal of Extension, 41(3). Retrieved November 22, 2004 from www.joe.org/joe/2003june/rb2.shtml.

Henderson, K. and Scanlin, M. (2004). Information is Power: A look at the latest data and emerging trends in youth development and the camp industry. Camping Magazine, September-October.

Hendricks, P. A. (1996). Developing youth curriculum using the Targeting Life Skills model: Incorporating developmentally appropriate learning opportunities to assess impact of life skill development. Ames, IA: Iowa State University Extension.

Pratt, C. C., McGuigan, W. M., & Katzev, A. R. (2000). Measuring program outcomes: Using retrospective methodology. The American Journal of Evaluation, 21(3), 341-349.

Roth, J. L. & Brooks-Gunn, J. (2003). What exactly is a youth development program? Answers from research and practice. Applied Developmental Science, 7, 94-111.

Powell, G. M. (2003). What happens to campers at camp? The social, health, and psychological impact on children attending camp. Camping Magazine, September-October.

Slatter, T. (1984). The temporary community. Albatross Books, NSW. Australia.

Smith, P. L. (2001). A view from the woods: Camping as character-building experience for children and youth. Camping Magazine September/October.

Straw, R. B., & Herrell, J. M. (2002). A framework for understanding and improving multi-site evaluations. New Directions for Evaluation, 94, 5-15.

Acknowledgements

This research was possible because of the contributions of many people. Specifically, we would like to thank:

Tammy Skubinna and Robin Van Winkle. 4-H Youth Development County agents; and the Clackamas County 4-H Camp Advisory group for their thoughtful review and critique of the evaluation instrument.

Jana Mienhold, M.S., (ABD), doctoral candidate in Human Development and Family Sciences at Oregon State University entered the data and conducted the camp-by-camp analyses. She also prepared reports that were sent to each camp.

County 4-H Youth Development faculty members at each camp site were responsible for data collection. A special thanks is extended to each person who helped make the data collection go so smoothly and effectively.

Finally, a big thank you to the 849 campers who took the time to complete the questionnaire at the end of a very busy and fun-filled week at camp!

All photos courtesy of Janet Nagele, Clackamas County Oregon 4-H Camp.

Janet Nagele is a 4-H youth development agent with the Oregon State University Extension Service. She is responsible for the Environmental Education and Latino Outreach initiatives in Clackamas County and serves on the state 4-H Natural Science committee. Nagele has worked in the youth development profession for eighteen years, developing and administering camp programs, natural science education, club programs, and after-school, oudoor education. Her e-mail is [e-mail].

Virginia D. Bourdeau is an extension specialist with the Oregon State University Department of 4-H Youth Development. She has authored and co-authored five 4-H leader publications. Bourdeau has worked in the camp profession for twenty-seven years.

Mary Olszewski Arnold holds a Ph.D. in Adolescent Development and is a program planning and evaluation specialist with the Oregon 4-H program. Her work centers on helping 4-H youth educators articulate and measure the impacts of 4-H educational programs.

A version of this article including an analysis of gender differences has been accepted for publication by and will be published in an upcoming issue of the Journal of Extension (JOE) <www.joe.org>.JOE has granted permission for publication of the present article.

Janet Nagele, Virginia D. Bourdeau, and Mary Olszewski Arnold, Ph.D.

Table 1 Camper Skills Before and After Camp

                                        Before  After  Mean
                                   N    Camp    Camp   Difference  SEM

Being away from home               844  3.58    3.68   0.10        0.02
Knowledge about nature             839  2.79    3.30   0.51        0.03
Presentations in front of others   838  3.20    3.47   0.27        0.02
Managing free time                 841  3.29    3.54   0.25        0.02
Liking nature and the outdoors     840  3.54    3.64   0.10        0.02
Meeting new people and friends     842  3.66    3.79   0.13        0.02

                                   t       df   Sig.***

Being away from home                -4.53  843  0.00
Knowledge about nature             -19.66  838  0.00
Presentations in front of others   -11.12  837  0.00
Managing free time                 -10.95  840  0.00
Liking nature and the outdoors      -6.38  839  0.00
Meeting new people and friends      -8.03  841  0.00

a = .56 "before"; .61 "after"
*** Statistically significant differences for all items

Table 2 Camper Ratings of Life Skill Development

                                       N    Min.  Max.  Mean  SD

To learn new things that I like to do  841   1     4    3.47  0.77
To make me want to try new things      840   1     4    3.42  0.81
To feel good about myself              844   1     4    3.41  0.82
To be responsible                      841   1     4    3.35  0.82
To cooperate with others               836   1     4    3.35  0.80
To talk to others more easily          828   1     4    3.32  0.87
To work with others as a team          846   1     4    3.28  0.77
To work through disagreements          841   1     4    3.10  0.92

a = .89

None       23
A little   98
Some      346
Lots!     379

To work with others as a team

Note: Table made from bar graph.

Table 3 Camper Ratings of Camp

                                     N    Min.  Max.  Mean  SD

I liked my camp counselor(s)         839   1     4    3.80  0.56
I made new friends at 4-H camp       836   1     4    3.71  0.59
I am going to share about camp with  835   1     4    3.54  0.79
  friends who were not here
I want to invite others to 4-H camp  834   1     4    3.45  0.83
4-H camp was one of the most fun     839   1     4    3.32  0.86
  things I have ever done

a = .73