A collaborative MAP for early interventions.

Abstract

As institutions of higher education create programs focused on student success, the links between collaboration and assessment become critical. This paper describes one project that uses collaboration and assessment to create early intervention opportunities with first year college students. It also illustrates the continued benefits of collaborative work.

Assessment and Collaboration

Collaboration in various assessment initiatives is being linked with a variety of positive outcomes including increased achievement, greater productivity, and the inclusion of multiple perspectives (Crossman, 2003; Katzenbach & Smith, 1993). Thus, it is not surprising that the value of collaboration is being discussed in numerous fields including public relations, library science, insurance, and family and consumer sciences (e.g., Buzzeo, 2002; Carter, 2003; Grunig, 2000; Kellet & Goldstein, 1999; Landberg, 2002).

Collaboration can be a powerful tool in higher education and assessment. The American Association for Higher Education (1993) linked involvement from "across the educational community" (p.770) with the potential for improvement. Palomba and Banta (1999) devoted an entire chapter in their book, Assessment Essentials, to the importance of involving various constituencies. Similarly, Terenzini (2001) suggested that collaboration is "the key to visible and credible assessment efforts" (p.11). Further, he outlined potential benefits of collaboration including the attention to various perspectives, increased credibility and assessment expertise, further discussion of assessment findings and implications, increased communication between constituencies, and increased awareness and respect among various campus units. Overall, collaboration and assessment seem to be strong allies, particularly if assessment is, as Palomba and Banta (1999) suggested, the "process that turns data into useful information" (p. 298).

This article describes one case study related to a strong collaborative initiative. It revolves around an assessment project with a long history of collaboration. The case study is used to explore the links between collaboration and assessment and to highlight the potential benefits of collaboration in a higher education setting.

Case Study--Making Achievement Possible (MAP)

In 1988, a collaborative team was formed at a large Midwestern university to develop a survey project aimed at helping students during their initial transition to college. The team included higher education professionals with expertise in academic advising, academic assessment, residence life, study skills, student learning, and first year student issues. The team spent more than a year carefully planning, developing, and implementing a survey project that has become a model of best practices in significant and effective collaborative assessment (Woosley, Whitaker & Knerr, 2003). To continue the collaborative nature of the project, an advisory committee was organized to oversee the project. The advisory committee currently includes members from Freshman Academic Advising, Academic Assessment and Institutional Research, Housing and Residence Life, and the University College Learning Center. Each year, the committee reviews the project including survey administration, reporting formats and procedures, and survey results. The committee subsequently makes recommendations for modifications in the project.

When designing the project, the development team chose the name Making Achievement Possible (MAP) because the project is structured, literally, to help make student achievement possible and to provide a "MAP" for early intervention with students. With this emphasis on creating early intervention opportunities, the project focuses on first-time matriculating freshmen during their third or fourth week on campus. Thus, students have had some time to experience campus life, but still have time to change habits or patterns that may cause or contribute to difficulties. The eight-page survey asks students to reflect upon and to assess their expectations, experiences, and goals. It includes questions about academic goals, plans for involvement in both academic and co-curricular activities, initial adjustment (academic, social, and personal), career goals, and personal skills and abilities. (See Authors' Note at the end of the article.)

The project involves extensive reporting that occurs in a timely manner. Administration of the survey occurs during the third or fourth week of a student's first semester at the institution. Because the development team was focused on providing useful information, they created a number of reports that are directed to the people who make decisions. First, an individualized student report was developed. This report is typically received within one week of completing the survey and provides feedback directly to each student based on the student's survey responses. The development team members used their expertise and various perspectives to create messages that beginning students may need to help them succeed. For instance, if a student indicates uncertainty about his or her skills or abilities in specific areas, then the student's report emphasizes the importance of seeking help in those areas and points the student toward campus resources that may be useful. Students are also given suggestions if they are not planning to study enough, if they are uncertain about their major, or if they are experiencing adjustment issues. Again, the purpose is to assist students in developing "healthy" expectations, to provide support to students experiencing difficulty, and to reinforce important campus messages. The reports use information from the previous year's survey to illustrate some of the messages. For example, students are told the percentage of students in the previous year who expected to receive A's and the percentage of students who actually achieved A's. The accompanying message is that many students overestimate the grades they will receive, and students may need to be aware of that as they plan study time and think about other academic behaviors. Each student report is approximately six to eight pages and is accompanied by a list of campus resources. Overall, the purpose of the student report is to provide appropriate feedback and resource information to students early in the semester when they can still make changes in their behavior.

Because the development team wanted to provide opportunities for early interventions, they also created individualized reports for students' advisors and hall directors. These professionals receive a one to two page confidential summary for each of the students assigned to them. These reports, along with the knowledge these professionals have about students, enable them to spot issues and provide support to students who need assistance. Research (e.g., Knerr, 2002) has shown that residence hall directors have used the reports in a variety of ways. For example, one hall director followed up with students who anticipated needing help to determine whether the students had, in fact, sought help or whether the hall director could provide assistance or referrals for the students. Another hall director reported using the MAP reports during mid-semester grade deficiency meetings and disciplinary/conduct meetings to focus on expectations, behaviors, and interests. The director said the information in the reports creates educational opportunities for those meetings.

Collaboration as a Success Mechanism

Previous authors have suggested that collaboration in higher education can lead to enhanced productivity, support, and improvement (e.g., AAHE, 1994; Palomba & Banta, 1999; Terenzini, 2001). The impact of the collaborative efforts of the MAP project is explored below.

Collaboration Leads to Continuation

The collaborative nature of the MAP project has helped to institutionalize it. The involvement of Housing and Residence Life (HRL) staff has been instrumental in the distribution and collection of the surveys. Thus, response rates are high, typically around 80 percent. The committee structure provides continued contact among the groups, even as personnel in particular areas have changed. Thus, the project which began more than 15 years ago continues on with support from all of the areas originally involved.

Collaboration Leads to Cooperation

The joint efforts between and among Academic Assessment and Institutional Research (AAIR), the office responsible for generating the reports, and both Academic Advising and Housing and Residence Life (HRL) have made the reports easier to use. For example, AAIR incorporated input from advisors to reorganize the advisor report and place the most referenced items on the front page. Similarly, a multi-method study investigating the use of MAP reports among residence hall staff (Knerr, 2002) raised issues that were addressed collaboratively. Overall, the Knerr study, which involved both a survey of residence life staff and follow-up focus groups, found that some staff members did not feel adequately trained to utilize the reports. As a result, HRL and AAIR made three changes. First, they developed a presentation to provide more information for HRL staff at the beginning of the MAP process. Second, the HRL Assessment Committee developed information sheets for new staff. Third, AAIR worked with HRL to develop a narrative report format that provides MAP summaries for each residence hall and suggestions on how to address the issues raised.

Collaboration Leads to Innovation

Further discussions between AAIR and HRL have led to the development of an internal web-based system called the MAP Freshman Intervention Tool (MAP-FIT). MAP-FIT uses MAP survey responses to generate numerous lists of students for residence hall staff For instance, MAP-FIT provides a hall director with the names of students who have indicated that they are having difficulty adjusting to college. Similarly, MAP-FIT provides a list of students who have expressed interest in leadership positions. Because these lists are generated within three weeks of survey completion, residence hall professionals can quickly determine which students may need attention or support or those who might like to be contacted when leadership opportunities arise within the hall or elsewhere on campus. In addition to simply improving the MAP project, the collaborations that occur have also led to an additional project. When reviewing the MAP project and issues related to transition to college, members of the MAP advisory committee discussed the many issues that confront transfer students. These conversations led to the development of a survey project called Making a Successful Transfer (MAST). The MAST project parallels the MAP project in survey content, processes, and reporting. MAST was implemented in Fall 2003, and preliminary results indicate that issues facing transfer students seem to be different than the issues facing first time students. Further research is in progress to identify these issues and subsequently refine the MAST initiative.

Collaboration Leads to Reflection

Campus interest in the MAP project is also encouraging a review of other campus survey projects. During the summer of 2004, a working team of departmental faculty advisors was convened to examine issues related to sophomore students. When reviewing other survey projects on campus, the team was overwhelmingly positive about the MAP project. Faculty advisors particularly liked the messages being sent to students, reports being sent to advisors, and information about students' expectations and experiences. As a result, they suggested that the university's existing sophomore survey project be expanded to include feedback mechanisms similar to those of the MAP project.

Discussion and Summary

Overall, the MAP project is a model in collaborative assessment. Previous researchers have characterized successful teams as those with a common mission, mutual respect for team members and the skills needed to accomplish the mission (Clark, 1998; Katzenbach & Smith, 1993). The MAP development team and the MAP advisory committee reflect those characteristics. The committee is focused on ensuring the success of first year students and using assessment tools to impact that success. The mutual respect shared by the committee members and the specific skills that members bring to the initiative are reflected in the successful administration of the project, the collaborative improvements and changes in the project, and the growth of new and related projects that have accompanied MAP over its 15 year history. The MAP project illustrates the benefits of linking collaboration and assessment to improve student learning and success in higher education.

Authors' Note Because the focus of this paper is on the collaborative process used in the MAP Project, rather than on the copyrighted survey itself, the MAP Survey is not included in the article. Readers who would like to receive a copy of the survey may contact either of the authors via E-mail at [e-mail] or [e-mail].

References

AAHE Assessment Forum (1993). Principles of good practice for assessing student learning. In J. S. Stark & A. Thomas (eds.), Assessment and Program Evaluation, Needham Heights, MA: Simon & Schuster Custom Publishing, p. 769-771.

Buzzeo, T. (2002). Collaborating to meet standards: Teacher/librarian partnerships for K-6. Worthington, OH: Linworth Publishing, Inc.

Carter, M. (2003). Walking the talk of collaboration: Ideas for training staff. Child Care Information Exchange, 150, p. 72-74.

Clark, D. J. (1998). Senior leaders reveal team truths. Journal for Quality and Participation, 21 (3), p. 16-21.

Crossman, J. M. (2003). Advantages and Assessment of Student Co-Authoring. Assessment Update, 15(4), p. 10-11, 15.

Grunig, J. E. (2000). Collectivism, collaboration and societal corporatism as core professional values in public relations. Journal of Public Relations Research, 12(1), p. 23-48.

Katzenbach, J. R., & Smith, D. K. (1993). The discipline of teams. Harvard Business Review, 71(2), p.111-120.

Kellet, C. E., & Goldstein, A. E. (1999). Transforming in the university and the community: The benefits and barriers of collaboration. Journal of Family and Consumer Sciences: From Research to Practice, 91(2) p. 31-35.

Knerr, A. (2002). The effectiveness of Making Achievement Possible (MAP) reports in aiding residence hall directors in hall decision-making. Unpublished master's thesis, Ball State University, Muncie, IN.

Landberg, S. (2002). The value of collaboration. Best's Review, 103(8), p. 107.

Pasearella, E., & Terenzini, P. (1992). Designing colleges for greater learning. Planning for Higher Education, 20, 1-6.

Terenzini, P. T. (2001). Collaboration: The key to visible and credible assessment efforts. In R. L. Swing (Ed.), Proving and Improving: Strategies for Assessing the First College Year, Columbia, SC: University of South Carolina, National Resource Center for the First-Year Experience & Students in Transition (p. 11-13).

Tinto, V. (1993). Leaving college: Rethinking the causes and cures of student attrition. Chicago and London: The University of Chicago Press.

Woosley, S. A. (2003). "How Important Are the First Few Weeks of College?: The Long Term Effects of Initial College Experiences." College Student Journal, 37(2), p. 201-207.

Woosley, S. A., Whitaker, D. R., & Knerr, A. R. (2003). "Making Achievement Possible, or a Unique MAP for College Freshmen." Assessment Update, 15(6), p. 1-2, 14-15.

Sherry A. Woosley, Ball State University

Donald R. Whitaker, Ball State University

Woosley, Ph.D. is Assistant Director of institutional Effectiveness and Whitaker, Ph.D. is Executive Director of Institutional Effectiveness and Professor of Mathematical Sciences