Knowing our students: a prior learning assessment.

Abstract

Students in higher education are more diverse than ever before in history. Understanding students' backgrounds, experiences and attitudes early can help faculty better serve this diverse population. This is a case study that outlines the development and implementation of a web-based prior learning assessment. It also describes how faculty and administrators have used the results.

Introduction

The demographics of college students are changing. In the last forty years, a more diverse student body has emerged. Nearly 75% of high school graduates now attend some higher education institution within two years after the diploma (Education TrustWest, 2002). Older students make up one-third of the college population (Schneider & Stevenson, 1999) and 28% of the college population are ethnic minority students (National Center for Educational Statistics, 2002). Along with the changing demographics comes differing needs, skills, and desires (Association of American Colleges and Universities, 2002). The changing demographics and needs of students demand that faculty understand who their students are so that they can better teach them.

The importance of knowing one's students is supported by research on learning.

This research concludes that a student's prior knowledge and experience impacts their future learning (e.g. Bransford, et. al., 2000). With an understanding of what a student brings to the enterprise of learning, teachers can better assure that what they teach will be learned. Yet, teachers rarely ask what their students bring to the class. If they do ask, the questions usually pertain to students' mastery of an academic skill or content area and are gathered through the use of placement tests or a course pre-test (Angelo and Cross, 1993). Information on students' prior knowledge of a content area may not be enough in helping us understand our students. Research on student learning also indicates that demographic factors, such as students' commute time and hours worked, and attitudinal factors, such as one's reason for attending college, impact students' ability to be successful as much, if not more, than academic skill or knowledge (Astin, 1992). Knowing these things may help faculty better address their students' needs. In addition, research has shown that student learning is enhanced when students feel they are part of a learning community (Shapiro and Levine, 1999). Knowing who their peers are, what their experiences have been and what they believe can help them identify their shared experiences and differences and promote a better classroom environment and a better sense of self (Lenning and Ebbers, 2000). While an institution may collect demographic and attitudinal data on the institutional level through its institutional research department, this data may not easily accessed or presented in ways that can be easily used by teachers or students. In addition, such data tends to be global versus local and personal.

Given that understanding our students is important to their learning, it is imperative that institutions and teachers find a way to gather data that can be used by faculty and students. Fortunately, gathering such data is possible given recent technological advances. Web based surveys and databases can easily provide a quick and useable way to collect and disseminate information about students. This paper presents a case study of a web-based Prior Learning Survey conducted in all freshman level courses in a general education program and describes the process used for gathering data and disseminating the data on students enrolled in these courses. Examples of how the data is used by faculty, students, and administrators will also be discussed.

The Context

University Studies is Portland State University'sfour-year, interdisciplinary general education program. It explicitly seeks to be student-centered and utilizes active learning pedagogies. (For a more complete description of the program, see White (1994) or http://www.ous.pdx.edu.) The program is organized around four goals: 1) inquiry and critical thinking, 2) communication, including writing and quantitative literacy, 3) awareness of the diversity of human experience, and 4) ethical issues and social responsibility. Courses taken at all levels of the program are designed to enhance student skill development and learning in these areas. Year-long, three term Freshman Inquiry courses serve as the foundation of the program. These courses are organized around interdisciplinary themes and are taught by an interdisciplinary team of faculty. Examples of the themes include: The Cyborg Millennium, Forbidden Knowledge, and The Constructed Self. Students earn 5 credits each quarter totaling 15 credits for the year. The courses were designed to replace lower-division general education social science, science, and writing courses. All freshman students at PSU, except a small number of student enrolled in specialized programs like the Honors College, enroll in these courses (n=1466 in Fall 2004). The majority of PSU's students are commuter students.

Most Freshman Inquiry faculty made formal and informal inquiries of his or her students to get to know them individually. For example, some conducted their own course surveys; others met individually with students early in the term. In Fall 2002, faculty decided that a systematic and comprehensive tool might be more helpful. In addition, it was thought gathering data about the entire freshman class would provide a rich dataset for students themselves. Providing students access to some of the data would help create a sense of themselves as a freshman class and provide them with an opportunity to begin to develop quantitative literacy skills by analyzing data about themselves. The Prior Learning Survey has now been administered to all freshman inquiry students for the last two years. A description of the survey follows along with a description of how we gather, disseminate, and utilize the data.

Prior Learning Survey

The Prior Learning Survey was created by a group of faculty teaching in Freshman Inquiry who started with the question: what would be helpful to know about our students? Examples of other prior learning surveys were also considered in the development of this survey (Angelo and Cross, 1993) as well as questions that are asked on the institutional research department's Entering Student Survey (see http://www.oirp.pdx.edu for more information). A small committee developed the first iteration of the survey and it was then discussed and edited at an all faculty meeting. Student mentors for the program completed the survey and made further suggestions. The survey was designed to be a fluid document with faculty being able to change the questions each year based upon the needs of the faculty and the program.

The current Prior Learning Survey asks questions in three general areas: demographic, attitudes about and desires for college, and curricular experience. The demographic section asks questions such as age, gender, place of residence, what they were doing prior to enrolling at PSU, and college experience of their parents or guardians. The survey explores why students are in college and why they choose PSU. Questions about hours worked and other stressors are addressed as are concerns and fears about college. Much of the survey is devoted to questions regarding students' prior experiences in the development of the four curricular goals of University Studies. For example, students are asked about their experience writing research papers. This relates directly to the Communication and Inquiry and Critical Thinking goals. The survey also asks questions regarding their experience participating in activities with people that are different from themselves. This relates to the Diversity of Human Experience goal. Their responses to questions about service learning activities directly relate to the Ethics and Social Responsibility goal. The web-based survey was developed using WebSurveyor software (http://www.websurveyor.com). This proprietary software allows one to easily create an online survey. A variety of such products are currently available.

Method

All Freshman Inquiry courses include a mentor session run by undergraduate peer mentors. These sessions take place in a classroom with a computer available for each student. During the first week of class, students take the survey online during the mentor session. Since the survey is completed in a dedicated classroom during a dedicated class period and supervised by a student mentor, the response rate is nearly 99%. The survey takes approximately 15 minutes to complete. The data collected in each class is then sent to a backend database. Software is used to create graphs for each class and the whole dataset (the collective freshman class) and is updated daily for two weeks until all students have completed the questionnaire. This means that faculty and students are able to access the results for their class the following day. The results for each class and the collective freshman class are posted on a website accessible to faculty and students. The individual courses are not identified by name but by a number.

Results

After the surveys are completed by first year students, the faculty and mentor for the class and administrators have the opportunity to determine how they will use this data. To date, faculty, mentors and administrators have found several ways to use the data. The survey results have been used to: 1) inform the practice of faculty and mentors; 2) create a sense of community in the class and at the institution; and 3) facilitate students' skill development in quantitative reasoning. Examples of how the data have been used follows. These examples were derived from interviews with faculty and mentors in the program and administrators.

Improving Practice

Faculty and mentors have described that the results from the survey help them to improve their practice in the classroom. One faculty describes the significance of understanding the diversity present in his classroom. Recognizing that he had a contingent of older and more experienced students enabled him to capitalize on their experience and enlist them in helping set the tone in the classroom. It also allowed him to consider ways he could engage and challenge all his students. Prior to administering the Prior Learning Survey, he had assumed that most of his students would be arriving from high school. Without the input from the Prior Learning Survey, he would have missed the opportunity to engage these students early on in ways that were more appropriate for adult students.

The emphasis in a course can be influenced by the results of the survey. For example, one faculty was surprised to learn that the majority of her students worked over 20 hours per week and at the same time expected to get high grades in the class. She was concerned that they might not understand the demands of time that both working and being successful in school entail. She engaged her students in a discussion about the reality of an academic workload. To reinforce this and help students recognize and personalize the issue, the mentor for the class developed a lesson plan on time management. As a result, the mentor reported that students re-evaluated their balance of work and school life. Some decided they needed to drop hours at work or the number of credits they were taking. Many reported that they were glad to have learned early on and in such a personally relevant way the need for realistic time management.

The survey results have influenced the faculty's pedagogical practices. Several faculty discovered that their students had not read anything on their own beyond reading required for classes. One responded to this fact by decreasing the number of texts that her students read but increasing instruction and assignments on critical reading skills. Another has incorporated the use of more video clips as a way to engage students in critical analysis of ideas discussed in the course. The faculty member did not argue that video clips are a substitute for written text, but knowing this about her students encouraged her to not exclusively rely on text as the only avenue for engagement.

Creating Effective Learning Communities

Developing effective learning communities is an explicit goal of the Freshman Inquiry courses. Helping students learn to work with and learn from each other is considered to be an essential learning outcome for the program. Working collaboratively and in small groups is important for students' success in the course. The Prior Learning Survey facilitated the development of learning communities in the program. "Discussing the data about my students with my students has had an amazing bonding impact," says one faculty member. He and others report that the discussion of the data often focuses on the similarities and differences amongst the students in the class. Students don't feel as alone when they learn that most other students have not written a 10 page research paper either, given many oral presentations, or have concerns about finances. They are encouraged when they learn that their peers share similar ideas about why they are in college and that most of them aspire to get good grades in class. It is also encouraging to know that where one has weaknesses, others have strengths. As many of the assignments in Freshman Inquiry are group projects, the survey results create an opportunity to discuss how students can help each other in the acquisition of skills. One faculty used the data to construct a bar graph of experiences illustrating what his students brought to the class and used it to facilitate a discussion about utilizing talents in group assignments. It is also significant for students to be able to compare themselves and their class with the entire freshman class. One mentor recounted this comment from her class:

   They were blown away by the sheer numbers of people who shared
   their experiences and thoughts and feelings about being at PSU.
   Several of them talked about being a little embarrassed about
   why they were in college and how fearful they were. They had
   assumed that others thought differently and that they were the
   anomaly. It created a sense of community in the flash of a
   moment just knowing that others felt similarly.

At a commuter institution like PSU, the ability of students to see this snapshot of their peers can help students feel a connection to the institution. Unlike residential campuses, students at commuter campuses report feeling disconnected from the institution. This disconnection has been associated with lower retention rates (Jacoby, 2000).

Skill Development in Quantitative Reasoning

Faculty and mentors have utilized the data gathered from the Prior Learning Survey in creative ways to advance one of the University Studies' communication goals-quantitative reasoning. The survey was purposefully constructed to contain a variety of types of data. Discrete, categorical variables, such as age, and continuous, scaled variables, such as the extent of agreement with a statement, are measured. Qualitative data is also collected. Many faculty use the dataset to allow students to have an initial experience in analyzing data and making meaning of the analyses. Through assignments and discussions students are invited to get hands-on experience with real, relevant, and personal data.

One faculty member's experience illustrates the use of the data. He provided instruction on basic statistics and as a small group assignment asked students to use the data to summarize the data through visual representations constructed with Excel software. He reports:

   Inevitably students misrepresent how the data should be
   presented. Discrete variables are represented as continuous
   variables and vice versa. The point of the activity isn't
   necessarily for students to get it 'right' but for students
   to begin to see how charts and graphs don't always tell the
   story. They need to learn how to critically analyze data
   that is reported in the world around them.

Activities such as this enhance students' understanding of statistics through data that are personal and relevant and help develop a student's skills in this area.

Conclusion

This case study highlights the impact of the Prior Learning Survey used in PSU's general education program on faculty and students. Through the use of the survey, faculty and mentors learn early on about their students' backgrounds, experiences, attitudes, and feelings and in turn this knowledge allows them to respond appropriately and quickly to their students' needs. Students, in turn, feel supported by the faculty and mentors. In addition, they also identify with their peers early in the academic year through the use of the survey. The survey also provides a personally relevant starting point for learning and utilizing quantitative reasoning skills.

Given the need for faculty to understand their students at the beginning of a course and the relative ease of creating web-based surveys, institutions and faculty should explore the possibilities of developing similar surveys on their campuses.

References

Angelo, T.A. and Cross, K.P. (1993) Classroom Assessment Techniques: A Handbook for College Teachers, second edition. San Francisco,CA: Jossey-Bass.

Astin, A. (1993) What Matters in College?: Four Critical Years Revisited. San Francisco, CA: Jossey-Bass.

Bransford, J., Brown, A., Cocking, R., editors (2000) How People Learn: Brain, Mind, Experience, and School: Expanded Edition, Washington, D.C.: National Academy Press.

The Education Trust-West. (2002) The High School Diploma: Making It More Than an Empty Promise. Prepared for Senate Standing Committee on Education hearing on Senate Bill 1731, April 2002.

Jacoby, B. (editor) (2000) involving commuter students in learning. New Directions for Higher Education, no. 109, San Francisco, CA: Jossey-Bass.

Lenning, O.T. and Ebbers, L.H. (2000) The Powerful Potential of Learning Communities : Improving Education for the Future. San Francisco, CA: Jossey-Bass.

National Center for Education Statistics. (2002) Digest of Education Statistics, 2001. Chapter 3: Postsecondary Education. Washington, DC: U.S. Government Printing Office. Available at nces.ed.gov/pubs2002/digest2001/ch3.asp.

Schneider, Barbara and Stevenson, David (1999). The Ambitious Generation. New Haven and London: Yale University Press. p. 5.

Shapiro, N. & Levine, J. (1999) Creating Learning Communities : A Practical Guide to Winning Support, Organizing for Change, and implementing Programs. San Francisco, CA: Jossey-Bass

White, C.R. (1994), A model for comprehensive reform in general education: Portland State University. The Journal of General Education, 43 (3), 167-228.

Labissiere, Ph.D., is Assistant professor of University Studies and Reynolds, Ph. D., is Associate Professor of University Studies and Director of Mentor Programs.