Some critical factors in student learning.
The Spellings Commission has recommended that colleges and universities measure their effectiveness by focusing on student learning outcomes rather than the usual measurement of resources available to students (Fischer, 2006). In other words, measure the outputs rather than the inputs as the idea is often expressed. The Commission has not recommended national testing for a universal set of outcomes although it does seem to encourage the development of such tests for voluntary use. Whether the Department of Education will advocate for an extension of the No Child Left Behind approach to higher education remains to be seen.
Long before the commission report, however, regional accrediting bodies have been encouraging and setting standards for the measurement of student outcomes as a regular feature of their accrediting process. The standards of the Higher Learning Commission of the North Central Association are a case in point (2003). For years the national assessment movement has had at its core the measurement of student learning outcomes.
Ever since Ernest Boyer's seminal work, Scholarship Revisited (Boyer, 1990), there has been a growing interest in not only how student learning outcomes are measured but how these outcomes are impacted by the teaching and learning process. Although Boyer did not explicitly define the scholarship of teaching and learning, his work began the development of the new field we know today as the Scholarship of Teaching and Learning (SoTL). The goal of this scholarship is to discover how students learn most effectively and efficiently, and to discover the best teaching methods to enhance that learning.
This study is an exercise in SoTL research. The teaching and learning process is a complex one. A successful outcome depends on many variables involving both the student and the professor as visually portrayed in Figure 1. The study seeks to examine those teaching and learning behaviors of both students and faculty that contribute to successful learning for students.
[FIGURE 1 OMITTED]
REVIEW OF LITERATURE
There have been many studies that pursued the question of what is effective student learning and how teaching contributes to that learning. About a century ago, John Dewey (1913) recognized the importance of effort in the learning process. More recently, Arnold, Kuh, Vesper and Schuh (1991) studied the influence of student effort and college environment on student learning and personal development at six metropolitan institutions. In their review of the research on student learning, they conclude that student learning and personal development "are significantly affected by two sets of factors: (a) such institutional environmental characteristics as the quality of relationships between student peers and faculty, and (b) characteristics of student involvement or effort, such as time spent studying in the library or participating in educational programs in the union building or resident hall."
Ravoi and Barnum (2003) studied student perceptions of learning in online courses and found that the results depended on active or passive participation by students. For example, if students were engaged in postings on discussion boards on a weekly basis this behavior would be associated with a positive perception of learning. The Ravoi study focused on student behaviors that were different than the ones in this study.
The Community College Survey of Student Engagement (2008) highlights the importance of student effort in their learning but also points out the need for academic support. The behavior elements are not the same as in the present study. The Community College survey emphasizes such behaviors as discussing ideas outside of the classroom and working with other students.
Chambers (1991) points out the importance of giving students the freedom to manage their own efforts for an effective learning experience. Bonham (2007) reports on an evaluation instrument that included a measure of student engagement in a physics course. He found that the scale for student effort correlated well with student performance on examinations.
Needham (1978) develops a framework for showing how faculty control in a course affects student learning. This is an earlier investigation than Chambers into faculty control and student effort.
In one major study, John Centra and Noreen Gaubatz (2005) examined the items in the Educational Testing Services' Student Instructional Report (SIR II) that measured student perception of their learning experience. Their research focused on how student perception of their learning related to their perception of the effectiveness of the teaching. In this study, the focus is on students' perceptions of their learning behavior or effort, and their perceived learning experience.
Ronald Marks (2000) reviewed the studies of student evaluations and found five variables that were common to evaluations: organization, workload/difficulty, expected /fairness of grading, instructor liking/concern and perceived learning. The study did not examine the effect of student self-reported behavior on the perceived learning variable.
John Centra (1977) did a study that focused on nine areas of the original SIR student evaluation form, including student effort, and the results of the final examination in 72 sections of seven college courses. The results found a strong correlation between student effort and the mean results on the final examinations. The present study is in line with the Centra study in that the student perception of learning is correlated with different elements of student effort. However, the present study correlates individual elements of student effort rather than the complete area which offers more insight into student study behavior.
OBJECTIVE OF STUDY
The objective of the study is to examine the relationship between student effort and faculty instructional behavior to student learning. Successful student learning depends on many variables involving both students and faculty. The study seeks to examine some of the teaching and learning behaviors that contribute to student learning.
One of the most important variables in this process is the effort a student exerts in the process. It seems obvious that the more effort a student puts into a course, the more learning that should be achieved. This hypothesis is visualized in Figure 2. It is often referred to as indicating the student's level of engagement. Student engagement is broadly related to positive learning outcomes.
[FIGURE 2 OMITTED]
The present study is designed to test the hypothesis that positive learning outcomes are partially a function of student effort. This is done by looking at students self-reported data about how much they report they learn from their courses and how much effort they report they put into their courses.
Sample and Data Collection
The data for this study is taken from end-of-term student course evaluations. The form used is the Student Instructional Report II (SIR II) course evaluation questionnaire developed and processed by the Educational Testing Service. The data was gathered over a two year period (fall 2004 through spring 2006) and includes over twenty-five thousand responses.
The SIR II form asks students to evaluate the course and the instructor on a wide variety of dimensions, including the student's engagement with the course, course organization, faculty-student communication, exams and grading, and instructional methods. However, this study only looks at student responses to questions about course outcomes, student effort and involvement and certain faculty behaviors.
Specifically, the study investigates the relationship between student responses to three questions about student effort and involvement as independent variables and the question "my learning increased in this course" (question 29) as the dependent variable. The three independent variable questions are, "I studied and put effort into the course" (question 34), "I was prepared for each class" (question 35), and "I was challenged by this course" (question 36).
The responses to each question are on the following five point scale:
1--Much less than most courses
2--Less than most courses
3--About the same as others
4--More than most courses
5--Much more than most courses
It is hypothesized that the higher the ratings on questions 34, 35 and 36, the higher the ratings will be on question 29. The data is analyzed using the multiple regression statistical method.
Student Engagement and Student Learning
Analysis of the data using multiple regression shows a relatively strong relationship among the variables and that much of the variation in student learning (question 29) can be explained by the three student engagement variables (see Table 1).
Faculty Teaching Behaviors and Student Learning
Besides student effort there are teaching behaviors of the faculty that affect student learning. It is in these areas that faculty can work to improve the learning environment for students. In this research project four areas were found to have a positive impact on learning. They were: 1) the way courses were organized, 2) how the faculty communicated with students, 3) the level of interaction between faculty and students, and 4) the quality of the course assignments and examinations, including the faculty's grading practices.
In the area of course organization, the extent to which the faculty explained the course requirements and emphasized the important points of the course helped the student learn. Also, the faculty's command of the course subject matter and the faculty's use of class time had a positive effect on student learning outcomes.
In Table 2 below the statistics on the regression of these behaviors against the students learning experience are shown.
Student Learning and Faculty Communication
The area of faculty communication included clear class presentations by the faculty, especially if examples were used to explain the material. The use of foreign language speakers as faculty reduced the understanding of students. It was important for the faculty to pose challenging question for the students to aid their learning. And faculty enthusiasm as always is an effective teaching strategy.
In Table 3 below the coefficients for these faculty classroom behaviors are listed.
Student Learning and Faculty-Student Interaction
Student interaction with faculty has always been considered an important component in student engagement. Even in large classes faculty can foster engagement. The elements of interaction measured in the study included the extent to which the faculties were responsive to students, the extent to which they showed respect and concern for the students and whether they were available and were thought to listen to the students.
These elements are listed below in Table 4 with their relative coefficients.
Student Learning and Assignments, Exams and Grading
The helpfulness of the course assignments must be weighed by faculty to provide for effective student learning. It is in these assignments where the concepts are reinforced and clarified. Also, feedback on examinations proves to be an effective means for helping the students learn. The exams should also cover the material to be learned to be an effective learning tool. Information on grading policies helps students organize their approach to learning in the course.
In Table 5 below the coefficients for this faculty behavioral elements are listed.
LIMITATION OF THIS STUDY
The research was based on student self-reported data reflecting their perception of learning and did not assess the relationship of the self-reported behaviors to real measures of learning. However, the study provides some insight into the factors that are important in the learning process. Further research should focus on the relationship between student efforts and involvement, student perceptions of learning, and actual measures of student learning.
The study highlights some important practices that faculty can adopt that will enhance student learning. Faculty must give attention to course organization; provide clear class presentations and challenging questions. They must also be responsive to students and listen to them. All of these behaviors create a supportive and encouraging learning environment. While it is intuitive that to be successful learners , students must study, exert effort and be prepared for class, this effort in many ways is dependent on faculty challenging the students and letting them know what kind of effort is expected, how much is enough and how much is not enough.
The effort to understand the most effective learning environments for students is a continuing one. This study was one such attempt. While it was limited to a study of the relationship between student perceptions of learning and certain student and faculty behaviors, it offers some insight into the learning process. Students have knowledge of what facilitates their learning. Although this varies with students, the study has identified some common ones that will aid faculty in the teaching and learning process.
Arnold, J., Kuh, G., Vesper, N., & Schuh, J. (1991). The Influence of Student Effort, College Environments and Selected Student Characteristics on Undergraduate Student Learning and Personal Development at Metropolitan Institutions. Paper presented at the Annual Meeting of the Association for the Study of Higher Education, Boston, MA, November, 1991, p.3.
Bonham, S. (2007 ). Measuring Student Effort and Engagement in an Introductory Physics Course. Physics Education Research Conference, Greensboro, NC, 951, 57-60.
Boyer, E. (1990). Scholarship Revisited: Priorities of the Professoriate, Jossey Bass, San Francisco.
Centra, J. (1977) Student Ratings of Instruction and Their Relationship to Student Learning, American Educational Research Journal, 14, 17.
Centra, J., & Gaubatz, N.B. (2005) Student Perceptions of Learning and Instructional Effectiveness in College Courses. A Validity Study of SIR II. Princeton, N.J. Educational Testing Service.
Chambers, D. (1991) The Effects of Faculty Control of Learning on Student Effort. Evaluation & the Health Professions, 14, 438-455.
Community College Survey of Student Engagement (CCSSE). (2008). Essential Elements of Engagement: High Expectations and High Support. Austin, Texas: The University of Texas at Austin, Community College Leadership Program.
Dewey, J. (1913). Interest and Effort in Education. New York: Houghton Mifflin.
Fischer, K. (2006, September 1). The Spellings report: lengthy fights are expected over student learning and effective measures on accountability. The Chronicle of Higher Education, A42.
Higher Learning Commission. (2003). Student learning and effective teaching. Handbook of Accreditation, 3rd Ed. Chicago.
Marks, R. (2000). Determinants of Student Evaluations of Global Measures of Instructor and Course Value. Journal of Marketing Education, 22 (2):108.
Needham, D. (1978) Student Effort, Learning, and Course Evaluation. Journal of Economic Education, 10 (1): 35-43.
Ravoi, A.P., & Barnum, K.T. (2003) Online course Effectiveness: An Analysis of Student Interactions and Perceptions of Learning. Journal of Distance Education, 18
About the Authors:
Norman E. Carroll is University Professor of Business and Economics in the Brennan School of Business at Dominican University. He is also Provost Emeritus of Dominican University. His research interest is in the area of organizational structure and change. He is currently focused on studying the entrepreneurial dimension in organizational effectiveness.
Michael O'Donnell is the Director of Institutional Research and Assessment at Dominican University. He is a former member of the faculty at Boston University and Pacific Lutheran University, and has been working in institutional research since 1990
Table 1 Summary of Regression Results Regression Statistics: * Multiple Correlation Coefficient (R) .623 * R Square .388 * Adjusted R Square .388 * Standard Error of the Estimate .793 Coefficients *: * Question 34 "studied" .235 * Question 35 "prepared' .143 * Question 36 "challenged" .327 * All coefficients are significant at p < .05 or greater. Table 2 Student Learning and Course Organization Standard Faculty Behavior Coefficients * [R.sup.2] Error Explanation of Requirements .176 Institution's Preparation .055 Command of Subject .159 Use of Class Time .185 Emphasizing Important .225 Points Multiple Regression Stats R = .633 .401 .785 * All coefficients are significant at p < .01 or greater. Table 3 Student Learning and Faculty Communication Standard Faculty Behavior Coefficients * [R.sup.2] Error Clear Presentations .271 Command of Spoken English -.045 Use of Examples .112 Challenging Questions .308 Enthusiasm .123 Multiple Regression Stats R = .623 .388 .791 * All coefficients are significant at p < .01 or greater. Table 4 Student Learning and Faculty-Student Interaction Standard Faculty Behavior Coefficients [R.sup.2] Error Responsive .257 Respect for Students * -.004 Concern for Students .181 Available .097 Listens .121 Multiple Regression Stats R = .539 .291 .851 * Not significant, all other coefficients are significant at p < .01 or greater. Table 5 Student Learning and Assignments, Exams, and Grading Standard Faculty Behavior Coefficients [R.sup.2] Error Grading Policy Information .103 Clarity of Exams * .009 Exam Coverage .148 Faculty Comments on Exams .109 Quality of Textbook .095 Helpfulness of Assignments .244 Multiple Regression Stats R = .625 .390 .779
|Printer friendly Cite/link Email Feedback|
|Author:||Carroll, Norman E.; O'Donnell, Michael|
|Publication:||International Journal of Education Research (IJER)|
|Date:||Jan 1, 2010|
|Previous Article:||Preparing for uncertainty? Paradoxical challenges for management education.|
|Next Article:||Revisiting gender bias in doctoral programs in economics.|