Tutorless PBL groups in a medical school.

Abstract

Problem-based learning (PBL) has become a popular teaching method in medical schools because of its emphasis on developing problem solving skills as well as delivering course content. Typically PBL depends on the availability of significant numbers of faculty to function as small group "tutors" and is therefore very resource intensive. This study compared achievement of content knowledge and student satisfaction in tutorless and physician facilitated small groups in a 2nd year medical school course, and found no significant difference in these areas between the two groups. The one significant difference found was that students in groups with tutors worked longer than those without tutors.

Introduction

Problem-based learning (PBL) was introduced into medical education in the 1960's at McMaster Medical School in Ontario, Canada. For years, there had been concerns by medical school professors about the overuse of lectures. It was believed that students were too passive and that the lecture method was ineffective. Studies have shown that medical students forget much of what they have memorized from lectures before they reach their clinical years (Woods, 1993). Woods also found another complaint to be that medical students were not being trained as critical thinkers or problem solvers and that they were unable to apply their knowledge in a clinical setting. Proponents of PBL theorize that students learn best when learning in context (Schmidt, 1983). PBL provides students with an opportunity to experience the process of patient care and decision making without putting any actual patients at risk. PBL is also believed to promote life-long learning and to mirror "real-life" use of resources (Schmidt, 1983).

Studies done comparing problem-based learning with traditional lecture curricula have found several positive trends with PBL. For example, in one study PBL graduates had similar, and sometimes better, performance on clinical examinations and faculty evaluations. They also were found to have board scores similar to those of traditional lecture students (Albanese & Mitchell, 1993; Norman & Schmidt, 2000). Learning appears to be better retained by PBL students as judged by faculty (Albanese & Mitchell, 1993) and PBL students have better problem-solving and information recall in clinical years (Norman & Schmidt, 2000; Vernon & Blake, 1993). Compared with lecture based instruction, students tend to report increased satisfaction with PBL. For instance, students considered the problem-based learning method to be more nurturing and enjoyable (Albanese & Mitchell, 1993; Norman & Schmidt, 2000; Vernon & Blake, 1993).

Oklahoma State University Center for Health Sciences (OSU-CHS) Tulsa, Oklahoma has used this method of instruction in the Clinical Problem Solving course for 2nd year medical students since the early 1990's. Specifically, the students take a hybrid course with four hours of traditional lecture per week combined with four hours of small group work. The groups are composed of 6-9 students and one physician/tutor, also referred to as a "facilitator". The groups work through carefully structured pre-designed cases to learn content while also developing problem-solving skills. Although this has been an effective format that is very popular with the students, there are several problems with the tutored groups:

* Difficulty recruiting faculty due to time constraints. Theoretically, a non-physician tutor could be used, but studies have shown that there is less student satisfaction with this arrangement (Dolmans, Gijselaers, Moust, deGrave, Wolfhagen, & van der Vleuten, 2002).

* Difficulty recruiting physician tutors from the community (non-faculty members), due to time and financial considerations. Small groups meet two mornings a week during prime office hours for most physicians.

* Expense to Family Medicine Department to hire non-faculty tutors.

* Difficulty enforcing uniformity between groups.

* Disparity among physician facilitator abilities; some facilitators are extremely popular, some get complaints concerning poor use of time, inadequate understanding of the material themselves, getting off track too easily, and/or inability to foster a positive learning environment.

* Occasional absence of tutors leaves groups unavoidably on their own.

If some cases could be done without tutors OSU-CHS could reduce the resource strain encountered with faculty-tutored small groups. We wanted to determine whether or not students learn the material as well when they work in groups without a physician/tutor, and if they found tutorless groups efficient and enjoyable.

Tutorless Small Groups

Non-medical school programs have successfully adopted tutorless groups. In these cases, students receive significant training in small group process, similar to what facilitators get in tutored small group process. There is little literature, however, on using tutorless groups in medical school.

At the UCLA School of Medicine, tutorless groups have been introduced to reduce demand on faculty time. Neville (1999) found the tutorless groups identified fewer learning issues compared with tutored groups. At Helsinki University of Technology, interviews were conducted with students regarding their experiences in tutorless groups vs. tutored groups. Students indicated that they felt a tutor served as a safety net against incorrect knowledge and reasoning. They felt that if they were stuck on material, the tutor could point them in the right direction. They also stated that the tutor added pressure to the students to work harder in the research phase of small groups. Students in the tutorless groups indicated that they were often frustrated, because they were uncertain if information being presented was correct. They occasionally digressed into irrelevant conversation. Working styles among tutorless groups were more varied than in tutored groups. There was occasional conflict due to all students not doing equal shares of work, with negatively affected attendance and trust. Also, it was more difficult to establish common work goals and standards among students in tutorless groups (Nuutila et al., 2005).

McMaster University has had experience with both tutored and tutorless PBL small groups (Woods, Hall, Eyles, Hrymak, & Duncan-Hewitt, 1996). In their chemical engineering program, they use five to ten tutorless groups with five students per group. The students receive about 50 hours of workshop style training in the processing skills. The students are required to submit journal reports frequently that indicate their progress and activities within the PBL tutorless groups. In the tutorless groups, the group's objectives are taken to the instructor who checks that all the issues have been identified. Conclusions from this institution were that there are very different problems encountered and priorities set in tutored versus tutorless groups. Attendance and students that did not do their fair share are primary complaints in tutorless groups. Their data suggest that these are not problems in tutored groups. In tutorless groups, it is important to train and empower the students to solve processing problems, just as tutors are trained to do this in tutored groups. The difference between tutored and tutorless groups is in who learns and applies the procedures. In tutorless groups, the emphasis must be on training the students (Woods et al., 1996).

As a quality assurance review of the CPS course at OSU CHS, in the fall semester of 2005, the Family Medicine faculty developed a radiology case to be completed by 2nd year medical students (MS IIs) during their regular two-hour small group sessions. The students were randomly assigned to work as usual with their facilitator or to work in their group without a facilitator. After the small group session, the students all took a 10 question multiple choice quiz and completed a four question satisfaction survey which asked students how helpful, efficient, interesting and enjoyable they perceived the format to be. The quiz proctor also monitored the time the students arrived to take the quiz to estimate the time they spent working on the case. Evaluation of the data revealed no significant difference in quiz scores. However, analysis of the satisfaction survey revealed that the students with tutors rated the format more efficient than the students without tutors (p>.05). There was no statistically significant difference regarding the other three questions. Additionally, the groups without a facilitator ended their small group sessions an average of 20 minutes earlier than groups with a facilitator.

Hypothesis

Based on what we know from our quality assurance review and from limited studies using tutorless problem-based learning in engineering and medical schools, we hypothesized that students would have similar achievement of content knowledge in tutorless groups and in faculty-tutored groups. We also hypothesized that there might be differences in student satisfaction between the two groups, knowing that decreased satisfaction in some areas of tutorless groups has been found in previous studies.

Research Design

Participants. MS II students at OSU-CHS who were enrolled in the Clinical Problem Solving course during the spring, 2006 semester participated in this study. There were 84 students, 47 male and 37 female. The students were already assigned to small groups and half of these groups were randomly selected to complete the case without tutors. The other half of the groups completed the case in the traditional faculty-facilitated small group setting.

Materials. A dermatology case was used in this study. The case consisted of 20 "mini-cases" in which the students were presented with a clinical scenario and a slide (a picture of a patient). They were then asked a series of multiple choice questions pertaining to the case, with the answers available on the following slide.

Measurement scales. The student's mastery of the material was measured by a l0 question multiple choice quiz covering that day's session. The student's satisfaction with the method (tutorless vs. tutored) was measured by a 4 item questionnaire scored on a 7-point Likert scale. The questions asked were: l) How helpful was this format in learning the material? 2) How efficient was this format? 3) How interesting was the material? 4) How enjoyable was this format? We also recorded the length of time each group spent working on the case.

Procedure and timeline. A two-hour small group dermatology case was presented during the Spring 2006 academic semester. The students had been informed that they were to report to the lecture hall immediately after completing the case in the small group "break out" rooms. They were at this time given the quiz and satisfaction survey. Approximately one week after the dermatology case and quiz, the students were debriefed about the study. They were also informed that they would not be identified in the data. The students all received a 100% on the quiz for the purposes of their final grade, so that there was no penalty to them for being part of this research project. This was recommended by the IRB, and the study was given exempt status.

Treatment group (predictor). This consisted of the groups working on the case in small groups without a tutor present in the room with them.

Control group. The control groups consisted of the students in the small groups with the traditional tutor present. The method used was the same as typically used throughout the course in small groups.

Data Analysis

The data compiled from the multiple choice quizzes and the satisfaction surveys were input into SPSS, a statistical analysis package and analyzed using Independent Samples t-test. Specifically, the groups were compared to determine the effect of the facilitator on the quiz scores. The quiz scores served as the outcome or dependent variable while the type of group (facilitator or non-facilitator) served as the predictor or independent variable. Additionally, the Likert scores for the items rating satisfaction were analyzed to determine if a significant difference existed regarding the type of instruction the students received.

Results

All 84 students in the class took the quiz following the dermatology small group session. Thirty-nine had a facilitator, and forty-five did not. There was no statistically significant difference in quiz scores between groups (with tutor = 91.6%; without tutor = 89.8%; p > .05). The satisfaction survey was completed by 34 students who had worked with a tutor, and by 38 students without a tutor. There was no statistically significant difference between groups on any of the questions on the survey (p > .05). Students in groups with a tutor stayed in their sessions an average of 13 minutes longer than the groups without a tutor, which was statistically significant (p = .023).

Discussion

A comparison between tutored and tutorless PBL small groups found no significant difference in content mastery or student satisfaction. Although we had hypothesized that there would be greater student satisfaction with tutored groups, this was not found.

The study was done at the end of the school year after students had been in tutored PBL small groups for two full semesters. Therefore, although students did not receive any formal training in PBL process (other than a 2 hour orientation at the beginning of the year) they were familiar with the function of the tutor. Throughout the year small groups had utilized a student/leader for each case as well as a physician/tutor. Perhaps because this study was done at the end of the year, the students had had enough experience in PBL process to function well without the guidance of a tutor. Our findings might have been different had the tutorless groups been used the entire year. This could lead to another study looking at differences between groups long-term.

Perhaps the most important finding was that the students with a tutor stayed in their small group sessions 13 minutes longer than the students in tutorless groups. It is difficult to tell from this study if students practice problem solving skills as well without a tutor. Students in groups without tutors may be more prone to rush through a case, moving more quickly to an "answer," and thereby missing some of the process steps. This is something that can be explored in future research. In conclusion, although students received similar quiz scores whether with or without a tutor, and seemed to have equal satisfaction with both formats, they did work on the case longer when they had a tutor present.

References

Albanese, M. A. & Mitchell S. (1993) Problem-based learning: A review of literature on its outcomes and implementation issues. Academic Medicine, 68(1), 52-80.

Dolmans, D. H. J. M., Gijselaers, W. H., Moust, J. H. C., deGrave, W. S., Wolfhagen, I. H. A. P. & van der Vleuten, C. P. M. (2002). Trends in research on the tutor in problem-based learning: Conclusions and implications for educational practice and research. Medical Teacher, 24, 173-180.

Neville, A. J. (1999). The problem-based learning tutor: Teacher? Facilitator? Evaluator? Medical Teacher, 21 (4), 393-401.

Norman, G. R. & Schmidt, H. G. (2000). Effectiveness of problem-based learning curricula: Theory, practice and paper darts. Medical Education, 34, 721-728.

Nuutila, E., Kinnunen, P., Torma, S., & Malmi, L. (2005). PBL cases and their role in learning programming.

Schmidt, H. G. (1983). Problem-based learning: Rationale and description. Medical Education, 17, 11-16.

Vernon, D. T. A & Blake, R. L. (1993). Does problem-based learning work? A meta-analysis of evaluative research. Academic Medicine, 68(7), 550-563.

Woods, D. R. (1993). PS-Where are we now? Journal of College Science Teaching, March/April, 312-314.

Woods, D. R., Hall, F. L., Eyles, C. H., Hrymak, A. N., & Duncan-Hewitt, W. C. (1996). Tutored versus tutorless groups in problem-based learning. American Journal of Pharmaceutical Education, 60, 231-238.

Laurie C. Clark, Oklahoma State University Center for Health Sciences

W. Stephen Eddy, Oklahoma State University Center for Health Sciences

Sherril M. Stone, Oklahoma State University Center for Health Sciences

Danny E. Thomason, Oklahoma State University Center for Health Sciences

Christopher C. Thurman, Oklahoma State University Center for Health Sciences

Clark, D.O., M.S., is Assistant Professor of Family Medicine; Eddy, D.O., M.P.H., is Associate Professor of Family Medicine; Stone, Ph.D. is Assistant Professor, in the Division of Research; Thomason, D.O., is Assistant Professor of Family Medicine, and Thurman, D.O. is Assistant Professor of Family Medicine