Fostering critical thinking skills through a web-based tutorial programme for final year medical students--a randomized controlled study.

Purpose. To study the performance of final year medical students, at the end of module assessments, following exposure to a web-based tutorial programme designed to facilitate critical thinking and logical reasoning.

Method. Multimedia enhanced web-based tutorials based on patient oriented problems, designed to develop higher cognitive skills in students, were used as a supplement to an existing curricular programme for final year medical students attending an orthopaedic surgery rotation. We tested this intervention in a group of 163 final year medical students. Clusters of 15 students were randomly assigned either to an intervention or a control group. Students were examined for their clinical examination skills, critical thinking abilities, and specific factual knowledge based on curricular requirements.

Results. Mean scores between control and intervention group were analyzed, and the intervention group outperformed the control group (Statistically Significant).

Conclusion. Interactive web-based tutorial programmes can be successfully designed to develop higher cognitive skills in medical students. This results in more sound clinical decision making skills than simple noninteractive, content-rich web resources.

**********

Jl. of Educational Multimedia and Hypermedia (2003) 12(3), 267-273

Clinicians across all disciplines are facing the difficulty of filtering large quantities of information and determining, which is based on evidence, in order to make safe and accurate management decisions. Although evidence-based decision support tools provide relevant and up-to-date evidence to clinicians, actual integration of the new information into practice is a function of experience. That is, interpretation and use of evidence as opposed to hypotheses depends on prior knowledge and experience of the clinician (Allen, Arocha, & Patel, 1998). In this aspect, undergraduate medical students are at a disadvantage as they are inundated with information that often, they are unable to use in a logical, analytical manner. Conventional lecture based curricula are not well designed to develop such analytical thinking and problem-solving skills.

The problem of information overload is especially severe in medical subspecialties, in which there is limited time and exposure allowed for any given subspecialty, through which medical students rotate in any academic year. Imparting practically relevant knowledge therefore needs to be done within short teaching modules. Without critical thinking ability, the student may not be able to use available information well. Consequently students have expressed difficulty in understanding the applicability of subspecialty content for general medical practice.

As a group of teachers in charge of planning the Orthopaedic curriculum for final year medical students, we strongly felt the need to implement a teaching strategy that would enhance learning and inculcate critical thinking abilities in students, without subjecting them to an information overload. We made use of a computer-based multimedia course for final year medical students doing their Orthopaedic rotation, that was comprised of a series of self-navigated problems designed to stimulate critical thinking and logical reasoning. We then compared the use of this intervention with the conventional lecture based didactic method of teaching in a randomized controlled study.

METHODS

During their Senior Surgical Clerkship, final year medical students attend a 3-week module with the Department of Orthopaedics, in 11 groups of 14-15 students, spread over the academic year. As part of this project, clusters of 15 students were randomly assigned either to a Study or to a Control group during their attachment to our department.

A preintervention test comprised of multiple-choice questions was administered to both groups at the beginning of their module.

Study Group Students

We designed a series of computer-based clinical case simulations (CCS) designed to foster reasoning and logical thinking abilities in students. The web-based problems were designed to stimulate higher cognitive skills, notably analysis, application, and evaluation. A group of experienced practicing orthopaedic surgeons evaluated the scenarios and felt that they scenarios required students to make clinical decisions reflective of real life problems. The basic design consisted of a patient oriented clinical problem (http://media.ort.cuhk.edu.hk/fats-scenario) the successful resolution of which depended upon the effective manipulation of a few critical elements of the problem. Thus, students were required to comment on radiographs, interpret clinical and diagnostic test results, select appropriate investigations, and provide logical reasoning to justify their clinical decisions. They could only proceed to the next step of the tutorial provided they typed in a response indicating the reasons for their clinical decisions. Students navigating through the steps of these scenarios were exposed to all aspects of clinical decision making--beginning with assessment, going on to selection of appropriate tests, interpreting their results, and making decisions regarding treatment. The authors facilitated this process by acting as moderators giving suggestions, participating in web-based discussions, and providing formative feedback. Each group of "study" students met their facilitator collectively, three times each week and discussed their answers. Facilitators encouraged inter-student discussion and ensured all students participated in the discussion. They did not provide direct answers to the student,s but instead persuaded them to see examples of similar cases in the wards and solve difficulties through further group discussion.

These computer-based case-simulations (CCS) were posted at a secure site on the World Wide Web (WWW or Web), and each student in the study group could log in from any terminal (home, library, or hostel) using his student identity number, which served as a unique identifier. During their module, each student discussed eight clinical case simulations representing common orthopaedic problems. As part of the exercise, links to the National Library of Medicine (U.S.A.), The Chinese University Medical Library, and other resources were provided. Peer group discussion was strongly encouraged. Each student's online access was monitored.

The web interface was designed using Active Server Pages (ASP) and a computer officer was specifically appointed for the development of this project. Server side computing enabled us to control student access and maintain security. The unique student identifier enabled the facilitator to monitor each individual student response and the number of CCSs completed. Intervention group students did not have access to any other web-based teaching material or formal didactic lectures, unlike their control group counterparts who did. Bedside tutorials and clinical rounds were similar to those in the study group.

Control Group

Students in the control group followed the conventional teaching programme during their 3-week attachment. This included didactic lectures, bedside tutorials, ward attachments, and outpatient clinics. Additional notes and lecture material were posted on the Web for student access and down-loading. The nature of the material posted on the web was didactic in format and included PowerPoint presentations, extracts from notes and handouts, as well as video clips. In addition, control group students also met tutors three times a week for bedside teaching and didactic presentations.

The teacher contact hours in both groups were similar. Facilitators for the study group did not teach the control group students. Control group students were unaware of CCS on the Web and did not have access privileges to these simulations.

At the end of the 3-week module, students were assessed and marked on the basis of (a) a computer based assessment, (b) an objective Structured Clinical Examination (OSCE) test, and (c) a clinical examination comprising patients in the wards. In the ward examination, students were assessed for their clinical examination skills, critical thinking abilities as well as context specific factual knowledge based on the curricular requirements stipulated in the curriculum handbook published by the medical faculty. Such examinations have been conducted by the Orthopaedic Department for the last 5 years and have been previously evaluated by a board of experienced clinicians. This board has accepted these examinations as a validated measure of clinical reasoning and logical thinking ability.

Orthopaedic surgeons from other hospitals and institutions, who were unaware of the ongoing research study, were recruited to conduct the end-of-module examination. Mean examination scores between the control and study group were compared and analyzed using t-test and a commercially available statistical programme (SPSS for Windows, Version 10.0, 1999).

In addition, the time spent by students in the hospital wards was also monitored. A ward-steward, (who was unaware of the study), recorded the time spent by groups of students engaged in self-study/clinical examinations in the ward. Students were also asked to provide feedback on the web-based scenarios at the end of each module.

RESULTS

In this study 163 students were recruited, 74 in the Control and 89 in the Study group. Web access and time "online" was noted for each student. The mean time spent on the using the Web was similar in both groups (mean 102 minutes/week, S.D. 21) and all students accessed web-based teaching materials from the web site posted by the department. There were no breaches of security as all web sites were password protected. Web site downloads were compared between the control and interventional group. Fifty five percent of students in the control group downloaded web site teaching materials, compared to 79% in the control group.

There was no difference in the premodule test results between the study and control groups. When we compared the End-of-module test results, mean score for the study group was 58.72 (SD [+ or -] 6.8), while mean score in the control group was 52.89 (SD [+ or -] 5.6) Students in the study group scored better than in the control group, and this difference was statistically significant (p<0.001).

The additional time spent in the wards, beyond that allocated for bedside tutorials was measured. Control group students spent an additional 5.15 hours, (SD 1.0) each week while the interventional group spent 7.0 (SD 2.1) hours for their self-study on clinical patients at the bedside in the wards and the difference was statistically significant.

DISCUSSION

An important goal in undergraduate medical education is the development of good clinical reasoning and problem-solving ability. This implies that the student must be able to use background information and apply it to the patient in context. Scenarios allow the student to engage in such behavior without jeopardizing patient safety. Importantly such scenarios allow the educator to evaluate physician behavior during a problem-solving exercise.

In our small study, using web-based tutorials, we led students through a thinking pathway that facilitated the development of higher cognitive skills such as analysis, application, and evaluation (Bloom, 1956). Problems based on the "key features" concept have been accepted as a valid measure of clinical reasoning and analytical ability (Norcini, Swanson, Grosso, Shea, & Webster, 1984; Page & Bordage, 1995). We postulate that these scenarios provided students the confidence to engage in clinical problem-solving. This was indeed reflected in the comments students provided, at the end of the module. These exercises stimulated students to apply these analytical skills in the wards and the bedside--a process that the facilitator was able to observe and monitor.

Web-based patient CCS were very well accepted by students. They felt that these simulations complemented and deepened their understanding of patient care as they could relate this information to real cases present in the wards. Students in the study group spent more time in the wards doing self-study and physical examinations as compared to the control-group. Infact the scenarios reinforced the need to obtain the necessary clinical history and to complete a comprehensive physical assessment of patients, in order to make clinical decisions. As a result, this group of students performed better in clinical and physical examination skills in their end-of-module assessment.

Students particularly valued the formative feedback that was provided by the facilitator. We reviewed student answers to CCS at our weekly facilitation meetings and were able to discuss, critique, and solve issues through collective discussion, without fear or embarrassment.

An active student teacher interaction ensured that evaluation of the thinking process, not just the resources uncovered, provided the value to the information gained through these CSS. These groups of students did not perceive the need to "download" everything that was posted on the Web, and in essence were able to discern between useful and useless information. Access to information is not an assurance that it will be used and there is widespread misunderstanding that learning will automatically occur merely because the framework of accessible information is available. Students already have access to internet sites that contain lecture notes, questions, examinations tips, and study guides. A large variety of such material is available on the Internet or on CD-ROMS. Teachers too, often use the Web as a mere repository of their teaching materials. This only reinforces the tendency to "download," which is omnipresent in students. Without sound underlying pedagogic principles, mere content, however rich, does not facilitate the process of knowledge acquisition and application (Alur, Fatima, & Joseph, 2002). Computer based simulations are available in almost every discipline of medicine today (Takabayashi et al., 1995). Because of the feedback they provide, such simulations allow students to explore their ideas and by incorporating multimedia and interactive features (Harasym et al., 1980), may ultimately provide a comprehensive learning experience.

The study does have some limitations besides being relatively small. It may be difficult to determine what additional factors could have contributed to the differences in assessment outcomes between the two groups. The enthusiasm of the facilitators (who were conducting this study) and the intensity of their interaction with students may have been significant contributing factors. Students do exchange ideas and communicate with their fraternity. It is also difficult to assess the extent of cross exposure that might have occurred between the two groups.

Notwithstanding these limitations, we have demonstrated that well designed web-based tutorials stimulate students to think and may complement the teaching resources of clinical departments. Such resources may foster better clinical and critical thinking skills in medical students, without subjecting them to an information overload. Simulations may also provide a validated and convenient online source of continuing professional education for practicing physicians. Such multimedia-rich, pedagogically sound environment may allow physicians to engage in complex problem solving tasks without endangering patient safety.

Note

This project has been funded by a Teaching Development Grant (TD 1998-2001, Ref: CUHK-6) under the auspices of the University Grants Council, Hong Kong.

References

Allen, V.G., Arocha, J.F., & Patel, V.L. (1998, August-September). Evaluating evidence against diagnostic hypotheses in clinical decision-making by students, residents, and physicians. International Journal of Medical Informatics, 51(2-3), 91-105.

Alur, P., Fatima, K., & Joseph, R. (2002, July). Medical teaching websites: Do they reflect the learning paradigm? Med Teach, 24(4), 422-4.

Bloom, B.S. (Ed.) (1956). Taxonomy of educational objectives: The classification of educational goals: Handbook I, cognitive domain. New York; Toronto: Longmans, Green.

Harasym, P., Baumber, J., Bryant, H., Fundytus, D., Preshaw, R., Watanabe, M., & Wyse, G. (1980). An evaluation of the clinical problem-solving process using a simulation technique. Med Educ, 14(6), 381-386.

Norcini, J.J., Swanson, D.B., Grosso, L.J., Shea, J.A., & Webster, G.D. (1984, December). A comparison of knowledge, synthesis, and clinical judgment. Multiple-choice questions in the assessment of physician competence. Eval Health Prof, 7(4), 485-99.

Page, G., & Bordage, G. (1995). The Medical Council of Canada's key features project: A more valid written examination of clinical decision-making skills. Acad Med, 70(2), 104-10.

Takabayashi, K., Fujikawa, K., Suzuki, T., Yamazaki, S., Honda, M., Amaral, M., Satomura, Y., Yoshida, S., & Tomioka, H. (1995). Implementation and evaluation of computerized patient management problems. Medinfo, 8 Pt 2:1218-21.

SHEKHAR M. KUMTA, PAK L. TSANG, LEUNG K. HUNG, AND JACK C.Y. CHENG

The Chinese University of Hong Kong Hong Kong

[e-mail]