Objective structured practical examination vs. traditional clinical examination in human physiology: student's perception.
An integral part of a medical curriculum is an appropriate assessment of the students' clinical competencies. Assessment drives learning. But to foster active learning, assessment needs to be informative.  Although many options are available to do this more consistently, the Objective Structured Practical Examination (OSPE) is most preferred.  This method is derived from Objective Structured Clinical Examination (OSCE) by Harden and Gleeson. [2,3] The OSPE assesses practical competencies in an objective and structured manner with direct observation of the student's performance during planned clinical test stations. [2-5] In terms of the Miller's framework of development of clinical competencies, which focuses on four levels of assessment: knows, knows how, shows how and does, the OSPE assesses the third shows how level focusing on the assessment of performance of specific skills in a controlled setting.  The use of OSPE for formative assessment has great potential as the learners can gain insights into the elements making up clinical competencies as well as feedback on personal strengths and weaknesses.  In India, Maharashtra University of Health Sciences (MUHS) Nashik, which comes under the Medical Council of India (MCI) was established in 1998 and it grants MBBS (Bachelor of Medicine and Bachelor of Surgery) degree after successful completion of 9 semesters of pre- clinical, paraclinical and clinical subjects. Human Physiology is taught as a pre-clinical subject in the first 2 semesters of the MBBS course.  A traditional clinical examination (TCE) in Human physiology involves performing a particular clinical procedure which is followed by the bedside viva and assessment based on global performance rather than candidate's individual clinical skills.  TCE mainly focuses on the "knows" and "knows how" aspects, i.e the base of the 'Miller's pyramid of competence'.  It was felt that there is a need for (1) a more objective and structured assessment method; (2) feedback from the students; (3) feedback to the students to know their weaknesses and improve their clinical skills; and (4) sensitisation towards a new assessment system of the OSPE. We wanted to modernise our assessment methods and make it more competence based. Hence as a part of the FAIMER (Foundation of advancement in International Medical education and research) project, an OSPE was introduced as a formal method of assessment for the first time in the first MBBS physiology 2nd semester since we believed that the OSPE is a standardised tool and has proved advantages over the traditional assessment method.  The OSPE can also reduce the examiners' variability in marking the students.  The current study was designed to understand the first year undergraduate medical students' perception regarding the OSPE in comparison with traditional clinical examination method in the field of Human Physiology and whether it would be acceptable as an assessment method and what needs to be done to make it acceptable.
Materials and Methods
The study was conducted with 50 first year medical students in the Department of Physiology at the KJ Somaiya Medical College and Research Centre, Mumbai, India (part of MUHS university under MCI) after the approval from the Institutional Ethics Committee for Research on Human Subjects and a written informed consent from the participants. The students participating in the study were introduced to the OSPE system by short lecture and a role play organised by the faculty members. An orientation programme for faculty members was organised. A blueprint of the syllabus, structured checklist for observed and unobserved stations was prepared as per Bloom's taxonomy  along with examiners' and students' instruction manual and all validated by senior faculty members. A total of 50 students were divided into 2 batches of 25 each, examined by 3 examiners with teaching experience of 35 years, 6 years and 1 year respectively. All the 3 examiners conducted the TCE followed by OSPE for the same batch of students for 4 modules on 4 consecutive practical days in physiology. In the traditional assessment method, each student performed a particular clinical skill which was followed by bedside viva-voca on the same which was overall judged by the examiners. For the OSPE, students were oriented by an OSPE map and a written instruction list before the start of the exams. The OSPE consisted of 10 stations of 3 minutes each including: 1 station on Communication skills (1 mark), 2-3 observation/ procedure station on inspection, palpation and percussion of the abdominal system (2-3 marks), 6 unobserved stations with questions related to the procedural stations and MCQs (1-2 marks) + 1 rest station arranged in Physiology practical lab in a clockwise manner. Passing cut-off at 50 % based on criterion referencing was decided as per the MUHS rules.  At the end of the 4th day of the TCE--OSPE sessions, a 5 theme based 23 items self administered questionnaire was completed by all the 50 students. The 5 themes for the questionnaire evaluated the structure of the OSPE, content, administration of OSPE, reliability, objectivity and fairness of the OSPE and the value of OSPE compared with the TCE which they have experienced. The purpose of the questionnaire was to understand the effectiveness of OSPE as a teaching learning tool since in TCE, there is a less scope of step wise evaluation of every details in the clinical procedure hence often, there is a lacunae in the performance and the actual competency requirement from the students .Also the structure of the OSPE, whether it gives complete justice to the syllabus as blueprinted along with being relevant, reliable and valid since in TCE, the examiners may deviate from the set pattern of the syllabus. The questionnaire also was framed to understand the feasibility of OSPE in terms of its administration cost, conduction, time and manual effort in comparison with TCE. In addition, we wanted to gain an insight regarding their views about both the assessment methods. All items in the questionnaire were 'likert type items' having 5 response choice points. These points indicate the degree of agreement with the statement in ascending order. In the questionnaire, the students were instructed to tick mark the best response to the 23 statements viz; Strongly disagree, disagree, can't say, Agree, strongly agree.  1 = strongly disagree; 2 = disagree; 3 = can't say; 4 = Agree; 5 = strongly agree
At the end of the questionnaire, an open ended question was asked to elicit opinions regarding both the assessment methods. The participation was voluntary and anonymous. They were assured that no action will be taken against them if they wish not to answer to the questionnaire. The students were instructed to reply to their own answer sheet without any discussion with the peers. Basic statistical analysis of the 5-point Likert scale was done.
All 50 students (100%) answered the questionnaire.
Theme-1 [Is OSPE a better learning stimulus? (Table-1)]: Majority of the students felt that the OSPE pays more weight age on the practical details of the clinical procedure compared to TCE. They also felt that OSPE helps in identifying the weakness in their performances.
Theme-2 [Content of OSPE (Table-2)]: Regarding the content of OSPE, 72 % felt the combination of MCQs and short question is good. Almost 68 % students were satisfied by the sequence of the questions being more logical and majority appreciated the maximum coverage of relevant topics in the examination.
Theme-3 [Is OSPE a reliable and fair examination? (Table-3)]: 54 % strongly agreed that the checklist is valid marking system and 68 % agreed with reduction in the element of luck in examination.
Theme-4 [Administration of OSPE (Table-4)]: Majority of the students were satisfied by the arrangements of the OSPE. Almost 48 % felt OSPE less stressful compared to TCE. Majority of the students also did not feel the presence of examiners at the observed stations intimidating.
Theme-5 [Value of OSPE as an assessment method in comparison with TCE (Table-5)]: Majority of the students felt OSPE more satisfying compared to TCE however they also felt the interaction in TCE is better as compared to OSPE.
Currently in India, OSPE is conducted as a formative or summative examination in selected medical colleges all over India and allotted a very limited percentage of the marks.  This particular study was aimed to understand the acceptability of the OSPE among students as it was a relatively new assessment tool for them. This will be the basis of the future development to reform and refine the OSPE as an assessment tool. The majority of students perceived that OSPE had a better content and construct validity since the OSPE, in its structured checklist pattern helped them navigate smoothly through the clinical steps thus helping them know their weaknesses in clinical examination making OSPE a better examination method and a learning stimulus compared with the TCE. [12,13] Regarding the content of the OSPE, they felt that the questions were more relevant and logical with proper blueprinting of syllabus, the sequence of questions in the unobserved stations following the observed stations were proper and valid. Since all the students were exposed to similar types of questions with the same difficulty level, they felt that the checklist system is a fair and unbiased method with lesser element of luck playing any part in assessment. This also stimulated the critical thinking ability of the students which was welcomed by the students. Similar views were put forward by Duffield  however Imami M, Hosseinie et al  had a contrasting view regarding the fair and unbiased nature of OSPE which could be due to lack of awareness about the new system. Regarding the administration of OSPE, in spite of their first experience with OSPE, many felt that the instructions were clear and adequate. There was adequate time at both the observed and unobserved stations however few felt that the instructions provided were ambiguous and the checklist was too detailed and it compartmentalized the clinical procedure.  This could be attributed to habituation to the traditional format of examination. With the presence of examiners at the observed stations with the checklists in their hands, some students were intimidated and anxious while majority were relaxed since they only had to perform without answering anything feeling less stressful. This is paradoxical with many studies showing that OSPE is more stressful and anxiety driven.  Overall, they felt OSPE was a more satisfying experience and suggested for more OSPE sessions in future. However, some of the students did mention that, the lack of interaction with the examiners was depressing since, they could not get the clue by the facial expressions or the hints thrown by the examiners during the examination. During the entire OSPE session, the areas where majority of the students committed similar mistakes, were noted and at the end of the sessions, the feedback regarding their performance at both the observed and unobserved stations along with the common mistakes performed by the students were discussed.  This was positively welcomed by many students as their performance, areas of weaknesses were discussed and the correct approach in the examinations was explained. There were few limitations in the present study since it was the first encounter of the students with OPSE, the number of the students was fifty and only single batch of 2nd semester undergraduate students were exposed to the OSPE method of assessment, hence only limited views could be collected. However in the process of setting OSPE, the department of Physiology could develop a validated OSPE banks for future use. The feedback from the students was invaluable and it facilitated in forming more comprehensive, compact OSPE checklist which would evaluate more cognitive, psychomotor and attitudinal skills. This also emphasized the need of continuous faculty development in the field of medical education for its betterment. This also sent a clear message among students that achieving the better clinical competencies and not mere memorising and recall is a must for better grades.
Inspite of the limited use of OSPE in majority of the medical colleges across India, the current study showed a positive perception towards OSPE as a fair, unbiased, valid, reliable assessment method compared to traditional clinical examination. It also highlighted the requirement of a continuous faculty development for a more comprehensive and elaborates OSPE bank in future.
The researchers are grateful to the Dean Dr. Geeta Niyogi, Dr. Lalit Mehta (Director), Dr Avinash Supe, FAIMER and the first MBBS students 2010-2011 batch of K.J. Somaiya Medical College and Research Centre for their cooperation.
[1.] Gupta P, Dewan P, Singh T. Objective Structured Clinical Examination (OSCE) Revisited. Med Educ 2010;47(11): 911-20.
[2.] Harden RM, Gleeson FA. ASME Medical Education Booklet No. 8: Assessment of clinical competence using an objective structured clinical examination (OSCE) Med Educ. 1979;13:41-54.
[3.] Harden RM. What is an OSCE? Med Teacher 1988; 10(4): 19-22.
[4.] Harden RM, Stevenson M, Wilson DW, Wilson GM. Assessment of clinical competencies using objective structured clinical examination. Br Med J. 1975;1(5955):447-451.
[5.] Harden RM. How to assess clinical competence: an overview. Med Teacher 1979; 1(8): 289-96.
[6.] Miller GE. The assessment of clinical skills/competence/performance. Acad Med. 1990 Sep;65(9 Suppl):S63-7.
[7.] Bijlani RL. Assessment of laboratory exercises in Physiology. Med Educ. 1981 Jul;15(4):216-21.
[8.] Matsell DG, Wolfish NM, Hsu E. Reliability and validity of the objective structured clinical examinations in pediatrics. Med Educ 1991;25:293-99.
[9.] Bloom BS. Taxonomy of Educational Objectives: The Classification of Educational Goals. Chicago: Susan Fauer Company Inc. 1956. pp. 201-207.
[10.] Al Rubaish A, Wosornu L, Dwivedi SD. Using Deductions from Assessment Studies towards Furtherance of the Academic Program: An Empirical Appraisal of Institutional Student Course Evaluation. iBusiness 2011;3(2): 219-227.
[11.] Duerson MC, Romrell LJ, Stevens CB. Impacting faculty teaching and student performance: nine years' experience with the objective structured clinical examination. Teach Learn Med. 2000;12(4):176-82.
[12.] Newble DI. Eight years experience with a structured clinical examination. Med Educ 1988;22(3):200-4.
[13.] Duffield KE, Spencer JA. A survey of medical students' views about the purposes and fairness of assessment. Med Educ 2002;36(9):879-86.
[14.] Imani M. Is OSCE successful in pediatrics? Journal of Medical Education 2005;6(2): 153-58.
[15.] Woodburn J, Sutcliffe N. The reliability, validity and evaluation of the objective structured clinical examination in podiatry. Assess Eval High Educ 1996;21:131-47.
[16.] Allen R, Heard J, Savidge M, Bittergle J, Cantrell M, Huffmaster T. Surveying Students' Attitudes During the OSCE. Adv Health Sci Educ Theory Pract. 1998;3(3):197-206.
[17.] Pierre RB, Wierenga A, Barton M, Branday JM, Christie CD. Student evaluation of an OSCE in paediatrics at the University of the West Indies, Jamaica. BMC Med Educ. 2004;4:22. Available form: URL: http://www.biomedcentral.com/14726920/4/22
Source of Support: Nil
Conflict of interest: None declared
Pinaki Wani, Vrinda Dalvi
KJ Somaiya Medical College and Research Centre, Sion, Mumbai, Maharashtra, India
Correspondence to: Pinaki Wani (firstname.lastname@example.org)
Received Date: 04.02.2013
Accepted Date: 08.03.2013
Table-1: Is OSPE a better learning stimulus? Theme 1 Is OSPE a SD D CS A SA better learning stimulus? 1 OSPE encourages 0 0 02 29 19 us to pay more (4%) (58%) (38%) attention to practical examination 2 OSPE tests 0 0 05 31 14 details of (10%) (62%) (28%) procedure in steps 3 OSPE helps in 0 0 05 18 27 identifying (10%) (36%) (54%) lacunae in the clinical skills 4 OSPE is a good 0 0 04 26 20 form of (8%) (52%) (40%) examination & learning process Table-2: Content of OSPE Theme 2 Content of OSPE SD D CS A SA 1 MCQs & Short 0 0 06 36 08 questions after (12%) (72%) (16%) every procedural station is a good idea. 2 Sequence of 0 04 06 34 6 Question was (8%) (12%) (68%) (12%) more logical. 3 OSPE covers the 0 0 05 18 27 relevant and (10%) (36%) (54%) important topics and consistent with the learning objectives of the syllabus. 4 OSPE highlights 0 05 05 30 10 the areas of (10%) (10%) (60%) (20%) weakness in the subject. Table-3: Is OSPE a reliable and fair examination? Theme 3 Is OSPE a reliable and SD D CS fair examination? 1 Checklists provides fair & 0 0 05 (10%) unbiased system of marking 2 OSPE reduces the element of 0 04 (8%) 06 (12%) luck in examination Theme 3 Is OSPE a reliable and A SA fair examination? 1 Checklists provides fair & 18 (36%) 27 (54%) unbiased system of marking 2 OSPE reduces the element of 34 (68%) 06 (12%) luck in examination Table-4: Administration of OSPE Theme 4 Administration of OSPE SD D CS 1 The instructions were 0 04 (8%) 06 (12%) clear and adequate 2 There is ample time at 0 05 (10%) 04 (8%) MCQ stations 3 Aware about the nature 0 0 05 (10%) of OSPE examination 4 Observed stations are 08 (16%) 27 (54%) 05 (10%) time consuming 5 Unobserved stations are 07 (14%) 25 (50%) 18 (36%) time consuming 6 OSPE is less stressful 02 (4%) 10 (20%) 08 (16%) compared to TCE 7 It is confusing to 17 (34%) 30 (60%) 02 (4%) switch from one station to another 8 Examiners at the 12 (24%) 19 (38%) 02 (4%) procedural stations were intimidating Theme 4 Administration of OSPE A SA 1 The instructions were 30 (60%) 10 (20%) clear and adequate 2 There is ample time at 26 (52%) 15 (30%) MCQ stations 3 Aware about the nature 18 (36%) 27 (54%) of OSPE examination 4 Observed stations are 05 (10%) 05 (10%) time consuming 5 Unobserved stations are 0 0 time consuming 6 OSPE is less stressful 24 (48%) 06 (12%) compared to TCE 7 It is confusing to 01 (2%) 0 switch from one station to another 8 Examiners at the 10 (20%) 07 (14%) procedural stations were intimidating Table-5: Value of OSPE as an assessment method in comparison with TCE Theme 5 Value of OSPE as an SD D CS assessment method in comparison with TCE 1 OSPE is more satisfying 0 02 (4%) 10 (20%) compared to traditional method of assessment 2 There is no much 0 10 (20%) 04 (8%) difference between OSPE & traditional method of assessment 3 One cannot pass OSPE 01 (2%) 05 (10%) 0 without attending practical classes 4 OSPE reduces the chance of 03 (6%) 07 (14%) 05 (10%) failing in exam compared to TCE 5 Traditional examination 0 02 (4%) 08 (16%) has more interaction compared to OSPE Theme 5 Value of OSPE as an A SA assessment method in comparison with TCE 1 OSPE is more satisfying 26 (52%) 12 (24%) compared to traditional method of assessment 2 There is no much 26 (52%) 10 (20%) difference between OSPE & traditional method of assessment 3 One cannot pass OSPE 29 (58%) 14 (28%) without attending practical classes 4 OSPE reduces the chance of 28 (56%) 07 (14%) failing in exam compared to TCE 5 Traditional examination 30 (60%) 10 (20%) has more interaction compared to OSPE Table-6: Comments from the students Comments from the students No. "OSPE is a fair, unbiased means of assessment" 42 "OSPE was more uniform and objective since all the 45 students are asked similar questions with same difficulty level" "less fear of examiners"--no interaction with 38 examiners "Combine TCE with OSPE on regular basis" 40
|Printer friendly Cite/link Email Feedback|
|Title Annotation:||RESEARCH ARTICLE|
|Author:||Wani, Pinaki; Dalvi, Vrinda|
|Publication:||International Journal of Medical Science and Public Health|
|Date:||Jul 1, 2013|
|Previous Article:||Prevalence of hypothyroidism among patients of Meerut, Uttar Pradesh: a hospital based study.|
|Next Article:||Incidence of neonatal sepsis in tertiary care hospital: an overview.|