Printer Friendly

The making of a new certification exam.

The making of a new certification exam

An insider chronicles the conception, development, testing, and validation of the ASCP's Diplomate in Laboratory Management.

Passing a registry examination is one of the most important milestones in a laboratory career. Some laboratorians stop there, happy to have gotten that far. Others progress in certification from technician to technologist as a final goal; still others go on to be certified as specialists.

All the technical certification in the world, however, does not necessarily qualify a person to manage a laboratory. During the 1980s, as lab management grew more administrative and less technical--due in part to prospective reimbursement, cost containment pressures, and other economic factors--the need for a recognized credential for clinical laboratory administration became clear. Two professional associations, the National Certification Agency for Medical Laboratory Personnel (NCA) and the American Society of Clinical Pathologists (ASCP), now offer such a credential. This article will describe the development of the ASCP's Diplomate in Laboratory Management (DLM). The process is echoed in other ASCP registry examinations as well. * Behind the scenes. Shortly after passing the examination in 1990, I was asked to serve on the ASCP's laboratory management examination committee. That invitation provided me with a unique opportunity to explore the development and scoring of examinations. It enabled me to get behind the scenes of the ASCP's Board of Registry (BOR), an independent component of the ASCP. The BOR was created in 1928, shortly after the 1922 establishment of the ASCP, to develop criteria and procedures for professional certification--that is, to determine that qualified medical laboratory personnel meet specific standards of performance and expertise.

Four major ongoing activities assist the BOR in fulfilling its mission of certifying laboratorians: reviewing and evaluating applications for examination and certification, developing and administering exams, researching ways to monitor and improve tests and test methodologies, and maintaining a roster of certified individuals. The DLM examination is one of the ASCP's newest credentialing efforts. Its phlebotomy credentialing examination was similarly given for the first time in 1989. * Fact finding. In 1984, the BOR's research and development committee asked more than 200 pathologist laboratory directors nationwide whether the existence of a credential in laboratory management would be useful in hiring and promoting laboratory managers who were not physicians. Encouraged by positive response--approximately 75% said yes--the committee reported to the Board of Governors, the ASCP's advisory body, recommending further investigation. Committee members studied management certification methods used by the American College of Healthcare Executives, based in Chicago, and the Healthcare Financial Management Association, headquartered in Westchester, Ill. * Data collection. The BOR's committee collected information on certification levels, requirements for qualifying to take the examination, documentation of those qualifications, application procedures, and fees. The Board of Governors then asked the BOR to prepare the examination.

A task force was established to design an appropriate certification method. Panel members were recommended by the Clinical Laboratory Management Association (CLMA) of Paoli, Pa. The five-person task force has evolved into the current laboratory management examination committee, of which I am one of eight members. At the committee's first meeting, in the spring of 1986, members drafted a content outline, a detailed list of general subject areas, and competency statements consisting of commonly performed managerial tasks required for efficient and effective lab practice. * Opinions sought. The committee used the drafts of the content outline and competency statements as the basis for a survey sent to a random sample of more than 2,000 ASCP and CLMA members who had listed their titles as laboratory director, assistant laboratory director, chief technologist, administrative technologist, or laboratory manager. Respondents were asked to rank the importance of 52 commonly performed or delegated tasks and to evaluate 35 management content areas according to their importance to the job of manager and the time typically spent on each.

The committee based the final competency statements and content outline on several factors besides their own knowledge and experience. They sought a consensus of expert opinion and reviewed the literature and the survey of ASCP and CLMA members. Drafts of the competency statements and content outline were submitted to the Board of Governors and approved.

The result was the examination's final content outline (Figure I) and competency statements, which indicate the approximate percentage of exam questions in each of five areas: general, financial, operations, personnel, and marketing. * Developing questions. Writing test questions is a time-consuming task. Requirements include knowledge of the subject, an understanding of the examination population, and mastery of verbal and written communication skills.

Committee members set about writing questions that would test the abilities of prospective diplomates. Questions were written, reviewed, and field tested at the CLMA's annual conference, held in Dallas in the fall of 1987. After studying the results of that test, the committee revised the original questions and formulated additional ones. Guidelines on the content of the exam were published in the spring of 1988. Test questions ("items") whose results were found to be statistically valid were incorporated into the first actual DLM exam, given on schedule the following February. * Pass/fail. In 1980 the BOR adopted a pass/fail (criterion-referenced) format for its exams as preferable to grading the exams on a curve (norm-referenced). It is more appropriate, the BOR felt, to measure individual performance on a range of tasks than to rank each score in relation to the performance of other test takers. This method is now used on all examinations given by the ASCP and the NCA.

The multiple-choice format with only one best answer among four was selected because, the BOR believes, it measures ability most objectively and its validity is statistically supported. Questions are written at three levels of taxonomy (recall, interpretation, and problem-solving).

Figure II shows sample questions categorized by management content (that is, by subject matter) and taxonomy. Assembled after five years of planning and research, the first ASCP examination for the diplomate in laboratory management was given in February 1989. The DLM examination is administered on the third Friday in February in major cities across the country. Scoring is done by computer. The BOR has targeted the DLM exam to be given via computer beginning in 1993. (For a description of the first computer-referenced ASCP examination, see "Putting technology to the test" in the preceding article.) More information on the exam is available from the ASCP at (312) 738-1336. * Follow-up. Questions for each examination are derived from a pool assembled by the BOR committee, which meets each spring at the ASCP's Chicago headquarters. Members bring new questions they have devised. These are sorted at the meeting according to content area and revised as necessary to enhance or clarify their relevance, grammatical structure, spelling and punctuation, response validity, level of difficulty, classification, and taxonomy. Unapproved questions are scrapped or reworked for consideration the following year. The pool of test questions is increased each year so that a supply of fresh items is always available when the examination is being constructed.

Committee members assess the examination taken the preceding February, scrutinizing performance on each item. To insure fairness and overall quality of the exam, questions that have failed to measure ability as expected are discarded. The ASCP's full-time measurement specialist (psychometrician) and manager of examination analysis assist the committee in reviewing test statistics. * Evaluation. One useful index in determining the validity and usefulness of individual questions is the p value, which measures difficulty. Another is item discrimination, which indicates how the question differentiates between those who did well on the exam as a whole and those who did not. A third index is item calibration, which helps in estimating the difficulty of each item in relation to others. At least three other indices are used to evaluate overall performance on each question.

The laboratory management examination committee is one of nine committees involved with the BOR's certification examinations. The committees include technologists, technicians, Ph.D. laboratory scientists, and physicians. Committee members are responsible for developing new items, determining the accuracy and relevance of test items, constructing tests, and establishing the pass/fail standard.

Committee work continues with setting goals and objectives for the following year, electing a chairman and vice-chairman, and assigning members to review and rework questions. Membership extends for three years, subject to approval of the BOR. * Fair system. With the advent of the new examination, employers about to hire a laboratory manager have gained an objective basis on which to make a decision.

Anyone taking a certification exam wonders whether the test will actually measure what it's supposed to and how it can tell who is competent enough to pass. I certainly wondered such things as I awaited my exam papers in February 1990. Now that I have some experience in developing examinations myself, I feel confident that the process is fair and that it works. We can be proud that our profession's credentialing process is as scientific as the medicine of medical technology. [Figures I and II Omitted]

The author, a member of MLO's Editorial Advisory Board, is director of laboratory quality assurance at Elmhurst (Ill.) Memorial Hospital.
COPYRIGHT 1991 Nelson Publishing
No portion of this article can be reproduced without the express written permission from the copyright holder.
Copyright 1991 Gale, Cengage Learning. All rights reserved.

Article Details
Printer friendly Cite/link Email Feedback
Title Annotation:conception, development, testing and validation of American Society of Clinical Pathologist's Diplomate in Laboratory Management
Author:Berte, Lucia M.
Publication:Medical Laboratory Observer
Date:Oct 1, 1991
Previous Article:Certification of laboratorians: an update.
Next Article:Designing an in-house phlebotomy training program.

Related Articles
Giving comps between practicum and certification.
Certification of laboratorians: an update.
Personnel licensure: making slow inroads.
CLMA conference meets in the Mile-High City.
Continuing education resources for laboratorians.
Multitude of professional organizations offers choices and benefits for laboratorians.
New pathologists' assistant certification.
Mike Dugan and Maryam Sadri talk about their Specialty.
Training for the laboratory worker.
Credentials: what do they really mean? Comparing CLT/MLTs and CLS/MTs.

Terms of use | Copyright © 2017 Farlex, Inc. | Feedback | For webmasters