Addressing the Maintenance of Certification Challenge: The College of American Pathologists Response

According to recent Gallop findings, more than 75% of patients surveyed would choose a board-certified physician over a physician who was not board certified but was recommended by a trusted friend or family member.' Eighty percent of those surveyed would be likely to seek another physician if their current physician's certification had lapsed. Clearly, most patients believe that the quality of health care is closely linked with the competence of physicians as indicated by specialty board certification.2

The American Board of Medical Specialties (ABMS) grants board certification to physicians based on completion of 3 to 6 years of postgraduate training in an accredited training program, acceptable performance on a proctored examination of cognitive knowledge, and, for some specialties, satisfactory evaluations from program directors, oral examinations, audits of medical records, or observed performance with patients. Recertification examinations with some or all of these features are required for most medical specialties. This process will apply to all pathologists who complete their training in 2006 or thereafter.3

Historically, the American Board of Pathology (ABP) has granted board certification to all pathologists who perform satisfactorily on an initial cognitive examination. Board certification remains in force as long as the board-certified pathologists maintains their state medical licenses, which all require documented yearly participation in continuing medical education (CME). The number of required hours of CME is specified by state licensure requirements and varies from 15 to 50 hours per year. The ABP has stipulated that, beginning in 2006, all primary and subspecialty certificates issued by the ABP will be limited to 10 years.3

The recertification process has not been uniform in its form or content across medical specialties until recently. In March 2000, the ABMS adopted 6 general competencies and 4 basic components for assessing continuing competence of all physicians (Table 1). The ABMS directed each member board (including the ABP) to submit plans for assessing each of the 4 basic components: an initial plan for assessing components 1 to 3 by July 2003 and a final plan for assessing all 4 components by December 2004. This process has been designated Maintenance of Certification (MOC) and must be documented for physicians to maintain their certification.4

The ABP has provided a description of their plans for assessing each of the first 3 basic components in an application addressed to the ABMS in May 2004. Final plans for addressing all 4 basic components were due to the ABMS in December 2004. Each diplomate applying for MOC will be required to document compliance with the ABP requirements for each of the 4 components and 6 general competencies of its MOC program. As part of their May 2004 application to the ABMS, the ABP indicated that they are setting standards for the CME and self-assessment programs, including structure (pretest, tutorial, posttest components), need for performance feedback, mandatory participation with documentation, accessibility (Web-based, home self-study modules), and standardized format for learning objectives. The ABP is working with cooperating societies to accomplish this task. The ABP has asked national, state, and local pathology societies, university pathology departments, major medical centers, and/or publishers or other CME providers to sponsor and organize CME programs and self-assessment modules in accordance with the ABP curricula and content specifications.3 The society that represents board-certified pathologists, the College of American Pathologists (CAP), is working to respond to the ABP request. The CAP has identified education for pathologists as a strategic goal and invested in the growth and development of a professional education division beginning in August 2002. The organization is committed to providing pathologists with practical education that will help members be successful in their practices.

This article describes the process by which the CAP Education Committee and its professional education staff are working to define pathology-specific competencies within MOC categories and to create education courses targeted to competencies identified in each MOC category. We hope that publication of the process and its results will create dialogue among pathology education providers and the ABP to approach a more precise understanding of each MOC category and foster the development of courses that will specifically address those educational requirements.

MATERIALS AND METHODS

The CAP Education Committee implemented a 5-step process for identifying courses and programs to address MOC competencies.

Step 1

The 6 MOC competency categories, the ABMS definitions for each category, and the Accreditation Council for Graduate Medical Education (ACGME) descriptions of each competency category were reviewed and extensively discussed by the CAP Education Committee and Education Division staff. The ABMS categories were created to be generally applicable to all physicians. The ACGME had modified these initial definitions to make them more directly descriptive to physicians in training. Based on this review, customized competency category definitions specific for pathologists were developed by the Education Committee and Education Division staff.

Step 2

Working in small groups, pathologists and education staff next defined specific competencies for each MOC category. The MOC competencies were created based on a review of appropriate medical education literature and on personal practice experience with each MOC category.2,3,5-12

Definitions for achievement of competency in pathology residency programs were also reviewed.5,8-10 Education staff used their competency model development experience and knowledge of best practices to develop an accurate and comprehensive competency list.

To further define the competencies, a list of knowledge and skill statements was developed for each competency. Working in small groups, committee members were asked to identify pathology-specific skill statements for each MOC category. The entire Education Committee then reviewed the knowledge and skill statements for redundancies and completeness; appropriate refinements were made. Using this process, knowledge and skill statements were created for 5 of the 6 MOC categories.

Medical knowledge, the remaining MOC category, was addressed differently. The CAP Education Committee did not create specific knowledge and skill statements for medical knowledge. Because medical knowledge is the MOC category that constitutes the largest body of specific information on which pathologists must be examined, Education Committee members thought that the medical knowledge category would be the primary focus of many pathology societies and the ABP. Committee members used Henry's Clinical Diagnosis and Management by Laboratory Methods for clinical pathology and Rubbins and Cotran Pathologic Basis of Disease for anatomic pathology as general guides to the range of topics to include in education offerings devoted to medical knowledge.13,14 Using this framework, general competencies to be demonstrated in specific knowledge areas were developed.

Step 3

The draft competencies were validated using 3 separate approaches. First, Education Committee members elicited feedback from selected peers on the extent to which the competencies were relevant and comprehensive. Suggested refinements were discussed and approved in a face-to-face meeting.

Second, an on-line survey was developed to gather additional pathologist feedback on 3 competency categories where there appeared to be significant opportunity to create pathologist education programs: systems-based practice, practice-based learning and improvement, and patient care. The survey was distributed to a random sample of 2476 CAP fellows. Survey respondents (14% of those surveyed) rated the importance of each knowledge and skill to their overall effectiveness as a pathologist and their current level of proficiency in each area. Survey results were analyzed to identify education needs (ie, competencies where importance was high and proficiency was relatively low).

Third, a random sample of 2004 annual meeting attendees were invited to participate in focus groups to gather more detailed information on learning needs for each of these 3 competency areas and to determine whether the competency definitions addressed important and clearly specified aspects of practice. Education Committee members and education staff reviewed validation results and appropriate follow-up analyses were completed. Competencies were modified based on this feedback.

Step 4

Working in small groups of 2 to 4 Education Committee members, a matrix was created that mapped all 2004 CAP education activities to the 6 general categories and the component competencies. To create the matrix, pathologists reviewed objectives specified for each course and indicated which category and competency that course addressed. In many cases, the pathologists had personal experience with the education and could further clarify which competency or competencies the course addressed. The completed map clearly illustrated competencies that were addressed by many activities or courses in the existing curriculum (strengths) and those competencies that were addressed by relatively few activities or courses (gaps).

Step 5

Information on current curriculum gaps and needs assessment results was used to identify and set priorities for appropriate additions to the CAP education curriculum beginning in 2005.

RESULTS

Table 2 is a comprehensive list that defines each category, its component competencies, and the knowledge and skill statements identified for each component competency. This list is the basis for all validation activities and the creation of a curriculum map.

Table 3 provides summary information from our validation survey (see step 3 in the "Materials and Methods" section). All competencies were perceived as important by survey respondents. Mean ratings ranged from 3.69 to 4.66 on a 5-point scale (1 indicating not important; 2, slightly important; 3, important; 4, very important; and 5, extremely important).

Table 4 gives a summary evaluation of course offerings, which was used to identify the gaps and determine priorities for new course offerings.

Table 5 illustrates the actual mapping of course objectives to MOC categories and component competencies.

COMMENT

In this report, we describe the process of understanding and clearly describing MOC competencies with appropriate knowledge and skill statements for the purpose of applying these competencies to the educational offerings provided by the CAP. Although we easily understood the familiar competency categories of medical knowledge and patient care, newly defined competency categories such as system-based practice and practice-based learning and improvement required considerable effort to understand and apply in a meaningful way to the practice of pathology. When we conducted validation studies, we learned that these categories were also ones in which pathologists lacked significant skill and practice. Thus, the process of defining knowledge and skill statements for these competencies became highly significant to our overall curriculum development for the future. A review of the medical literature confirmed that other medical specialties have found it challenging to develop curriculum offerings for these competency categories.7,10,12,15-21

The results of our evaluation of the CAP education curriculum indicate that although CAP has courses targeted in each of the 6 defined competency categories, the number of courses that address each defined competency varies widely. Determining if that mix of courses is appropriate and developing education offerings that address other important competency categories is now the focus of the CAP Education Committee and staff. Our goal is to create a curriculum that provides pathologists with opportunities to increase their knowledge and skills in each of the 6 MOC competency categories so that they will be prepared for successful completion of MOC requirements, once they are defined and communicated.

The results of our initial needs assessment survey indicate that pathologists believe they are relatively poorly prepared for practice in competency categories related to systems-based practice and practice-based learning and improvement. This finding is consistent with similar studies performed with residents and medical students that identify deficiencies in the preparation of physicians specifically in these areas and leadership and management competencies generally.8-10 Since our curriculum development is driven by needs assessment results, the CAP will focus on developing and delivering education courses that address these MOC competency categories, in addition to continued emphasis on medical knowledge and patient care. We will focus on the professionalism and interpersonal and communication skills components in future years, when and if we determine that pathologists need and want education with this emphasis.

To date, residency training programs provide the most complete set of requirements for the demonstration of competency within each of the categories.'1 We have considered these requirements in the development of our knowledge and skill statements. In addition, residency programs are required to have in place methods for assessing the achievement of competency defined by these requirements. The competencies specified for achievement of competency are the same as those specified for MOC. Consequently, in looking for educational approaches to specifying competency requirements or defining training options for skills development in each competency category, it is useful to review the literature from residency programs on outcomes of various approaches.7,10,17-19 It is likely that alternative approaches to curriculum development will be required for some competency categories.

Medical Knowledge

Most CAP postgraduate courses have traditionally focused on medical knowledge in anatomic pathology and clinical pathology. This is entirely appropriate, since this category represents knowledge that all competent pathologists must have regardless of the circumstances of their individual practices. The CAP will continue to focus extensively on developing and delivering education programs that keep pathologists current on scientific and technical advancements in medical knowledge.

The CAP expects that the ABP will take the lead in identifying curricular needs in this category using a content specification process. Currently, the ABP pathology content specifications are being developed by the 12 Test Development and Advisory Committees of the ABP. It is our understanding that a final official list of content specifications for medical knowledge will be released to all organizations involved in pathology CME, published in pathology journals, and placed on the ABP Web site (www.abpath.org). The CAP, as one of the cooperating societies, will be able to use this information to direct the content of the medical knowledge offerings.

Patient Care

This competency category emphasizes the integrative aspects of practicing medicine, in addition to purely scientific and technical knowledge. Patient care knowledge and skill statements focus on the provision of clinically relevant information. For example, in pathology practice these elements include the use of information technology, decision support integration, patient and colleague education, and performance of procedures on patients, such as fine-needle aspiration and bone marrow biopsies.22

The CAP has consistently emphasized the need for inclusion of patient care skill-building elements in all offered courses. The CAP will continue to work with faculty to integrate elements of patient care competency into course objectives. Such learning is important to pathologists irrespective of the circumstances of their practices. In many settings, practice protocols involving decision support that incorporates pathology and laboratory knowledge are commonplace and must be understood and used by pathologists. They are also increasingly advocated by regulators, such as the Joint Commission on the Accreditation of Healthcare Organizations.

Systems-Based Practice

Systems-based practice competencies focus on the understanding of how a pathology practice integrates with other medical systems and economic models of health care. Since practice models vary, an understanding of these concepts requires at least a basic knowledge of systems thinking.

Many other medical specialties have identified similar gaps in the knowledge of systems-based practice among their physicians. 19-21 This gap exists because these concepts have not typically been taught in medical school and have historically been neglected during residency training. The paucity of knowledge in systems-based practice has been addressed by the new publication on ACGME General Essentials in the Undergraduate Medical Essentials for the 21st Century (UME-21). One of the identified essentials is systems-based learning, in which several specific skills are defined that are mimicked by those we define in this report.23

Doezema et al19 described an approach to teaching and fulfilling systems-based practice requirements in an emergency medicine residency. As one means of understanding the relationship between the health care provided in the emergency department and the larger community, residents were assigned to complete a community project during their residency. These projects included the teaching of health care in schools, mentoring of students, domestic violence education, and providing health care to the homeless. Residents were required to prepare a written report and an oral presentation with a description of the relevant outcomes for satisfactory completion of this assignment. Evaluations were based on the report and presentation and on feedback from community members. In family medicine, Rivo et al21 reviewed 8 UME-21 schools and described various approaches to teaching systems-based care. They found considerable variability in the approaches used across the 8 schools but also found that the programs had been successful in exposing students to critical skills within systems-based practice. We believe that the same benefits could apply to those involved in the continuing education of practicing pathologists. In all cases where curriculum adaptations are suggested, the recommendation includes an increased focus on experiential learning opportunities.

Although the UME-21 curriculum includes didactic lectures, the Curriculum Committee found that experiential learning was also important. If medical students had hands-on involvement in projects that emphasized systems-based skills, they were more likely to retain this information and be able to use the skills in their daily practice. This finding is entirely consistent with known principles of adult learning and with education research findings.24-26 The ABMS has adopted similar goals for physicians seeking accreditation or reaccreditation and similar principles apply. Experiential learning for pathologists and residents in pathology is available through the CAP Laboratory Inspection Program, in which pathologists must address the relationship of the laboratory to the hospital in several areas, such as medical staff satisfaction with service and the utility of informatics solutions. The CAP Education Committee and staff are mindful of this important component of the inspection process and are working to increase the educational value of these experiences.

The didactic components of systems-based learning are being addressed through the design of a comprehensive program in laboratory practice management, which will be offered at the national meeting. A possible experiential format to be offered will include a problem-based, day-long session, where teams of pathologists and staff can come together with management professionals to work on problems they face in specific aspects of laboratory management.

Practice-Based Learning and Improvement

Practice-based learning and improvement competencies require that pathologists are able to conduct an analysis of their practice, assimilate external data regarding scientific literature or patient populations, and identify appropriate improvements to processes in place and measure results of these improvements. The UME-21 has also defined the curriculum for practice-based learning and improvement, another important educational focus, in its outcome project. The UME-21 argued that to address topics within this category, physicians need skill development in evidence-based medicine evaluation, outcome analysis, and clinical quality improvement methods.23

Practice-based learning and improvement topics may not seem immediately relevant to pathologists and may benefit from being included in the context of other learning content. For example, one such topic, which might be included in a clinical pathology session, is the discussion of how to set reference ranges in clinical laboratories, where laboratories are expected to set reference ranges for tests based on their own data. In anatomic pathology sessions, discussion can emphasize evidence-based guidelines and can include methods to measure the incorporation of new diagnostic information into clinical practice using practice outcome analysis. One way that the CAP is planning to address this competency category is through the use of outcome analysis tools, such as the CAP outcome templates, that have been created for a variety of topics in pathology. The templates can be readily accessed at www.cap.org. Application of these templates, among other efforts, will better prepare pathologists to design and lead patient care improvement efforts. Additionally, the CAP is planning to augment education course content by providing guidance and general tools for performing outcome evaluation to improve clinical practice.

Innovative Education Approaches

The CAP is committed to monitoring the effectiveness of our education program and will be guided by the results of rigorous, outcome-focused program evaluations. We will also perform periodic needs assessment through on-line surveys and focus groups to refine our curriculum. We recognize that our approach may require significant modification to be effective. Innovative educational approaches may be needed. Several authors report the success of such curriculum modifications.2,10 Providing learning opportunities for pathologists in practice like the ones used for resident education will be challenging.

The educational requirements of the MOC focus of the ABMS are extensive and far-reaching for the future pathologist, but they are also applicable to pathologists in practice with unrestricted certification. The CAP has focused our education efforts during the past year on identifying a process for defining and responding to the MOC challenge. We have learned that these competencies are perceived as important to the practice of all pathologists and thus justify curriculum offerings that will meet these needs. Pathologists, through needs assessment survey results and focus groups, have told us that they have a significant need for learning such concepts, in addition to their need to update their medical knowledge. We invite other pathology organizations to reflect on the process we have used and to share their reactions, comments, and insights. To the extent that CME providers can share information about addressing the MOC challenges, we can jointly build on the body of knowledge and communicate lessons learned and best practices. We look forward to the ongoing dialogue.

Members of the Education Committee include Charles L. Abbott, MD, Paul Bachner, MD, M. Elizabeth H. Hammond, MD, William F. Hickey, MD, Henry A. Homburger, MD, Peter J. Howanitz, MD, Rebecca L. Johnson, MD, Jay F. Schamberg, MD, Frances Edward Sharkey, MD, Thomas M. Wheeler, MD, Thomas L. Williams, MD, and Richard J. Zarbo, MD, DMD.

© 2005 College of American Pathologists Provided by ProQuest LLC. All Rights Reserved.