A measure of professionalism.

How do you measure professionalism in medicine?

[ILLUSTRATION OMITTED]

The National Board of Medical Examiners has taken on that problem with a pilot project at two medical schools and two residency programs. But the assessment they've come up with is not a standardized test. Instead, people who work together are observing--and evaluating--each other.

The project grew out of a conference on the assessment of professionalism the NBME sponsored nearly three years ago in Baltimore with the Association of American Medical Colleges. A task force eventually agreed on a set of 60 core professional "behaviors" to be assessed--from "accepts constructive feedback and modifies behavior appropriately" to "uses resources effectively to ensure optimal patient care."

Increasingly, experts in medical education say it is vital to inculcate professional values in doctors before they begin their careers. A study published last year by Maxine Papadakis, associate dean of student affairs at the University of California, San Francisco School of Medicine, found that students who received comments about unprofessional behavior while they were at the medical school were more than twice as likely to be disciplined later by the state medical board.

The NBME assessment works much like the "360-degree" evaluation common in the corporate world, where feedback comes not only from a boss, but from virtually every type of worker the person being tested regularly interacts with.

After observing a resident or medical student for two months, workers at each site--from nurses to supervisors--are asked to evaluate him on about 20 of the 60 behaviors. Then there is a series of questions: Would they want the person on their team? To care for a loved one?

While the residents and medical students know they're being evaluated, "they don't know who is observing them for what," says Stephen Clyman, executive director of NBME's Center for Innovation, which is overseeing the project. The field trial will end in June.

Why do it this way? "The assessment of professionalism is not going to be as straightforward as a sit-down multiple choice exam," says David Stern, an associate professor of internal medicine and medical education at the University of Michigan Medical School who has worked with NBME on the pilot project.

Professional behavior, he says, has to be measured in context. "The way you fill out a multiple-choice exam at two in the afternoon is not the same way you're going to respond to a nurse at two in the morning."

Some medical schools have been using their own 360-degree evaluations of professionalism for years, Stern adds. A key difference in the NBME project, he says, is that the survey results literally leave the institution to be processed, so that no respondent need worry that an administrator could reveal who said what about whom. When the results come back, they are identified only by the type of evaluator (nurses, fellow students or attending physicians, for example).

Unlike a typical test, there is no score at the end, and so far no standardized consequences for people whose professionalism skills are found to be lacking.

"Hopefully, that feedback information would be part of a conversation regarding professional development," Clyman says. "We believe strongly that it should be a local interpretation. We believe the service we're providing is in standardizing the vocabulary."

Clyman says it's too early to discern any patterns in the surveys. But he is hopeful the assessment eventually can be put to wide use. "I think everybody has been scratching their heads over how to do this in an objective fashion."

A patient survey has yet to be developed. Stern, whose book Measuring Medical Professionalism will be published in September, says capturing accurate and useful information about professionalism from patients can be difficult because they "tend to love their doctors." Their responses, he says, can be influenced "by who the patient is, not who the doctor is."

Some of the sites haven't received feedback from their evaluations yet, but they are already reaping information just from watching the process unfold.

Lauren DellaCava, field trial administrator for the project at Lankenau Hospital outside Philadelphia, was struck by the 60 to 70 percent return rate among those asked to observe and turn in surveys. Nurses in particular, she said, were excited about the opportunity to share what they see "in a constructive way."

At Lankenau, core faculty and chief residents are being evaluated along with first-, second- and third-year residents. DellaCava thinks the process will be beneficial to all.

"You can focus on what needs to be improved without making it personal," she says. "It's a great way to make something so subjective into something that's objective. I think it'll be very interesting to see people's behaviors once they see the results."

RELATED ARTICLE: Practicing Professionalism

Here are 10 of the 60 behaviors of professionalism being used to evaluate residents and medical students in the NBME pilot project:

[ILLUSTRATION OMITTED]

1 Accepts personal risk in provision of health care

2 Admits errors and assumes personal responsibility for mistakes

3 Demonstrates appropriate boundaries for interprofessional relationships

4 Elicits patients' understanding to ensure accurate communication of information

5 Maintains a positive attitude amidst increased and unanticipated additional work

6 Offers advice when appropriate

7 Requests help when needed

8 Responds promptly when paged or called

9 Signs over and ensures coverage of patients when unable to fulfill responsibilities

10 Upholds ethical standards in research projects and other scholarly activities

Kate Shatzkin is a staff writer for The Baltimore Sun newspaper in Baltimore, Md.