"I would like to see the survey system evolve...." (interview with University of Michigan Institute of Gerontology professor and research scientist Brant E. Fries)(Interview)

Was OBRA '87 worth it? That's a question celebrating its 10th anniversary this year since Congress s adoption of the Nursing Home Reform Act and the subsequent creation of the OBRA survey process. The question remains as controversial as ever - not so much over the value of reform, but whether the Federal government's approach to achieving it is workable. The criticisms of the survey process can be summed up thusly: too much confusing and meaningless paperwork, too much inconsistent and unfair policing. Paradoxically, just when nursing homes were once again being lambasted by Federal officials and the media for large-scale "noncompliance" with OBRA standards, the August 1997 Journal of the American Geriatrics Society offered a set of scientific papers indicating that nursing homes are indeed improving under OBRA '87. The six papers disclosed significantly improved outcomes data for several resident care areas between 1990, when the Minimum Data Set had yet to be widely implemented, and 1993, after the MDS's Resident Assessment Instrument (RAI) and Resident Assessment Protocols (RAPs) were strictly enforced. Sampling data on approximately 4,000 elderly residents in 250 facilities in 10 states, the investigators found:

* a 28% decline in hospitalization, with no increase in mortality;

* reduced declines in ADLs, cognitive performance and social engagement;

* decreases in the prevalence of stasis ulcer and, particularly, of dehydration, which declined by 50%, sparing 18,000 residents nationwide the difficulties and dangers of this condition;

* fewer residents experiencing declines in vision and nutrition.

Interestingly, one condition for which there was no specific or related RAP - daily pain - showed an increase during the study period.

Questions remained, however, whether the OBRA process was in fact responsible. Three commentaries accompanying the studies acknowledged that, because of the studies' design, the case for OBRA was not proven. They noted the absence of information reflecting the nursing home practices and policies that led to the improvements. One commentary included the line, "What is really going on here?"

Recently, one of the lead investigators, Brant E. Fries, PhD, agreed to address questions about the studies to help clarify the significance of the findings for nursing home staff, as well as to offer his thoughts on the survey process. Questions were posed by Nursing Homes Editor Richard L. Peck.

Peck: What is the background of these studies - why were they done?

Dr. Fries: My coauthors - Catherine Hawes, John Morris, Charles Phillips and Vincent Mor - and I originally coauthored the RAI. HCFA asked us at the time to evaluate the performance of the RAI after a period of time. Some have suggested that this is like asking the fox to watch the sheep - but we were asked, and we did our best to make our evaluation as scientifically fair and accurate as possible.

Unfortunately, we couldn't do an "intervention" study which, in this case could have compared states that had implemented the RAI with states that hadn't because, of course, OBRA had been mandated nationwide. As a result, there was no control group, and we had to settle for a "before and after" design. Therefore, one can never eliminate other factors from having possible influence on the results. One cannot unequivocally say that the RAI was directly responsible for the changes observed. What you can do, though - and what we did - is try a variety of tests to see if things closer to what you're doing are more affected than things that are further away. And that is what we found.

Had facilities' medical records changed to reflect the RAI? They had. Were care plans more accurate? They were. Were residents' specific clinical problems more likely to have things in the care plans that addressed them? They were. Did the records reflect measurable improvements in specific clinical areas? They did.

In short, what we attempted to do with these studies is establish a causal path, showing all the pieces along the way. You get to a point where you see, say, a child standing next to broken crockery and you suspect that the child had something to do with it.

Peck: But how closely did these documented improvements reflect real improvements in nursing home practices?

Dr. Fries: We went beyond records review and had trained nurse-investigators interview administrators and DONs using a set of questions reflecting both positive and negative responses to the RAI. Two-thirds thought that the RAI had in fact improved their programs - and they felt that it increased paperwork (which, by the way, I think is a fair statement). Bottom line, though: Most felt they knew more about their residents than they did before. Also, my own belief that what we found was real was bolstered by the facts that we randomized sufficiently to get a representative sample and that conditions addressed by the RAI showed improvement, while those that were not addressed by the RAI did not

Peck: There is a significant degree of sentiment out there, though, that what these records reflect is paper compliance to avoid survey deficiencies. Some say nursing homes can improve on their own without the "paperwork." Your thoughts?

Dr. Fries: It's interesting. When my coauthors and I first devised the RAI, we had a wide spectrum of opinion as to whether this would really work. We ranged from complete and utter skepticism to strong belief. I would say that now it looks to all of us as though two-thirds of nursing facilities in this country seem to have made substantial improvements in their care since implementation of the RAI.

Yes, there is a solid core of facilities out there that, quite simply, should be put out of business. Maybe it's 10% - we really don't know. But we are certain that it is not, as some would have it, 80%. We are seeing that, for the most part, assessments are being done, they are accurate, they are moving.into care plans and outcomes are improving. In short, something is going on here.

Peck: But, as the commentator said, "What is really going on here?"

Dr. Fries: I agree. But let's look at the RAPs. These started with the thought that if you create a really good assessment but don't tell people what to do with it, nothing will happen. As it turns out, the RAPs are proving to be a sort of middle ground - some facilities are going far beyond them and doing some highly creative and effective things to improve their care.

For example, coauthors Morris and Mor recently conducted a regional meeting of nursing homes in which facilities experiencing very low rates of incontinence discussed how they got there (see "Quality Improvement Meetings," p. 36). Other facilities invited to this meeting were able to learn from and benefit from their colleagues' experience. This is how many of us think the process should really work - not just sanctions, but identifying good practices and showing others how to implement them.

Peck: You feel this should be done on a national basis?

Dr. Fries: It is obviously the way to go. What we have right now in the RAI is a very clear basis for that conversation to occur.

Peck: There are nursing homes, including several that have contacted our magazine directly, that feel that what they're seeing with the current survey system is quite the opposite - a "gotcha" process focused on punishment. Your thoughts?

Dr. Fries: I think part of the problem is that they are missing the point of the whole exercise. I recently had a note taped to my door from a nurse saying that she had learned I had been involved in developing the RAI, she didn't understand it, she didn't know what RAPs were, she was leaving the nursing profession and it was all my fault. What can I say? If you don't know what the RAPs are, of course you'll get nothing out of this process.

I don't think the government or some of the facilities themselves have done enough training on this. I'm sure that, in some facilities, the DON plopped down this document as some crazy government scheme that everyone had to fill out whether they liked it or not, and I can imagine how some staff members reacted to that. We have to do better training.

Peck: What was your reaction to HCFA's recent rejection of the South Dakota Medicaid waiver that would have injected a quality improvement collaboration between surveyors and facilities into the process, based on University of Wisconsin quality indicators?

Dr. Fries: I really don't understand HCFA's view on this. There was some innovation here, and innovation should be encouraged. I suspect that part of the problem is that folks are skeptical that good sampling can be done. Many people don't understand that there's a difference between sampling and bias. There's the thought that you're always going to pick all the wrong people, or all the right people, and get biased results. But with scientific sampling that's not possible. If you randomize appropriately, it's like a coin flip - the results average out after a point and you begin to see what's real and what isn't. A scientific sampling can give you very generalizable results with very small samples. Yes, we've all seen examples of bad sampling, but HCFA seems to feel that it can't be done any other way, and that's a misconception.

Personally, I would like to see the survey system evolve to a point where we can say to some facilities, "You're doing so badly that you might as well give us an office; we'll be here until you get it right," and to others, "You're doing a good job - see you in a couple of years and use RAI data to monitor what's happening." Combine this approach with the quality improvement meetings that I discussed earlier, and I think this will get us to where we want to be.