Does the Global Assessment of Functioning assess functioning? (Research).

Adaptive functioning/impairment is an important mental health outcome domain that is conceptually distinct from symptom severity. The Global Assessment of Functioning (GAF; American Psychiatric Association, 1994) is the most commonly used measure of adaptive functioning/impairment in mental health settings. We suspect that GAF scores may be influenced by factors other than functional impairment. In this study, three raters classified, with high reliability, the reasons given by eight clinicians for 80 GAF ratings. As expected, GAF ratings were strongly influenced by factors other than adaptive functioning/impairment, like symptom severity. The GAF is not a good measure of adaptive functioning, yet important decisions affecting clinicians and clients are made on the basis of GAF scores. Better measures of adaptive functioning are needed.

**********

The assessment of adaptive functioning and impairment is frequently underemphasized or overlooked altogether in mental health treatment outcome studies. Instead, most studies focus on changes in symptom severity (Krupnick, 1999; Mintz, Mintz, Arruda, & Hwang, 1992). This is unfortunate for several reasons. First, recognizing changes in adaptive functioning--the ability to lead an independent and productive life--is important if we are to understand mental health treatment from a biopsychosocial perspective rather than from the narrower biomedical model (Engel, 1977, 1980; Schwartz, 1982). The assessment of adaptive functioning allows us to look at skills and strengths that are developed through counseling in addition to abnormalities and deficits that are corrected. Second, changes in adaptive functioning and symptom severity may not always occur together. We might expect psychosocial interventions that emphasize coping, resiliency, and rehabilitation to have greater effects on adaptive functioning than psychotherapies and pharmacotherapies that target only symptoms. Failing to measure general functioning may underestimate the impact of these psychosocial interventions (Krupnick, 1999). In addition, the lukewarm response of many clinicians to controlled clinical trials may, in part, be due to their recognition that most efficacy studies leave this important aspect of clients' experiencing unmeasured (Addis, Wade, & Hatgis, 1999; Goldfried, 1999; Seligman, 1995). Finally, an exclusive focus on symptom change severely limits the ability to compare treatments and their cost effectiveness across diagnostic groups (e.g., how many hours of insomnia are equal to a delusion?). Adaptive functioning/impairment is a metric that can be applied more easily across groups of clients.

There is little consensus about the meaning of adaptive functioning or its flip-side, impairment (Brekke, 1992; Phelan, Wykes, Goldman, 1994). For clarity's sake, when discussing adaptive functioning, functional abilities or impairments, we will refer to a domain that includes work functioning and productivity, the ability to actively participate in social relationships and to manage appropriate social roles, and the ability to care for one's daily physical needs (e.g., grooming, feeding, managing money). This use of functional impairment represents a very different outcome domain than symptom severity. Symptoms typically refer to how well parts of a person are working. Examples of symptoms include mood states (e.g., depression, anxiety, euphoria), other internal states (e.g., low self-esteem), physical abnormalities (e.g., insomnia, poor appetite, psychmotor agitation), and cognitive difficulties (e.g., aphasia, distractibility, obsessions). Functional impairment and adaptive functioning focus on what individuals can do, the quality of their daily activity, and their need for assistance. Adaptive functioning emphasizes the integrated behavior of whole people. Examples of functional difficulties include poor school performance, relationship difficulties, trouble with the law, neglected parental responsibilities, the inability to work, and deficits in grooming.

It makes conceptual sense to consider symptoms and adaptive functioning independently, although they are related and sometimes overlap. Imagine, for example, an anxious client whose symptoms are reduced through medication or relaxation techniques, but who continues to have strained social relationships and difficulty maintaining employment after treatment. In most clinical trials, this symptom-responder would be considered a treatment success. In contrast, consider a second client who receives treatment for an anxiety disorder but continues to endure occasional high levels of anxiety. As part of counseling, she develops the skills to cope with her anxiety and returns to a high level of productive activity at home and work, in spite of residual symptoms. In clinical trials that focus exclusively on symptom change, she would likely be judged a treatment failure despite significant changes in her level of functioning.

Empirical studies suggest that this is not a trivial distinction and that symptomatic and functional outcomes deviate frequently enough to warrant measuring them separately. Behavioral interventions for chronic pain, for example, aim to increase adaptive functioning in clients whose impairments often exceed what is expected given their symptom severity (Fordyce, 1976; Sternbach, 1974; Turk, 1994). In the psychiatric literature, several studies suggest that the timing of changes in symptoms and functioning may be quite different (Gordon, Plutzky, Gordon, & Gerra, 1988; Howard, Lueger, Maling, & Martinovich, 1993; Mintz et al., 1992). Studies like these suggest that symptom severity and functional impairment are not redundant measures and that the relationship between the two is neither direct nor simple.

The Global Assessment of Functioning scale (GAF) is one of the most widely used measures of impairment and functioning in clinical and research settings (Basco, Krebaum, & Rush, 1997). The GAF serves as Axis V in the Diagnostic and Statistical Manual of Mental Disorders (4th ed.; American Psychiatric Association, 1994) multiaxial classification system for psychiatric disorders. It represents a revision of the Global Assessment Scale (GAS; Endicott, Spitzer, Fleiss, & Cohen, 1976), which itself derives from Luborsky's Health-Sickness Rating Scale (1962). Clinicians rate clients on a 1 to 100 scale in terms of their psychological, social, and occupational functioning. The scale includes 10 sets of anchor descriptions spaced at 10-point intervals. Anchors allow clinicians to consider both symptom severity and social/occupational functioning in making their ratings. For example, the anchor for the 51-60 range reads "moderate symptoms OR moderate difficulty in social, occupational or school functioning" (APA, 1994, p. 32).

The expressed purpose of multiaxial diagnosis in DSM-IV (1994) is to assess clients along several dimensions, each of which contributes unique information. Axis V, the GAF, was designed to measure general functioning which should predict treatment response and assist in treatment planning. Several authors, however, have challenged its usefulness in predicting future outcomes (Goldman, Skodal, & Lave, 1992; Moos, McCoy, & Moos, 2000). Our concern with the GAF is even more fundamental with respect to its use in clinical treatment settings and in clinical outcome research: Does the GAF measure what it purports to measure--functioning?

A number of authors have expressed concern that the GAF combines symptom and impairment ratings into a single scale and that this could affect the interpretation of GAF scores (Goldman et al., 1992; Phelan et al., 1994; Skodal, Link, Shrout, & Horwath, 1988). Some have questioned whether the GAF measures functioning at all, pointing to evidence that GAF ratings sometimes correlate more strongly with ratings of symptom severity than with other measures of impairment (Moos et al., 2000; Skodal et al., 1988). It is possible that clinicians base their GAF ratings mostly on observations of client functioning, in which case concerns about confounded measurement might be moot. But do they? Although clinicians are given the option of considering symptom severity when making GAF ratings, we do not know the extent to which they exercise this option. If ratings are strongly influenced by symptom severity, the GAF becomes a poor measure of adaptive functioning, and any relationships between the GAF and other process or outcome measures become hopelessly confounded with symptom severity.

The purpose of the current study is to determine the extent to which clinicians consider symptom severity and functional impairment when making GAF rating decisions. Clinicians will rate clients on the GAF then describe the reasons for their ratings. Their reasons will be classified as to whether they reflect symptom severity, functional impairment, both or neither. If the GAF is truly a measure of impairment or adaptive functioning, as its name implies, we would expect clinician's to base their ratings predominantly on observations of patient functioning. If, however, the GAF is a weaker measure of the impairment construct, we would expect clinicians' ratings to reflect other strong influences as well. We predict that GAF ratings will reflect clinicians' judgments about impairment, but will also be strongly influenced by symptom severity.

METHOD

Participants

Twenty-eight licensed practicing clinicians (psychologists, marriage and family therapists) from a variety of agencies and private practices in Kern County, California, were invited by letter to participate in this study. Kern County is an ethnically diverse county of 650,000 located in California's central valley, 100 miles north of Los Angeles. Eight clinicians agreed to participate; their characteristics are shown in Table 1. Participating clinicians were not aware of the purpose of the study. For their participation, clinicians received a payment of $50.

Procedure

Clinicians were asked to complete brief summaries for 10 clients whom they had recently interviewed and diagnosed using DSM-IV (1994). The summaries included each client's DSM-IV Axis I and Axis II diagnosis, age, gender, and Global Assessment of Functioning (GAF, Axis V) rating. In addition, clinicians listed, in a free-response format, the three most important reasons for each GAF rating they made, with their most important reason listed first. Eight clinicians, each contributing 10 cases, generated 80 GAF ratings. Three reasons for each GAF rating would have created 240 reasons, however, one clinician provided only two reasons for one of her GAFs, reducing the number of reasons to 239.

Two undergraduate research assistants with no formal training in counseling and a licensed clinical psychologist with 10 years of experience served as judges who sorted each of the 239 reasons into one of four categories: (a) symptom; (b) functional impairment; (c) symptom and functional impairment; (d) description of status, circumstances, history, or environment without reference to symptoms or impairment (a residual category). Judges received an outline of definitions for each category (consistent with discussion earlier) and saw several examples for each category. Some of these examples are shown in Table 2. The symptom and impairment category was used only when both categories were clearly implied in a single reason provided by a clinician.

RESULTS

The 80 clients whose GAF ratings were evaluated in this study were predominantly female (72.5%) with a mean age of 37.3 (SD = 15.0). Table 3 shows the Axis I diagnoses for these clients. These diagnoses were consistent with an outpatient treatment setting; severe psychopathology (e.g., schizophrenia) and severe impairment were not well represented. As expected, more severe impairment was associated with more serious DSM-IV (1994) diagnoses. Eight clients received Axis II diagnoses, but in all cases the Axis I diagnosis was primary.

The interrater reliability for judges' sorts of GAF reasons was evaluated using the kappa statistic, a coefficient of agreement representing the proportion of agreement between judges after chance agreements have been removed (Cohen, 1960). Kappas for the judge1-judge2, judge2-judge3, and judge1-judge3 sorts were .65, .70, and .69, respectively, suggesting substantial agreement among judges.

Table 4 summarizes the results of these sorts. When we examined the primary reasons given by clinicians for their GAF ratings, we found their decisions influenced most by symptom severity, followed by functional impairment and the combination of the two. When all three reasons were considered--a broader measure that included important and less important reasons together--GAF ratings were influenced about equally by symptom severity and functional impairment.

DISCUSSION

As we predicted, clinician's GAF ratings were strongly influenced by factors other than functional impairment. This is not surprising since clinicians are permitted, in fact instructed, to consider symptom severity when making GAF ratings. This study demonstrates that clinicians are indeed strongly influenced by symptom severity, at least when rating outpatient clients, rendering the GAF a confounded and limited measure of adaptive functioning/impairment.

We are not the first to point out limitations of the GAF. Our study simply adds to the body of evidence arguing against the use of the GAF as a measure of functioning. The GAF, however, has been surprisingly resilient. Its predecessor, the GAS (Endicott et al., 1976), was used in the influential NIMH Treatment of Depression Collaborative Research Program (Elkin et al., 1989) and the GAF continues in clinical trials today.

Our results may be limited by our small sample of clinicians from a single, albeit diverse, county. Most of our clinicians were master's-level counselors. To the extent that length of graduate training affects GAF ratings, a group of psychiatrists or doctor-level psychologists may have responded differently. It is worth noting, however, that our predominantly female, middle-aged, mostly master's-trained counselors were quite representative of mental health professionals who use the GAF on a regular basis. These clinicians were not selected for their expertise in the GAF, although most had some training and considerable experience with the GAF. In spite of these limitations, we saw no evidence that our clinicians were less than fully competent in the way they administered the GAF; on the contrary, they considered both symptom severity and functional impairment when making their ratings, just as the scale instructs them to do. Our point is not that the incompetent use of the GAF creates measurement problems; rather, the GAF is an inadequate measure of adaptive functioning when it is used in the way it was designed to be used.

It may be argued that we have misrepresented the purpose of the GAF, that it was never meant to be a pure measure of functional impairment. That may be true. Nevertheless, the GAF is frequently used as a pure measure in clinical settings and in outcome studies. In clinical settings, we would expect Axis V and Axis I (and II) to be correlated due to the positive, though imperfect, relationship between symptom severity and impairment; we would hope, however, that the correlation was not due to confounded measurement. Likewise, in clinical outcome studies that use the GAF (or GAS), it is typically listed as a measure of functioning, not as a measure of roughly how patients are doing in a general way that may be influenced by symptoms, functioning and a variety of other unspecified factors.

Given the importance of measuring functional outcomes, what alternatives to the GAF are currently available? Instead of the GAF, clinicians and researchers could include various objective indicators of functioning--like employment status, number of arrests, or school classes failed--in their clinical evaluations and outcome studies. Brief instruments that are part of a standardized battery, however, have the advantage of being more easily compared and their findings more easily integrated across clinical evaluations and larger outcome studies (Horowitz, Strupp, Lambert, & Elkin, 1997). Phelan et al. (1994) have developed a list of criteria for an ideal scale of this sort. First, an ideal scale should be appropriate for patients with different levels of impairment and different origins for their impairment (e.g., different diagnoses). In addition, an ideal scale of functional abilities should be applicable across a wide range of cultural backgrounds and should be reliable, valid, and sensitive to change. Finally, an ideal scale should be convenient and should require little additional effort on the part of clinical staff to use it. To this list we add that an ideal scale should measure impairment globally, across one, two, or three major dimensions of impairment (e.g., social functioning, occupational functioning) as well as specifically across a larger number of smaller dimensions (e.g., money management, intimate relationships). Recent reviews of the literature suggest a dearth of good impairment measures that fit these criteria (Goldman et al., 1992; Krupnick, 1999; Phelan et al., 1994). Nevertheless, we are hopeful that as the limitations of current impairment measures become more clear, investigators will be challenged to create better measures of impairment, as we have begun to do (Bacon & Bacon, 2001).

In the short term, the limitations of the GAF should have little impact on the day-to-day work of individual counselors. In order to receive insurance reimbursement, counselors will continue to formulate complete DSM-IV (1994) diagnoses, including GAF ratings, for each of their clients. Until better measures are available, counselors can do this with a clear conscience since there is no ethical obligation to abstain from nonoptimal measures. Nevertheless, mental health counselors should be concerned. Results from clinical outcome studies are used by managed-care companies to set practice standards and quality guidelines, and as the trend toward accountability for clinical outcomes continues to grow, practitioners will be under increased pressure to document the effectiveness of their interventions (Lawless, Ginter, & Kelly, 1999; Pistole, 1995). Counseling interventions are likely to affect independence and the quality of day-to-day living as well as symptom severity. To the extent that these important dimensions of client functioning go unmeasured, the impact of successful counseling may be underestimated. Counselors who understand this should endeavor to educate the public, their colleagues, and other mental health stakeholders. At the same time, counselors should keep their eyes open for better measures of impairment that more fully demonstrate the effectiveness of their interventions.

Table 1. Characteristics of Clinicians Participating in the Study

Characteristic                                  M     SD     %

Gender (% female)                                           87.5
Ethnicity
  White                                                     87.5
  Native American                                           12.5
Degree
  Ph.D.                                                     37.5
  Master's                                                  63.5
Age                                            52.4   8.0
Years in practice                              12.0   8.0
Training in GAF
  Graduate courses                                          75.0
  Workshops                                                 62.5
  Supervised training after graduate school                 62.5
  Discussion with colleagues                                62.5
  Professional newsletters/magazines/books                  25.0
  Research journals                                         25.0
  Research project                                          12.5
Table 2. Categories of GAF Reasons

Reason for GAF Rating           Examples

Symptom                         Depression, worry, insomnia,
                                obsessions, memory loss

Functional Impairment           Poor school performance Communication
                                in marriage is poor Neglects
                                children during binge drinking

Symptom and                     Mood has improved and client has
  Functional Impairment         returned to work Appetite has
                                improved and there is improvement in
                                general functioning Client is
                                withdrawn and suicidal

Other (Status, Circumstance,    Client is an adult child of
  History, or Environment       an alcoholic
  Without Reference to          Client's husband recently died
  Symptoms or                   of cancer
  Functional Impairment)        Client is prescribed Valium
  Symptoms or                   Client has incest issues
Table 3. DSM-IV Axis I Diagnoses and Mean GAF Ratings

DSM-IV Diagnosis              n *     M      SD

Adjustment Disorder           29    61.59    6.55
Major Depression              23    54.22    8.26
Anxiety Disorders             11    56.36   11.06
Dysthymia                      9    59.00    9.89
Bipolar Disorder               5    52.40    2.51
Alcohol Dependence             3    58.00    8.00
Eating Disorders               2    53.00    2.83
Pain Disorder                  2    54.50    9.19
Attention Deficit/
  Hyperactivity Disorder       2    56.00   5.66

Note: * Total n is greater than 80 because
6 clients received two primary Axis I diagnoses.
Table 4. Classification of Reasons for GAF Ratings

                                Percentage of GAF Reasons Classified

Reason for GAF                   Primary reason        All three
Rating                                only (a)          reasons (b)

Symptom                               44.2                 36.3
Functional Impairment                 22.9                 37.2
Symptom and
  Functional Impairment               20.4                 12.6
Other (Status, Circumstance,
  History, or Environment
  Without Reference to
  Symptoms or
  Functional Impairment)               12.5                 13.9
Total                                 100.0                100.0

Note. Percentages are averaged across the three raters.
(a) n = 80. (b) n = 239.

REFERENCES

Addis, M. E., Wade, W. A., & Hatgis, C. (1999). Barriers to dissemination of evidence-based practices: Addressing practitioners' concerns about manual-based psychotherapies. Clinical Psychology: Science and Practice, 6, 430-441.

American Psychiatric Association. (1994). Diagnostic and statistical manual of mental disorders (4th ed.). Washington, DC: Author.

Bacon, S. F., & Bacon, N. M. K. (2001). Scale of Functional Ability Ratings (SOFAR). Unpublished manuscript, California State University, Bakersfield.

Basco, M. R., Krebaum, S. R., & Rush, A. J. (1997). Outcome measures of depression. In H. H. Strupp, L. M. Horowitz, & M. J. Lambert (Eds.), Measuring patient changes in mood, anxiety, and personality disorders: Toward a core battery (pp. 191-245). Washington, DC: American Psychological Association.

Brekke, J. S. (1992). An examination of the relationships among three outcome scales in schizophrenia. Journal of Nervous and Mental Disease, 180, 162-167.

Cohen, J. A. (1960). Coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20, 37-46.

Elkin, I., Shea, M. T., Watkins, J. T., Imber, S. D., Sotsky, S. M., Collins, J. E, Glass, D. R., Pilkonis, P. A., Leber, W. R., Docherty, J. P., Fiester, S. J., & Parloff, M. B. (1989). National Institute of Mental Health Treatment of Depression Collaborative Research Program: General effectiveness of treatments. Archives of General Psychiatry, 46, 971-982.

Endicott, J., Spitzer, R. L., Fleiss, J. L., & Cohen, J. (1976). The Global Assessment Scale. Archives of General Psychiatry, 33, 766-771.

Engel, G. L. (1977).The need for a new medical model: A challenge for biomedicine. Science, 196, 129-136.

Engel, G. L. (1980). The clinical application of the biopsychosocial model. American Journal of Psychiatry, 137, 535-544.

Fordyce, W. E. (1976). Behavioral methods in chronic pain and illness. St. Louis, MO: Mosby.

Goldfried, M. R. (1999). The pursuit of consensus in psychotherapy research and practice. Clinical Psychology: Research and Practice, 6, 462-466.

Goldman, H. H., Skodal, A. E., & Lave, T. R. (1992). Revising axis V for DSM-IV: A review of measures of social functioning. American Journal of Psychiatry, 149, 1148-1156.

Gordon, R. E., Plutzky, M., Gordon, K. K., & Guerra, M. (1988). Using the axis V scale to evaluate therapeutic outcome of psychiatric treatment. Canadian Journal of Psychiatry, 33, 194-196.

Horowitz, L. M., Strupp, H. H., Lambert, M. J., & Elkin, I. (1997). Overview and summary of the core battery conference. In H. H. Strupp, L. M. Horowitz, & M. J. Lambert (Eds.), Measuring patient changes in mood, anxiety, and personality disorders: Toward a core battery (pp. 11-54). Washington, DC: American Psychological Association.

Howard, K. I., Lueger, R., Maling, M., & Martinovich, Z. (1993). A phase model of psychotherapy: Causal mediation of outcome. Journal of Consulting and Clinical Psychology, 61, 678-685.

Krupnick, J. L. (1999). The assessment of functional outcomes in cost-effectiveness of psychotherapy studies. In N. E. Miller & K. M. Magruder (Eds.), Cost-effectiveness of psychotherapy: A guide for practitioners, researcher, and policymakers (pp. 182-193). New York: Oxford University.

Lawless, L. L., Ginter, E. J., & Kelly, K. R. (1999). Managed care: What mental health counselors need to know. Journal of Mental Health Counseling, 21, 50-65.

Luborsky, L. (1962). Clinicians' judgments of mental health: A proposed scale. Archives of General Psychiatry, 7, 407-417.

Mintz, J., Mintz, L. I., Arruda, M. J., & Hwang, S. S. (1992). Treatments of depression and the functional capacity to work. Archives of General Psychiatry, 49, 761-768.

Moos, R. H., McCoy, L., & Moos, B. S. (2000). Global Assessment of Functioning (GAF) ratings: Determinants and role as predictors of one-year treatment outcomes. Journal of Clinical Psychology, 56, 449-461.

Phelan, M., Wykes, T., & Goldman, H. (1994). Global function scales. Social Psychiatry and Psychiatric Epidemiology, 29, 205-211.

Pistole, M. C. (1995). Publishing research in the Journal of Mental Health Counseling. Journal of Mental Health Counseling, 18, 307-311.

Schwartz, G. E. (1982). Testing the biopsychosocial model: The ultimate challenge facing behavioral medicine? Journal of Consulting and Clinical Psychology, 50, 1040-1053.

Seligman, M. E. P. (1995). The effectiveness of psychotherapy: The Consumer Reports study. American Psychologist, 50, 965-974.

Skodal, A. E., Link, B. G., Shrout, P. E., & Horwath, E. (1988). The revision of axis V in DSM-III-R: Should symptoms have been included? American Journal of Psychiatry, 145, 825-829.

Sternbach, R. A. (1974). Pain patients: Traits and treatments. New York: Academic Press. Turk, D. C. (1994). Perspectives on chronic pain: The role of psychological factors. Current Directions in Psychological Science, 3, 45-48.

Steven F. Bacon, Ph.D., is an assistant professor, Department of Psychology, California State University, Bakersfield. Email [e-mail]. Michael J. Collins is a graduate student at California School of Professional Psychology-Los Angeles at Alliant International University. Edmund V. Plake is deputy director, Community Support Services, National Training Center, United States Marine Corp, Quantico, VA.

This research was supported by a University Research Council Grant from California State University, Bakersfield.