Avoid the perils of 'instrument abuse': good tools badly used can harm clients.
ABUSE: to put to a wrong or improper use.
No matter how well-constructed, an instrument can be "abused" or misused. This typically takes the form of using the instrument for purposes or with populations for which it is not designed.
In some cases, this results from the confusion between assessment functions such as screening vs. determining a diagnosis. In other cases, the problem may stem from a failure to understand the appropriate uses for which a given instrument is designed, such as using a program evaluation tool for conducting a clinical intake. This problem takes on its most egregious forms when a particular instrument is mandated for an inappropriate use.
One of the classic and pervasive misuses of instruments consists of using screens instead of diagnostic instruments for clinical intakes and treatment planning. This would be analogous to an oncologist using results from a mammogram or colonoscopy in recommending a mastectomy or colon surgery for a possible malignancy, instead of using a definitive biopsy for developing the treatment plan.
The proper function of a screen is to determine level of risk for a given condition. A screen is basically a quick, easily administered and inexpensive means of identifying the probability that an individual is likely to have a given condition or disorder. This is sometimes referred to as "ruling in" (likely that the individual has the specified condition or disorder) or "ruling out" (unlikely that the individual has the specified condition or disorder). If the condition is "ruled in," further assessment is required to make a definitive determination of whether or not an individual actually has the condition in question.
Public-domain screens such as the CAGE, AUDIT, DAST, UNCOPE, TWEAK, and MAST are attractive to many because there is no charge for their use. However, all screens are not equivalent.
The AUDIT was developed by the World Health Organization (WHO) to "identify persons with hazardous and harmful patterns of alcohol consumption." It was not designed to detect diagnostic conditions, and may be more appropriate for prevention rather than clinical applications. The CAGE has been criticized as lacking accuracy when used with women or some ethnic groups, thus calling into question blanket recommendations for use across all populations. Of the screens mentioned above, only the UNCOPE was initially designed to detect dependence and abuse of both alcohol and drugs.
In addition to predating the current diagnostic criteria, the MAST suffers from a multitude of problems that make it an inappropriate screen in its original form. For example, the item "Have you ever attended a meeting of Alcoholics Anonymous (AA)?" was originally considered to warrant a score of five on the MAST. However, many professionals in the field would score positive on this item even if they have no alcohol abuse or dependence issues. Items such as the inquiries about losing a job and having delirium tremens (DTs) or cirrhosis tend to be very late-stage symptoms. A screen should pick up problems that occur early in the development of a condition. Thus, while the MAST does contain some good screening items, the instrument is both too long and inappropriate as a screening tool and outmoded as a diagnostic instrument.
For an individual presenting at intake, a determination of risk has already been made, so screens of any kind are not appropriate instruments for use. Likewise, screening a DUI offender is redundant. What is required for intake is a diagnostic interview to assist the clinician in documenting whether abuse or dependence diagnoses by substance apply to the individual case.
The ASAM PPC-2R requires a diagnosis for either abuse or dependence for admission to outpatient treatment. A residential or inpatient admission requires a diagnosis of dependence on one or more substances. Research has shown that dependence on a substance such as alcohol is not simply a more advance form of abuse, but manifests itself distinctly from abuse and has a differential prognosis. This means that conducting an intake requires determining not just whether the individual is likely to meet criteria for a substance use disorder, but verifying that there is compelling evidence that the diagnosis is either abuse or dependence.
Perhaps the single most prevalent case of misusing an instrument involves the Addiction Severity Index (ASI).
The ASI is probably the most widely used program evaluation instrument cited in clinical research on treatment outcomes for addictions. While it is a fine outcome evaluation tool, problems arise when it is inappropriately promoted as a screen, a diagnostic tool, an intake instrument, or a treatment placement tool--even though it is not designed to be any of these.
Why is the ASI not an appropriate screening, diagnostic, or treatment placement tool? Let's begin with the issue of screening. The proper administration and scoring of the ASI takes the better part of an hour by a clinician or a very well-trained technician. It does not fit the description of a screen, nor is it designed as such.
The ASI does not assist in determining a diagnosis. The primary ASI measure in the alcohol and drug section is days of use in the previous 30 days. The only items for possible diagnostic documentation are two questions: one about having DTs and the other about overdosing on drugs. All the other questions are of a subjective nature, such as, "Which substance is the major problem?" and "How troubled or bothered have you been in the past 30 days ...?" Clearly, the ASI is relevant only if a diagnosis has already been determined.
Given that the individual meets criteria for substance dependence, days of use can be a very meaningful measure of the extent to which a treatment program has been able to assist the individual in achieving abstinence or reducing use. However, in the absence of such a diagnosis, days of use can be of little value or even misleading. For example, an individual who has one glass of wine with dinner each evening would score 30/30 on the alcohol use item. In contrast, a binge drinker who drinks excessively only on weekends and experiences a host of problems indicating dependence might score only 12/30. This illustrates how a respected program evaluation tool can be misused as a diagnostic instrument.
Treatment planning and placement requires determining the immediacy of various needs, the intensity of services required, and the extent to which treatment requires any type of restrictive environment such as a residential placement. The most commonly used criteria for treatment placement and structure for treatment planning are the criteria (ASAM PPC-2R) of the American Society of Addiction Medicine. These criteria stipulate six dimensions for assessment. The ASI does not address these dimensions in a way to facilitate treatment placement or planning.
Although the ASI asks about days of use, this does not necessarily provide information on the recency or quantity of use so as to address concerns about intoxication or withdrawal (Dimension 1). The ASI does ask whether the individual has a chronic medical problem and how "troubled" the person is by medical conditions, but does not address the question of whether these conditions require services or are as stable as they might be (Dimension 2).
The psychiatric section of the ASI does ask about being troubled by "serious" depression, anxiety and tension, but does not provide indications as to whether these are of a nature requiring services or if they are likely to interfere with treatment (Dimension 3). Even the questions on suicide do not address whether this is a current concern. There is no coverage concerning stages of change other than a question about how "important" treatment is for the individual (Dimension 4).
Relapse risk or risks for continuing problems are not covered (Dimension 5). The social and family sections of the ASI cover whether other family members have had problems, as well as a number of relationship questions. In general, the questions do not provide much insight as to whether family or friends will be an asset or liability in the treatment or recovery effort (Dimension 6).
In short, the ASI questions touch upon most of the six dimensions, but do not provide the type of information required to develop a treatment plan or justify a placement.
Another general misconception about the ASI is that since it is a public-domain instrument, it is free. It is free in that anyone can copy the items and use them without paying anyone for the rights to use. However, done properly, the ASI requires about an hour for administration and scoring--this time is not free. In a clinic, it adds an hour of salary and overhead to the cost of an evaluation; for the private practitioner, it is an hour that may not generate additional revenue. Also, there is the burden on the client. It adds an hour to the assessment process and may interfere with getting information relevant to case disposition.
For some states, the potential value of the ASI in monitoring the outcome of treatment has resulted in mandates requiring its routine use. This may be justified if the state also monitors outcomes. Otherwise, more appropriate instruments exist to provide clinically relevant information for screening, diagnostic determinations, and treatment planning and placement.
When considering mandates with clinical intent, the most prudent approach for state agencies, other governmental bodies, or entities such as third-party payers is to mandate the information to be collected along with the specifications for what it is to include, rather than mandating a specific instrument. For example, a mandate for diagnostic information might state that whatever instrument is used, it must be based on the current accepted diagnostic criteria.
In the case of treatment placement, the same type of mandate might stipulate that the dimensions supporting a given placement be documented in accordance with the criteria used in that state. Another approach is to stipulate the assessment requirements and to establish a list of instruments that meet those requirements.
By now, it should be abundantly clear that "instrument abuse" can occur through the inappropriate use of good tools. Misuse of assessment instruments can have serious consequences for those afflicted with substance use disorders and those professionals providing treatment services. Instrument abuse can result in inappropriate care for the client and may expose the clinician to professional liability claims.
While it is imperative that state and federal agencies carefully consider assessment mandates so as to promote appropriate practices, all professionals have the responsibility to be fully informed about the instruments they use and the strengths and weaknesses of each.
Norman G. Hoffmann, Ph.D., a clinical psychologist, has been evaluating treatment effectiveness and developing a broad range of practical clinical assessment instruments during the past 25 years. He is president of Evince Clinical Assessments, and can be reached at firstname.lastname@example.org.
Gerald Shulman is a clinical psychologist and master's-level addiction counselor with over 40 years of experience in addictions. He is a trainer and consultant whose specialties include treatment planning and assessment. He is president of Shulman & Associates, and can be reached at GDShulman@aol.com.
Hoffmann and Shulman were co-authors of an article on partnerships between addiction and child welfare professionals in the March 2004 issue.
By Norman G. Hoffmann, Ph.D., and Gerald D. Shulman, M.A., M.A.C., FACATA
|Printer friendly Cite/link Email Feedback|
|Author:||Hoffmann, Norman G.; Shulman, Gerald D.|
|Date:||Sep 1, 2005|
|Previous Article:||2005 events.|
|Next Article:||A partnership in healing: women's treatment that considers childhood sexual abuse.|
|Where did my assets go?|
|Judges' group releases guidelines for protecting victims of family violence.|
|Balancing disclosure and the client relationship.|