Agreement between Occupational Therapy Practice Framework classifications and occupational therapists' classifications.

Terminology is important in every profession. As bodies of knowledge grow and expand, a profession requires the development of a system to categorize and organize information to provide understandable and usable concepts. For example, classifications in biology have developed so that organisms and the characteristics of organisms can be identified, compared, and contrasted in useful and meaningful ways (Christiansen, 1994).

The field of occupational therapy has grown and developed over the past 80 to 90 years, and the methods, scientific research, and underlying knowledge base have greatly expanded (Nelson, 1997). Over the years, individual scholars or groups of scholars have attempted to organize and classify the profession of occupational therapy. Kielhofner's descriptions of the foundations of the profession (2004) and Mosey's description of the profession's configuration (1981) are attempts to conceptualize occupational therapy and articulate its core constructs clearly and consistently. The American Occupational Therapy Association (AOTA) also attempted to organize fundamental concepts within the profession of occupational therapy by developing Occupational Therapy Product Output Reporting System and Uniform Terminology System for Reporting Occupational Therapy Services (1979), also known as Uniform Terminology. This document was originally developed to create a uniform system for reporting occupational therapy services for reimbursement. Uniform Terminology underwent two revisions (AOTA, 1989, 1994), with an expansion of objectives. Uniform Terminology-III aimed to depict a common and consistent terminology for clinical practice, policy making, and education.

The only published research concerning Uniform Terminology was conducted by Borst and Nelson (1993). This study surveyed registered occupational therapists and asked them to both define and classify terms from Uniform Terminology. In the first part of the questionnaire, participants were given definitions of terms and then asked to match them with the correct terms, which were provided in a list format. The second portion of the questionnaire asked participants to classify terms into the appropriate Uniform Terminology categories, given the terms and their definitions. This study's results showed an imperfect degree of agreement between therapists and Uniform Terminology on definitions of terms (71.9% mean agreement, SD = 12.4) and a low level of agreement on classification of terms (34.9% mean agreement, SD = 11.7). Given the assertion that these terms and classifications were supposed to represent the "uniform" and foundational concepts of the profession, the results suggested that Uniform Terminology was not entirely effective in meeting its goals (Borst & Nelson, 1993).

Faced with a mandate to review Uniform Terminology-III, the AOTA Commission on Practice noted its omission of the concept of occupation; its exclusion of pertinent or significant occupational therapy terms; and its use of terms that were ambiguous, inaccurate, or improperly categorized (AOTA, 2002, p. 637). Youngstrom (2002) argued that in recent years there has been a steady transformation in the practice patterns of occupational therapy, including expanded service venues and an increasing importance placed on the therapeutic value of occupation in human life. According to Youngstrom, the Uniform Terminology documents did not speak to this expanding arena of occupational therapy and, as a result, a new framework was devised to "reaffirm and clarify what occupational therapy is all about" (p. 607).

The decision was made to abandon the Uniform Terminology series and to develop a new system. The Occupational Therapy Practice Framework: Domain and Process (the Framework) was the product of this endeavor, developed to "describe the domain that centers and grounds the profession's focus and actions" (AOTA, 2002, p. 609). The newly formed Framework was designed to categorize and organize the field of occupational therapy in a logically consistent manner that would be understandable and accessible to persons within the field as well as to external or outside readers (AOTA, 2002). The Framework is divided into two components: the Domain, which explains the profession's purpose, focus, and areas in which it assists people, and the Process, which explains the methods by which occupational therapy intervention takes place. Six main categories make up the domain portion of the document: areas of occupation, performance skills, performance patterns, context, activity demands, and client factors. These main categories are then divided into subcategories, sub-subcategories, sub-sub-subcategories, and so forth, with terms explained by either definitions or examples. For example, the category Areas of Occupation is divided into seven subcategories: activities of daily living, instrumental activities of daily living, education, work, play, leisure, and social participation. Within each subcategory are lower level terms; for example, Social Participation has subcategories of Community, Family, and Peer/friend, with definitions given. The hierarchy of the Process portion of the Framework is similarly organized into categories of Evaluation, Intervention, and Outcomes, with subcategories, definitions, and explanations given.

According to Nelson (2006), the domain section of the Framework has many logical errors that do not follow the general principles required in a system of definitions and classifications. For example, the category Context overlaps with entries from other categories (e.g., Activity Demands and Client Factors). The term tools is listed under both the category Context and the category Activity Demands. In the category Areas of Occupation, the subcategory Social Participation overlaps with Work and Education. Therefore, when should a work or school occupation be classified as Work or Education, and when should the occupation be classified as Social Participation? Other flaws in the Framework, according to this logical analysis, include the incompleteness of the Domain in addressing the subjective meanings of individuals. The only items that address the subjective sense of individuals are the spiritual and cultural subcategories of Context, and these are poorly described. According to Nelson (2006), the Framework has not met the criteria necessary for a systematic and complete classification of the profession of occupational therapy, and he predicted a lack of reliability among occupational therapists attempting to classify and use terms in the Framework.

A vital feature of professional terminology is the degree of agreement that constituents of the profession maintain about the nature of the concepts. Indeed, the chief task in developing a classification system of a profession's terminology is to select appropriate terms with precise and usable definitions on which most professionals agree (Bloom, 1956). A high degree of agreement is necessary for common understanding as well as the ability to pass on the information to future learners (Reynolds, 1971). For the Framework to be a valid reference and guide for the field of occupational therapy, it must be representative of practice occurring in the field and must be agreed on and recognized by clinicians in the field.

The current study investigates how occupational therapists categorize the basic, lowest-level terms found in the Occupational Therapy Practice Framework. The main research question is, What is the level of agreement between therapists and the Framework in terms of categorization of Framework terminology? In addition, data concerning therapists' knowledge and use of the Framework are presented.

This study is somewhat similar to Borst and Nelson's (1993) evaluation of Uniform Terminology. However, the current study exclusively focuses on the categorization of terms in a one-step method rather than use of a two-step process that considers definitions and categorizations separately. In this study, participants were given one task of classifying terms into the proper categories instead of two separate tasks (matching terms to their definitions, and then a second task of classifying terms into their categories). Studying participants' responses to the categorization task simultaneously assessed participants' knowledge of the terms as well as their places within the categorization structure.

Method

Participants

The study was ruled exempt by the institutional review board. Surveys were mailed to 200 registered occupational therapists, selected at random from the database of registered occupational therapists who are members of AOTA.

The participant information was received from AOTA's List Rental Service through e-mail transfer and then printed on the survey envelopes. A 60% response rate was predicted based on Borst and Nelson (1993).

Instrumentation

A two-page questionnaire was designed according to the Tailored Design Method (Dillman, 1999). Participants were asked to complete the pages in order (Page A first, then Page B) and to refrain from looking at the second page of the questionnaire until they had finished the first page. The first page listed 30 randomly selected terms from the Framework along the left side of the page and the six main categories of the Domain section of the Framework along the right side. Terms in the Framework included in the randomization were the lowest level terms with definitions.

For example, the category Client Factors is subdivided into Body Function Categories and Body Structure Categories. The next level of categorization (using Body Function Categories as the example) divides Body Function Categories into Mental functions, Sensory functions, Neuromuscular and movement-related functions, and so forth. From there, the category Mental Functions is further divided into Global mental functions and Specific mental functions. Below this level are the lowest level terms in this category, such as Consciousness functions, Orientation functions, Sleep, and so forth. These lowest level terms have definitions and therefore are included in the randomization list for the study.

The Domain category with the fewest levels of subcategories is Performance Patterns. This category is divided into three subcategories--Habits, Routines, and Roles. Habits is further divided into three subterms--Useful habits, Impoverished habits, and Dominating habits--which have definitions and are the lowest level terms included in randomization. However, Routines and Roles have no further subdivisions, and those two terms are the lowest level terms with definitions in their line of development; therefore, they were included in randomization.

Terms were randomized in a stratified way, with five terms per major domain category, so that each of the six domain categories had equal representation. A randomized selection of the 30 terms was conducted using a random numbers table, randomized both for terms selected from the Framework as well as the order in which terms appeared on the questionnaire.

Participants were asked to choose the proper category for each term and mark the blank beside each term with the corresponding category. They were told that there were no correct or incorrect answers, and the Framework was not mentioned in the survey or the accompanying letter. They were told only that the survey dealt with their ways of categorizing terms. To increase the response rate, the task was designed to be as simple as possible and to require the least amount of time possible. According to Fowler (2001), one can expect a higher response rate from participants by limiting the number of items as well as the number of question forms.

The second page of the questionnaire inquired about demographics and level of familiarity with the Framework. This page revealed that participants' responses would be compared to categorizations within the Framework. After learning that the study was based on the Framework, participants were asked not to change their answers. Rather, a pure evaluation of occupational therapists' conceptualization of occupational therapy was elicited. In addition, participants were asked not to record their responses on the categorization task and not to save the record.

Procedure

The survey materials were pilot-tested with 13 occupational therapy graduate students and 3 occupational therapy program faculty members to test for clarity, ease of use, and time. The argument for validity is that the 30 selected terms were chosen randomly and directly from the Framework; therefore, no bias was shown in selection of terms and no possibly biasing interpretations were made of the terms. Surveys were mailed to participants in July 2005. The mailing included a cover letter, the two interstapled questionnaire sheets, and a stamped return envelope.

Follow-up reminder postcards were sent to participants whose responses were not received within 2 weeks. When surveys were returned, blocks of three were formed according to the order in which the surveys were received, and one participant from every block was randomly selected for readministration of the same categorization task to test for stability of responses. The retest survey was sent out 14 days after receipt of the initial survey. The same procedure was used with follow-up reminder postcards sent after 2 weeks without receiving a response.

The first mailing had a low return rate (N = 72, 36%); therefore, with IRB approval, another mailing was conducted to increase the response rate. The survey forms and a cover letter asking for participation were sent to those occupational therapists on the original participant list who had not yet returned a survey. This mailing took place approximately 3 months after the original surveys were sent. The retest survey for stability was not administered to this group because additional requests would potentially become burdensome for participants. All data were collected between July 2005 and December 2005.

Analysis

To determine the level of agreement between each occupational therapist and the Framework, a kappa coefficient was computed. If all 30 terms were classified by the therapist in the same way as in the Framework classification, the kappa coefficient was equal to 1. Disagreements occurred when the therapist classified a term in a different category from the Framework; the more disagreements, the lower the kappa coefficient. Next, the central tendency and range of the kappa coefficients across all participants were computed as the best overall description of level of agreement. Percentages of agreement were computed to supplement the kappa coefficients. Percentages of agreement are easier to understand but do not account for chance agreement, as do kappa coefficients. Stability was assessed in a parallel way.

Results

All returned surveys with 28 or more of the 30 responses filled out were included in data analysis. One hundred surveys were returned, 94 of which had at least 28 responses, which resulted in a 47% (94/200) response rate. Six of the 94 usable surveys were not complete with all 30 responses and had one or two items blank. These surveys were not penalized for blank answers; analysis was based solely on responses that were given.

Participants' mean number of years of practice was 11.5 (SD = 3.8). Mean number of work hours per week was 28.4 (SD = 15.3). Twenty-two percent of participants classified their area of practice as physical disabilities, 13% as school systems, and 12% each as gerontology or sensory integration.

The remainder varied across 13 practice areas. Participants also were asked questions concerning their level of familiarity with the Framework (see Table 1). The main result of the study is that the median agreement between occupational therapists and the Framework categorizations was .17 (see Table 2), suggesting slight agreement, according to Landis and Koch (1977). The median was selected as the best measure of central tendency because the data were positively skewed, with just a few respondents showing high levels of agreement with the Framework (see quartile scores in Table 2). Of 30 possible responses, the median number of agreements was only 9; hence, the median percentage of disagreement was 70%.

Twenty-five retest surveys to investigate stability were sent, and 18 surveys were returned with 28 or more responses (one returned survey had 23 responses and was not used in data analysis), resulting in a return rate of 72%. The median agreement between participants' first and second tests was .30, considered fair agreement according to Landis and Koch (1977). Still, only a median of 13.5 of 30 items was classified the same way at a later date by these 18 participants (see Table 3).

Discussion

The level of agreement comparing Framework classifications to classifications by the surveyed practicing therapists is slight, with a median kappa coefficient across all participants of .17. To be considered a substantial level of agreement, the kappa coefficient would have to be in the range of .60 to .80. No participant agreed with Framework classifications in 30 of 30 items. The median percentage of agreement was only 30%, indicating much more disagreement than agreement.

Some disagreements might have been due to participants having different viewpoints on how terms should be classified, and other disagreements might have been due to unfamiliarity with the terms altogether. One participant commented, "I was trying to help you, but I know my responses are not accurate." Another commented, "Most of these were guesses--I have no idea what these terms are in reference to." Multiple participants placed question marks beside terms, including benefits, flows, contexts, expresses, and bends, indicating that they were unfamiliar or unclear about these Framework terms. Although the survey participants were experienced occupational therapists--with an average of 11.5 years of experience--they were unfamiliar with certain terms. It should be noted that the terms included in the survey were selected randomly. No attempt was made to deliberately select Framework terms that might be especially obscure to participants. In addition, no participant stated that the format of the categorization instrument was confusing.

As shown in Table 1, 39.4% of participants had not heard of the Framework, and 76.6% of participants had not read the Framework. Only 7 participants (7.4%) reported use of the Framework at their workplace. Large numbers of therapists were unfamiliar with a document that is supposed to "describe the domain that centers and grounds the profession's focus and actions" (AOTA, 2002, p. 609). This survey took place more than 3 years after the official approval of the Framework.

It should be noted that those therapists who listed their main area of practice as education (N = 4) had especially high levels of agreement with the Framework, with a median of 22.5 of 30 agreements. The educators' median kappa coefficient was .70, which was much higher than the overall median kappa coefficient of .17. We speculate that educators in academia have special interests in terminology and regular exposure to the Framework because of their teaching responsibilities. However, their categorizations varied from the Framework's categorizations 25% of the time.

The test for stability indicated that occupational therapists frequently disagreed even with themselves in the classification of these terms. Perhaps some of the terms or major categories were unfamiliar. Given unfamiliarity, participants might not have had a strong basis for their selections, and multiple categories might have seemed appropriate for categorization. Possibly the format of the survey itself did not make sense to participants. However, participant comments pertained to the terms and concepts of the survey rather than to the layout or format of the survey itself. In addition, the survey instrument underwent pilottesting, and the survey instrument was designed and modified to be understandable and user-friendly. Results concerning instability should be interpreted with caution, given the small sample (N = 18).

A limitation of the study is a lower-than-optimal return rate. Although strategies were used to elicit a higher response rate, a 47% response rate is not uncommon (Kerlinger, 1986). All efforts were made to ensure that participants would complete and return surveys, including reminder postcards and a follow-up mailing (Dillman, 1999). Given the comments received from participants, we perceived a general sense of uncertainty from many participants concerning the "correctness" of their responses, although they were told repeatedly that there were no correct or incorrect responses. We reasoned that if participants did not feel confident in how they categorized the terms, their motivation to complete the survey was thereby reduced, because they may have believed it would reflect poorly on their abilities as occupational therapists. This general sense was particularly conveyed in one participant's comment on a blank survey, "I'm sorry--I can't complete this questionnaire because I am not familiar with much of the terminology." There is no reason to think that the therapists who did not participate in the study would have agreed with the Framework at a higher rate than those who did participate.

Another approach to the research question would be to separate the classification task into two components--definition and categorization--as was done in Borst and Nelson (1993). The current study evaluated therapists' thoughts on definition and categorization in one step. Possibly, however, the large degree of variability and low agreement rate with the Framework resulted from therapists not being clear on the meanings of the terms (definitions), which then impaired their ability to categorize the terms as precisely as they would have liked. On the other hand, definitions should not be necessary to understand essential terminology among members of the profession. Terminology that is not readily understandable to therapists should not form the basis of an officially sanctioned framework. In addition, dividing the classification task into two steps would make the survey instrument longer, which would tend to decrease the response rate (Fowler, 2001). The response rate already was problematic in the relatively simple current study. A potential method for future research would be to administer the survey face-to-face to increase the response rate; however, this method is expensive and time consuming and would require special funding.

A possible criticism of the research design is that the classification task was done without the opportunity to view the entire Occupational Therapy Practice Framework, with its many different levels of subcategories. Classifying lowest-level terms might have been easier if the classifier knew not only the highest level category but also the intermediate-level categories. However, the purpose of the study was to see whether occupational therapists agreed with the Framework; the purpose was not to see whether they could correctly classify Framework terms if given multiple cues. A study could be designed to see whether therapists can classify Framework terms correctly if provided most of the Framework with some blanks to be filled in. But such a study would not address the question of whether therapists use terms in the same way as the Framework. We believe that it would be difficult to design such a study in a way that would be acceptable to potential research participants, in that therapists would be making right-or-wrong responses in reference to a questionnaire of many pages in length.

The main implication of this study is that the Domain section of the Occupational Therapy Practice Framework does not reflect the thinking of occupational therapists. Occupational therapists see things differently. The argument could be made that the Framework is forward-looking and that therapists ultimately will learn to classify terms in the Framework way. However, Nelson (2006) found many logical errors of definition and classification in the Framework. We believe that the profession would not progress if therapists were to master an illogical system, because an illogical system cannot be the basis of research or reasoned practice.

Should a profession strive to have a single framework for organizing practice knowledge? Mosey (1985) argued that a profession should be pluralistic, not monistic, in its continuously evolving approaches to frames of reference (models of practice). According to Mosey (1985), a profession loses its dynamic capacity to respond to change if it commits to a single, comprehensive method of organizing practice knowledge. Kielhofner (2004) suggested that each age of a profession has a dominant paradigm but that practice knowledge is organized by several different practice models, each of which has its own terminology. Paradigms described by Kielhofner can be summarized in a few organizing propositions and do not have the depth and breadth of the complex hierarchical schemes within the Framework. Largely, terminology is model specific. Nelson has argued that the profession of occupational therapy has a definite core in its use of occupation and has attempted to define basic concepts within the overall construct of occupation. However, like Kielhofner and Mosey, Nelson asserted that the detailed principles of professional practice should be and indeed are found in the rich set of models of practice developed by many authors working with highly variegated theoretical material.

Whether or not there should be a single framework for occupational therapy practice knowledge, an additional question arises as to whether a professional association should have the dominant role in its development and adoption. This question can be answered from a pragmatic perspective or from a political-ethical perspective. From a pragmatic perspective, it can be argued that AOTA has encountered much difficulty in the past when trying to deal logically with these complex issues. The past track record does not predict success. Periodically, AOTA has published radically different perspectives on the basic nature of the profession, from the three versions of uniform terminologies to the Guide to Occupational Therapy Practice (Moyers, 1999) to the Framework. Has the basic nature of the profession changed as much as the documents suggest? How can something that changes so often be considered "uniform"? AOTA has conducted no published, peer-reviewed research on any aspect of all these terminologies, nor have AOTA members published research in their support. As for the political-ethical perspective, Nelson (2006) argued that a professional association has no right or obligation to dictate terms to its members. A complex official framework tends to snuff out alternative thinking and creative approaches to describing the nature of the profession. Occupational therapists and students understandably question why they should have to learn anything except the official version of the profession. As their publisher, AOTA has a vested interest in the Framework and other official documents. Officially sanctioned AOTA products compete both in the financial marketplace and in the marketplace of ideas. It is appropriate to raise the issues of conflict of interest and fairness.

The current study presents empirical data that are consistent with Nelson's 2006 analysis that the terminology and categorization system of the Framework are not useful for practice, education, and research. We recommend that AOTA refrain from developing systems of categorization, at least until a consensus develops in the field. The focus of AOTA in terms of terminology should be on consensusbased explanations of the nature of the field to outsiders, so that the benefits of occupational therapy can be realized fully in society.

References

American Occupational Therapy Association. (1979). Occupational therapy product output reporting system and uniform terminology system for reporting occupational therapy services. Rockville, MD: Author.

American Occupational Therapy Association. (1989). Uniform terminology for occupational therapy--Second edition. American Journal of Occupational Therapy, 43, 808-815.

American Occupational Therapy Association. (1994). Uniform terminology for occupational therapy--Third edition. American Journal of Occupational Therapy, 48, 1047-1054.

American Occupational Therapy Association. (2002). Occupational therapy practice framework: Domain and process. American Journal of Occupational Therapy, 56, 609-639.

Bloom, B. S. (Ed.). (1956). Taxonomy of educational objectives: The classification of educational goals. Handbook I: Cognitive domain. New York: David McKay.

Borst, M. J., & Nelson, D. L. (1993). Use of uniform terminology by occupational therapists. American Journal of Occupational Therapy, 47, 611-618.

Christiansen, C. (1994). Classification and study in occupation: A review and discussion of taxonomies. Journal of Occupational Science, 1, 3-21.

Dillman, D. A. (1999). Mail and Internet surveys: The tailored design method (3rd ed.). New York: Wiley.

Fowler, F. J. (2001). Survey research methods (3rd ed.). Newbury Park, CA: Sage.

Kerlinger, F. N. (1986). Foundations of behavioral research (3rd ed.). New York: Holt, Rinehart & Winston.

Kielhofner, G. (2004). Conceptual foundations of occupational therapy (3rd ed.). Philadelphia: F. A. Davis.

Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33, 159-174.

Mosey, A. C. (1981). Occupational therapy: Configuration of a profession. New York: Raven.

Mosey, A. C. (1985). The 1985 Eleanor Clarke Slagle Lecture--A monistic or a pluralistic approach to professional identity? American Journal of Occupational Therapy, 39, 504-509.

Moyers, P. A. (1999). The guide to occupational therapy practice. American Journal of Occupational Therapy, 53, 247-322.

Nelson, D. L. (1997). The 1996 Eleanor Clarke Slagle Lecture--Why the profession of occupational therapy will continue to flourish in the 21st century. American Journal of Occupational Therapy, 51, 11-24.

Nelson, D. L. (2006). Critiquing the logic of the Domain section of the Occupational Therapy Practice Framework: Domain and Process. American Journal of Occupational Therapy, 60, 511-523.

Reynolds, P. D. (1971). A primer in theory construction. Indianapolis, IN: Bobbs-Merrill.

Youngstrom, M. J. (2002). From the Guest Editor--The Occupational Therapy Practice Framework: The evolution of our professional language. American Journal of Occupational Therapy, 56, 607-608.

Denea S. Butts is Occupational Therapist, Harborside Healthcare, 395 Harding Avenue, Defiance, OH 43512. While conducting the study, she was a Student, Occupational Therapy Doctorate Degree Program, College of Health Science and Human Service, The University of Toledo Health Science Campus, OH; [e-mail].

David L. Nelson, PhD, OTR/L, FAOTA, is Professor, Department of Occupational Therapy, Collier Building, College of Health Science and Human Service, The University of Toledo Health Science Campus, OH.

Butts, D. S., & Nelson, D. L. (2007). Agreement between Occupational Therapy Practice Framework classifications and occupational therapists' classifications. American Journal of Occupational Therapy, 61, 512-518.

Table 1. Prior Familiarity With the Framework

Question                  Yes (%)    No (%)

Have you heard of the    57 (60.6)   37 (39.4)
  Occupational Therapy
  Practice Framework:
  Domain and Process
  (the Framework)?
Did you know that the    36 (38.3)   58 (61.7)
  Framework replaced
  Uniform
  Terminology-III?
Have you read the        22 (23.4)   72 (76.6)
  Framework?
Were you part of the      3 (3.2)    91 (96.8)
  development process
  for the Framework?
Do you use the            7 (7.4)    87 (92.6)
  Frameworkat work?
When completing the      21 (22.3)   73 (77.7)
  categorization
  task, did you
  realize that the
  terms were from
  the Framework?

Note. These questions were asked only after the participants
had completed the categorization task.

Table 2. Overall Test for Relationship Between Awareness
of Disability and ADL Motor and Process Ability

                    Relationship Between      Relationship Between
                    Awareness of Disability   Awareness of Disability
                    and ADL Motor Ability     and ADL Process Ability
Effect              (P value)                 (P value)

Time                  .099                      .2317
AAD                 < .001                    < .001
AAD by time           .043                      .0105
(AAD) (2)               --                      .8585
(AAD) (2) by time       --                      .0034

Note. ADL = activities of daily living; AAD = Assessment of
Awareness of Disability (Tham, Bernspang, & Fisher, 1999).

Table 3. Degree of Agreement Between Therapists' First
Categorizations and Second Categorizations (N = 18)

Indexes of Agreement       M      SD   Skewness   Median

Total agreement        14.80    5.00        0.1     13.5
Percentage of          49.60   17.10        0.1       45
  agreement
Kappa coefficient        .40     .20        0.2      .30

Indexes of Agreement       First      Second       Third      Fourth
                        Quartile    Quartile    Quartile    Quartile
Total agreement           6-10.5     10.5-13     14-18.5     18.5-23
Percentage of          20.0-35.0   35.0-43.3   46.7-62.8   62.8-79.3
  agreement
Kappa coefficient        .10-.23     .23-.33     .37-.54     .54-.75