Focus groups: a tool for consumer-based program evaluation in rehabilitation agency settings.
Rehabilitation agencies face many challenges as they seek to improve their services in the coming decades. Increased demand for accountability and effectiveness combined with dedication to empower clients present major program goals (Mason, 1990; Emener, 1991). Although traditional evaluation methods that measure program process and outcome continue to have merit, additional techniques need to be incorporated as technological, social, political, and economic pressures occur. One such technique, focus groups, appears to have much potential for rehabilitation program evaluation.
Focus groups have been used in market research since World War II, often to assess the impact or potential impact of a new product or service (Stewart & Shamdasani, 1990). More recently, studies have shown that focus groups can provide valuable feedback about ongoing experiences and general opinions in applied settings other than market research. For example, focus groups have been used to help evaluate products for hearing impaired individuals (Harkins & Jensema, 1987), to assess the public's perceptions of health risk factors (Morgan & Spanish, 1985), and to support international family planning projects (Bertrand, Brown, & Ward, 1992). These and other studies suggest that focus groups can contribute significant information during the program decision-making process as well as lead to the formation and development of further evaluation efforts.
Although focus groups vary considerably across situations and purpose, they can be broadly defined as a technique whereby 8 to 12 individuals discuss a particular topic of interest under the direction of a moderator (Stewart & Shamdasani, 1990). Typically a group session lasts for 1 to 2 hours; as a result, only a relatively narrow range of group size is practical. More than one group per specific topic is usually conducted and is strongly recommended. A primary strength of the focus group technique "is the explicit use of group interaction to produce data and insights that would be less accessible without the interaction found in a group" (Morgan, 1988, p. 12). The primary role of the moderator is to promote interaction, probe for details when warranted, and ensure that the discussion remains directed toward the topic of interest (Stewart & Shamdasani, 1990). The fundamental data produced by focus groups are the transcripts of the group discussion (Morgan, 1988).
Focus groups can serve a variety of program-related purposes. Major among these are to: (1) obtain general background information about a program, (2) generate program ideas that can be implemented and tested, (3) diagnose program problem areas, (4) gather information about consumers' impressions about a program, and (5) learn how consumers talk about the program or topic of interest (Stewart & Shamdasani, 1990).
Programmatically, this information can be used to help orient a service provider who is new to the field or program. It can also be used to generate impressions about consumers' experience and insights as well as to empower consumers to actively participate in changing their environment. Focus-group based information can also support research efforts, such as: to evaluate programs at different sites or for different target groups, for questionnaire development, to test hypotheses, and to facilitate interpretation of results from earlier or ongoing studies (Morgan, 1988; Mitra, 1992).
Despite the fact that focus groups can meet numerous program assessment needs, this technique has been under-utilized in rehabilitation program evaluation. The purpose of this article is to describe a recent application of focus groups used in a vocational training and rehabilitation setting. After a brief overview of the agency, a description of our use of focus groups will be highlighted. Particular attention will be given to the steps involved in the conduct of focus groups, followed by a general discussion of the utility of this technique. Conclusions and future implications for rehabilitation program evaluation are then presented.
Jewish Vocational Service (JVS) Chicago is a non-profit, non-sectarian agency that provides employment assistance to a wide range of people, including people with disabilities. The rehabilitation and skills training programs at JVS include: supported employment; sheltered workshops; janitorial, clerical, and health care aid training; and job placement. Initiated by recommendations from the Board and with the full support of senior management, our agency established an ad hoc committee to help develop a delivery system that measured client satisfaction with programs. An initial starting point in this process was conducting a series of focus groups. The purpose of these focus groups, in terms of the general categories outlined by Stewart and Shamdasani (1990), was to: (1) identify areas of program strengths and weaknesses, (2) learn how clients discuss and conceptualize agency services: and (3) to guide the design of a client satisfaction questionnaire.
Focus Group Structure
A total of 70 clients participated in eight focus groups; each group ranged in size from 7 to 14 participants. Eligible group participants were individuals who were currently enrolled in a specific JVS program and who had expressed interest in participating in a focus group when recruited by program staff. When necessary, program managers selected participants to limit group size.
These eight focus groups were conducted in five vocational rehabilitation programs of interest: supported employment (1 site), rehabilitation workshops (3 sites), janitorial training (1 site), clerical training (2 sites), as well as placement assistance and training (1 site). Each group was moderated by JVS staff. To avoid possible bias, however, a program manager did not moderate a focus group from his or her program. Except to serve as an interpreter for hearing-impaired clients, other program staff were not present during these focus groups.
Prior to conducting these focus groups, staff attended several meetings. The purpose of these meetings was to familiarize staff with the structure and procedures used in focus groups and to obtain staff input in the development of a procedures guide.
The focus groups were conducted over a 7-month period. Each group lasted about 1 hour and was held at the program site. To provide a comfortable setting, all groups were conducted during lunch or break, and refreshments were provided.
Each group was led by two moderators. One served as principal moderator while the other took notes throughout the group process. At the start of the focus group, participants were assured that their comments were confidential and would not be identified by individual or by program. To help facilitate discussion and to use a standard procedure, each principal moderator followed a focus group guide. Specifically, clients were asked to:
1. Describe their initial impressions about the agency before they started a program.
2. Identify what they liked best about the agency.
3. Discuss agency changes that clients would recommend.
4. Indicate whether staff understood their unique needs and interests.
5. Indicate whether they were offered sufficient opportunity to discuss their goals with agency staff.
6. Indicate whether the agency helped them find non-vocational activities outside the agency (e.g., referral to other community groups, help with social issues).
7. Identify anything else they wanted to share.
During this dialogue, the moderator probed responses for specific details when warranted. Particular attention was also given to ensuring that all participants had an opportunity to voice their opinions.
Comments from clients, based on the moderator's hand-written notes, comprised the primary data compiled from these focus groups.
Results and Discussion
As previously noted, the purpose of the present article is to outline the steps that a rehabilitation agency can take in the planning and conduct of focus groups and not to describe in detail results that are, for the most part, highly specific to the agency and its programs. Nonetheless, several aspects of the findings from this particular series of groups are generally applicable and therefore warrant mention.
To facilitate their usefulness, client comments and recommendations were categorized into four main areas. These were:
1. Staff Support and Related Issues
(For example: "Staff are supportive, patient, and understanding." "Need for more staff.")
2. Environment (For example: "We can make mistakes here and not be in trouble.")
3. Specific Program Features (For example: "We can use what we have learned on the computer and apply it to other things." "Role-playing is helpful.")
4. Agency-wide Administrative Issues (For example: "Include clients more in discussions about work in the workshops.")
To preserve client and program confidentiality, responses were not identified in written reports either by individual or program. The number of individuals and different programs that supported a particular comment or recommendation, however, were noted. These results were then reported to agency decision-makers along with corresponding recommendations. These recommendations were used both programmatically to aid program change in select areas and in research planning to identify topics for inclusion in a client satisfaction questionnaire. A more complete description of these results is provided elsewhere (JVS, 1992).
The level of effort that goes into the planning stage of focus groups is critical to their success. An important starting point in this planning is to determine whether focus groups can help answer or address the programmatic topics of interest. To this end, a review of available resources is needed. In addition, enlisting the support of key stakeholders as well as obtaining support from management is instrumental in ensuring the necessary level of cooperation needed to conduct these groups. Such early involvement can also uncover additional topics or areas of interest for focus group discussion and facilitate the utilization of the recommendations once the results are reported (Morgan, 1988).
The following steps are recommended when conducting focus groups (Morgan, 1988; Stewart & Shamdasani, 1990):
1. Determine the purpose and topics to be addressed by these focus groups.
2. Enlist support from key stakeholders and management.
3. Determine the number and size of groups to be conducted.
4. Define the procedures used to select and recruit participants.
5. Determine the level of moderator involvement. Identify and train moderators.
6. Select the site(s).
7. Determine the content and format for the group. Prepare a moderator's guide to provide a standardized structure.
8. Determine analysis procedures. Consider use of a tape recorder to facilitate the process of transcribing group discussion.
9. Check to ensure that facilities and resources are adequate before groups are run.
10. Prepare report(s) with utilization of results in mind.
Strengths and Limitations
A major strength of focus groups is the ability to clarify the clients' understanding of the programmatic topics of interest, that is, to learn about client's experiences and perspectives. As Morgan (1988) succinctly states, focus groups are useful for "investigating what participants think, but they excel at uncovering why participants think as they do". Moreover, focus groups can provide a vital mechanism whereby clients can influence their environment; thus, enabling them to engage as active participants in rehabilitation program efforts as "consumers" or "customers."
Indeed rehabilitation program professionals have recently examined the various terms -- client, customer, patient, consumer -- that describe rehabilitation service recipients (Emener, 1991; Patterson & Leach, 1987; Patterson & Marks, 1992). This re-examination reflects an acknowledgement of the active role of participants in program planning, implementation, and monitoring. It also helps to expand the concept of rehabilitation service recipient to include family members as well as staff, funders, and board members; all of whom are meaningful participants in rehabilitation program focus groups.
The use of focus groups is especially timely as amendments to the Rehabilitation Act of 1992 move into the arena of evaluation standards and performance indicators (Department of Education, 1993). Although these indices have yet to be determined, consumer satisfaction standards are likely to be a priority. In this context, focus groups seem particularly suited to assess the degree to which an agency has engaged in a full partnership with clients regarding the vocational services they receive, and whether clients are making meaningful and informed choices about their vocational goals and objectives. Moreover, such focus group efforts are also in line with current accreditation criteria specified by the Committee on Accreditation of Rehabilitation Facilities (CARF, 1993) regarding program evaluation in general and specifically the requirement to obtain input from those served.
Focus groups, however, are not without limitations. One such limitation when used in rehabilitation agency applications is that the moderator-focus group setting can resemble a therapist-therapy group arrangement. This similarity can be a source of confusion for clients as well as staff. Accordingly, care should be taken to explain to clients the purpose of the focus groups and to alert the moderator to this potential problem beforehand so that he/she can, as needed, redirect discussion toward the topics of interest. Also, it is important for staff to recognize that the goals and assumptions of focus groups are very different from those of therapy or support groups. As a result, there may be need to caution staff and administrators not to interpret focus group behavior within a clinical context. For example, criticism of rehabilitation programs and services by clients may be legitimate focus group responses and not expressions of pathology (e.g., diagnostic criteria for passive-aggressive personality APA, 1987). To avoid these possible sources of confusion by staff, the agency may choose to enlist the assistance of a moderator who has little or no therapy experience but who is well versed in focus group facilitation.
The popularity of focus groups within the field of market research may be a mixed blessing. On the one hand, because of its pervasive use there is an abundance of relevant literature (e.g., Moran, 1986). On the other hand, its extensive use in market research may dictate methodological applications in other settings using a single model. This single model approach may overshadow important differences in social science research, rehabilitation program evaluation, and market research, and prevent the evolution of multiple models that incorporate these differences. Indeed it seems reasonable that variations in focus group models would emerge as this technique is used more extensively in other fields (Morgan, 1988). Such expected variations speak to the robustness of the technique and the need for multiple models when applied across diverse applications.
Finally, it should be noted that focus group results are not immune to factors that are often likely to mitigate utilization of program evaluation results in general, such as: program funding, new legislation, and administrative and staff commitment. As is often the case in other research applications, utilization of focus group results is affected by every aspect of the research effort, from initial conceptualization to data analysis and interpretation with the early support of stakeholders and management an important link to the ultimate usefulness of the information (Patton, 1986).
Nevertheless, listening to client needs and designing programs and services to meet them through such evaluation techniques as focus groups can create an effective, efficient and compassionate rehabilitation environment for the demands of today and the challenges of tomorrow.
Special thanks are due Alan Goldstein, Executive Director, JVS for his thoughtful suggestions during preparation of this manuscript.
American Psychiatric Association. (1987). Diagnostic and statistical manual of mental disorders (3rd ed. rev.). Washington, DC: Author.
Bertrand, J. T., Brown, J. E., & Ward, V. M. (1992). Techniques for analyzing focus group data. Evaluation Review, 16(2), 198-209.
Commission on Accreditation of Rehabilitation Facilities. (1993). Standards manual for organizations serving people with disabilities. Tucson: Author.
Department of Education. (1993, February, 19). The state vocational rehabilitation services program 34 CFR p 361. Federal Register, 58(32), 9458-9560.
Emener, W. G. (1991). Empowerment in rehabilitation: An empowerment philosophy for rehabilitation in the 20th Century. Journal of Rehabilitation, 57(4), 7-12.
Harkins, J.E., & Jensema, C. J. (1987). Focus-group discussion with deaf and severely hard of hearing people on needs for sensory devices. Washington, DC: Gallaudet Research Institute.
Jewish Vocational Service. (1992, July). Summary of focus groups. Internal communication. Chicago: Consumer Satisfaction Measurement Development Committee.
Mason, C. Y. (1990). Consumer choice and satisfaction. Washington, DC: Technical Association of Rehabilitation Facilities.
Mitra, A. (1992, November). Use of focus groups in questionnaire design. Paper presented at the American Evaluation Association Annual Meeting, Seattle, Washington.
Moran, W. T. (1986). The science of qualitative research. Journal of Advertising Research, 26(3), 16-19.
Morgan, D. L. (1988). Focus groups as qualitative research (Sage University Paper Series on Qualitative Research Methods, Vol. 16). Beverly Hills, CA: Sage.
Morgan, D. L., & Spanish, M. T. (1984). Focus groups: A new tool for qualitative research. Qualitative Sociology, 7, 253-270.
Patterson, J. B., & Leach, R. (1987). Client satisfaction in vocational rehabilitation. Journal of Rehabilitation, 53(2), 40-43.
Patterson, J. B., & Marks, C. (1992). The client as customer: Achieving service quality and customer satisfaction in rehabilitation. Journal of Rehabilitation, 58(4), 16-21.
Patton, M. Q. (1986). Utilization-focused evaluation (2nd ed). Beverly Hills, CA: Sage.
Stewart, D. W., & Shamdasani, P.M. (1990). Focus groups: Theory and practice (Applied Social Research Methods Series, Vol. 20). Newbury Park, CA: Sage.
|Printer friendly Cite/link Email Feedback|
|Author:||Hotch, Deborah F.|
|Publication:||The Journal of Rehabilitation|
|Date:||Jul 1, 1994|
|Previous Article:||The modification of general chemistry laboratories for use by students with disabilities.|
|Next Article:||Multiple sclerosis: strategies for rehabilitation counselors.|