Analyzing single-subject design data using statistical process control charts.
The two-standard-deviation-band method, also known as a "Shewhart" chart, is a statistical method that is well known to social workers familiar with SSDs. It has been described and illustrated in a number of social work texts (for example, Bloom & Fischer, 1982; Bloom, Fischer, & Orme, 1995, 1999; Blythe & Tripodi, 1989; Rubin & Babbie, 1989, 1997). What has not been discussed in the social work literature, or in well-known texts on SSDs outside of social work (Barlow & Hersen, 1984; Barlow, Hayes, & Nelson, 1984; Kazdin, 1982; Kratochwill, 1978; Kratochwill & Levin, 1992), is that there are many different types of Shewhart charts, known more generally as "statistical process control charts," or simply "control charts."
Statistical process control (SPC) charts date back to the 1920s, and they are at the heart of SPC, a large and versatile body of industrial quality control techniques (for example, Doty, 1996; Ostle, Turner, Hicks, & McElrath, 1996; Wheeler & Chambers, 1992). SPC itself is a key part of an overall management system known as "total quality management" (TQM), which originally was implemented in U.S. manufacturing environments and is now being used in social work settings (Berman, 1995; Boettcher, 1998; Martin, 1993; Moore & Kelly, 1996). However, in a critique of TQM Gummer (1996) noted the frequent failure to incorporate SPC into TQM.
SPC charts increasingly are being adapted to diverse areas, including human services (Brannen & Streeter, 1995), health care, and other service industries (for example, Albin, 1992; Blumenthal, 1993; Carey & Lloyd, 1995), program evaluation (for example, Posavac, 1995), and organizational behavior management (for example, Mawhinney, 1988). However, there has been limited discussion of the use of SPC charts to analyze SSD data (Hopkins, 1995; Pfadt, Cohen, Sudhalter, Romanczyk, & Wheeler, 1992; Pfadt & Wheeler, 1995; Sideridis & Greenwood, 1996).
Not only has there been limited discussion of the use of SPC charts to analyze SSD data, but the only method for constructing an SPC chart described in the social work literature is incorrect (for example, Bloom & Fischer, 1982; Bloom et al., 1995, 1999; Blythe & Tripodi, 1989; Rubin & Babbie, 1989, 1997). This article describes and illustrates the correct method for constructing this particular type of chart, discusses it more fully than has been done in the social work literature, and places it in the context of SPC.
A major advantage of SPC charts is the simplicity of the computations. Computations for most SPC charts can be done easily with a calculator or spreadsheet program. However, there is a wealth of computer software that can simplify the construction of SPC charts (Institute of Industrial Engineers Solutions, 1997). Thus, this article discusses briefly the microcomputer software available to construct SPC charts.
SPC charts were developed by industrial quality control engineers to ensure the consistent delivery of quality products. Specifically, SPC charts are used to detect systematic changes in manufacturing processes through the identification of unusual product samples (for example, a sample of a part that is distinctly larger or smaller than other samples of this part). When unusual product samples are detected, attempts are made to discover their cause in the production process (for example, an improperly calibrated machine). Finally, this causal information is used to improve the production process (for example, recalibrate the machine) (Doty, 1996; Pitt, 1994; Wheeler & Chambers, 1992).
In an SSD chart time is plotted on the horizontal axis, and the outcome variable is plotted on the vertical axis. This provides a running record of outcome variability. This same chart is known in the SPC literature as a "run chart."
SPC charts are constructed by adding three elements to a run chart, which provides a more objective basis for the visual analysis of SSD data (Pfadt & Wheeler, 1995). A central value (for example, mean outcome) is computed and plotted as a solid line; this is referred to as the center line (the CL). Upper and lower control limits corresponding to plus and minus three standard deviations from this midpoint are computed and plotted as dashed lines parallel to the CL; these are referred to as the upper control limit (UCL) and the lower control limit (LCL), respectively.
Measured outcomes, whether manufacturing products or social work outcomes, are samples. Samples, even those selected randomly from the same population, exhibit variability. It is neither practical nor desirable to investigate the cause of all variations, so criteria are needed to determine which variations to investigate and which to ignore.
In the SPC literature variability is categorized as either within or outside the limits set by chance. Chance variability is defined as variability within the boundaries of the UCL and LCL. When all outcomes are within these boundaries, the process producing these outcomes is said to be "in statistical control." That is, the output of this process is predictable and stable within the limits of the UCL and LCL, and there is no change in the process parameter (for example, mean).
A process that is in statistical control is not necessarily producing an acceptable product. The UCL and LCL are established by the inherent variation in the process, not by external specifications that define a product as acceptable. For example, the typical product represented by the CL may be too large or too small. Or, the UCL and LCL may be beyond the acceptable limits for the product, and so some products within these limits may not be acceptable. However, SPC charts can be used to identify such situations, the first step in discovering their cause, and a prerequisite for improving the process.
When there are outcomes above the UCL or below the LCL, the process producing these out comes is said to be "out of statistical control." Stated another way, there has been a change in the process parameter (for example, mean). A process can be out of statistical control either in the sense that undesirable or desirable outcomes are produced, either of which is referred to as a "nonnormative" outcome. In either case the output of this process is not predictable and stable within the limits of the UCL and LCL.
POTENTIAL SOCIAL WORK APPLICATIONS
There are a number of SSD situations in which it is important to identify nonnormative events and to monitor the stability of a process (for example, Pfadt et al., 1992; Pfadt & Wheeler, 1995). First, in prevention programs the occurrence of undesirable nonnormative outcomes (for example, excessive weight loss or gain) may signal the need to modify the prevention process. Indeed, the prevention of undesirable outcomes is one of the main goals of SPC. Second, in testing intervention effects nonnormative outcomes that occur when an intervention is implemented may signal an intervention effect. Third, the identification of a nonnormative outcome after successful implementation of an intervention may signal relapse and a need for "booster shots." Fourth, the identification of nonnormative outcomes can stimulate a search for the cause of such outcomes, which then can be incorporated into the process (for example, an increase in caregiver satisfaction associated with increased social support). Fifth, in observation-only designs it is necessary to determine if some nonnormative outcome occurs (for example, postpartum depression), which then may require intervention. Finally, in comparing data across phases, and in deciding when to change phases, it is important to determine if a stable and predictable pattern of data exists within a given phase.
VARIABLES CONTROL CHARTS
A continuous variable is a variable for which infinitely small distinctions can be made (for example, duration of a behavior or event, body weight) (Cohen & Cohen, 1983). In the SPC literature outcomes measured on a continuous scale are referred to as variables (or sometimes measurement data) and their associated charts as variables charts (for example, Doty, 1996; Pitt, 1994; Wheeler & Chambers, 1992). The type of SPC chart described in the social work literature is a variables chart.
Gottman and Leiblum Procedure
Gottman and Leiblum (1974) are the source of the SPC chart described in the social work literature, and Bloom and Fischer (1982) introduced it in the social work literature. Jayaratne and Levy (1979) presented a different variation of the two-standard-deviation-band procedure, but this method has not been discussed in the SPC literature, to our knowledge. This type of chart is widely discussed in the SPC literature, but the method presented by Gottman and Leiblum for constructing it is not correct. The Gottman and Leiblum method is presented here as background for presentation of the correct method, and for those who are not familiar with it.
An example of the method described by Gottman and Leiblum may be helpful: A social worker conducts an eight-week follow-up of a client successfully treated for an eating disorder. The social worker monitors this client's weight weekly for eight weeks to detect any abrupt changes in weight. (Hypothetical data for this example are shown in Table 1 and plotted in Figure 1). The chart is constructed as follows:
1. Compute the mean. Plot a horizontal line representing the mean (the CL). (The letter "n" represents the number of samples.)
[CL.sub.X] = [bar]X = [Sigma]X/n = 951/8 = 118.88
2. Compute the estimate of the population standard deviation.
[[Sigma].sub.X] = [square root of [Sigma][(X - [bar]X).sup.2]/n -1] = [square root of 16.875/7] = 1.55
3. Plot horizontal lines representing two standard deviations above and below the mean.
[UCL.sub.X] = [bar]X + 2[[Sigma].sub.X] = 118.88 + 2(1.55) = 121.98
[LCL.sub.X] = [bar]X - 2[[Sigma].sub.X] = 118.88 - 2(1.55) = 115.78
TABLE 1--Data for Gottman and Leiblum Chart and X-mR-chart Sample Moving (Week) Weight Range 1 119 -- 2 122 3 3 120 2 4 118 2 5 118 0 6 117 1 7 118 1 8 119 1 Mean 118.88 1.43
If two successive data points fall above the upper two-standard-deviation limit or below the lower two-standard-deviation limit, there has been a change in the process mean--that is, the process is out of statistical control. If the normal distribution is the correct underlying model for this process, the probability of two successive data points falling above or below the two-standard-deviation limit is .002 (that is, [Alpha] = .002).
Typically in the SPC literature, the decision rule used to signal a change in the process mean is a single data point falling above the upper three-standard-deviation limit or below the lower three-standard-deviation limit. A two-standard-deviation/two-successive data point decision rule is used sometimes (we discuss this and other decision rules in more detail below). However, the three-standard-deviation/single data point decision rule is the norm (Doty, 1996; Pitt, 1994; Wheeler & Chambers, 1992).
In addition to the two-standard-deviation/two successive data point decision rule, the Gottman and Leiblum method differs from the SPC literature in a more significant way. This difference is in how the standard deviation is computed, and thus how the control limits are computed (for example, Doty, 1996; Pitt, 1994; Wheeler & Chambers, 1992). In the SPC literature they are computed as follows, using the hypothetical data in Table 1, and the resulting SPC chart is referred to as an X-mR-chart (X-moving range-chart) (for example, Doty, 1996; Pitt, 1994; Wheeler & Chambers, 1992) (see Figure 2):
1. Compute the range (R) of each two adjacent data points. This is known as a "moving range" for n = 2, and these ranges are shown in Table 1. (Note that there will be one fewer range value than data points.)
2. Compute the mean moving range. (The letter "n" represents the number of observations.)
[bar]R = [Sigma]R/n - 1 = 10/7 = 1.43
3. Compute the estimate of the population standard deviation.
[[Sigma].sub.x] = [bar]R/1.128 = 1.43/1.128 = 1.27
4. Compute the UCL and LCL, and plot these along with the mean, as illustrated in Figure 2.
[UCL.sub.x] = [bar]X + 3[[Sigma].sub.x] = 118.88 + 3(1.27) = 122.67
[LCL.sub.x] = [bar]X - 3[[Sigma].sub.x] = 118.88 - 3(1.27) = 115.08
If the normal distribution is the correct underlying model for a process, there is a 99.73 percent probability that any one data point will fall between these limits, and there is a .27 percent chance that any one data point will fall outside of these limits (that is, [Alpha] = .0027). Most SPC charts are constructed directly or indirectly from the basic normal distribution model, and so these probability values are approximately the same for most SPC charts. The difference is that some other SPC charts use approximations to the normal distribution (for example, normal curve approximation to the binomial and Poisson distributions), as will be discussed below (for example, Doty, 1996; Pitt, 1994).
The identification and treatment of out-of-control data points is common to all SPC charts. If a data point falls outside of the control limits, it indicates a change in the process parameter (for example, mean). If there is a change in the process parameter, a search should be instituted to identify the cause of this change, and appropriate action should be taken (which is why the UCL and LCL sometimes are referred to as "action limits"). These causes are referred to in the SPC literature as "assignable," "special," or "nonrandom" causes. Such causes can be identified more easily and in a more timely fashion if an ongoing annotated log or chart is maintained (Carey & Lloyd, 1995; Griffith, 1996).
If the cause is identified and appropriate action taken, the CL, UCL, and LCL should be recomputed without the out-of-control values; a revised chart should be constructed with the new limits; and the out-of-control values should be plotted along with the revised limits (the original values sometimes are referred to as "trial" limits and center lines). The out-of-control values are omitted from the computations because they are not representative of the population being estimated. If no cause can be identified, it should be assumed that the out-of-control values are part of the normal pattern of chance causes and the original chart should be retained (for example, Doty, 1996; Pitt, 1994).
In general, the Gottman and Leiblum method overestimates the standard deviation because it pools all data points to compute the standard deviation. A larger standard deviation results in a wider range between the UCL and the LCL, which in turn makes it more difficult to detect a change in the process mean (wheeler & Chambers, 1992).
There is a variation of the X-mR-chart in which the moving average--that is, the mean of successive adjacent data points, is used in place of the overall mean, and this chart is referred to as the m [bar]X-mR-chart. This chart is useful when the process is changing slowly and considerable variability exists (Wheeler & Chambers, 1992).
When there are multiple observations per sample ("grouped" data) the [bar]X-R-chart (X-bar-range-chart) should be used to detect changes in the process mean. An example of this would be if the client in the earlier example were weighed four times per week (n = 4), at randomly selected times, for eight weeks (eight samples). Multiple observations per sample, in general, provide a more accurate basis for decisions.
The [bar]X-R-chart is constructed as follows using the hypothetical data in Table 2 (for example, Ostle et al., 1996):
1. Compute the mean and range for each sample (that is, one week in this case) (Table 2). Plot these means as illustrated in Figure 3.
2. Compute the mean of the means for each sample. Plot a horizontal line representing this value, as illustrated in Figure 2 (the CL). (The letter "m" represents the number of samples.)
[CL.sub.x] = X = [Sigma][bar]X/m = 957.75/8 = 119.72
3. Compute the mean range.
[bar] = [Sigma]R/m = 30/8 = 3.75
4. Compute the estimate of the population standard deviation of the mean. (The letter "n" represents the number of measurements per sample.) (When there are 2, 3, or 5 observations per sample [d.sub.2] equals 1.128, 1.693, and 2.326, respectively. Values for other ns can be obtained from tables in most SPC texts.)
[[Sigma].sub.[bar]x] = [bar]R/[d.sub.2]/[square root of n] = 3.75/2.059/[square root of 4] = .91
5. Compute the UCL and LCL, plot these as illustrated in Figure 3.
[UCL.sub.[bar]x] = X + 3[[Sigma].sub.[bar]x] = 119.72 + 3(.91) = 122.45
[LCL.sub.[bar]x] = X - 3[Sigma] = 119.72 - 3(.91) = 116.99
TABLE 2--Data for X-R-Chart and R-Chart Weight Sample (Week) 1 2 3 4 Mean Range 1 117 120 120 119 119.00 3 2 122 119 115 122 119.50 7 3 121 120 122 120 120.75 2 4 122 119 118 118 119.25 4 5 119 119 120 118 119.00 2 6 122 120 122 117 120.25 5 7 123 120 122 118 120.75 5 8 120 118 120 119 119.25 2 119.72 3.75
This method is derived from the normal distribution and is based on the assumption that observations are independent. However, Wheeler and Chambers (1992) presented simulation results indicating that this method is quite robust for nonnormality even when the sample size is small (n = 2), if the three-standard-deviation rule is used. The X-mR-chart, which has n = 1 per sample, is more affected by nonnormality (Ostle et al., 1996).
An algebraically equivalent method for computing control limits often found in the SPC literature is as follows, using the data in Table 2 (for example, Pitt, 1994):
[UCL.sub.[bar]x] = X + [A.sub.2][bar]R = 119.72 + (.729)(3.75) = 122.45
[LCL.sub.[bar]x] = X + [A.sub.2][bar]R = 119.72 - (.729) (3.75) = 116.99
([A.sub.2] varies according to the sample size. For n = 2, 3, and 5, it equals 1.880, 1.023, and .577, respectively. Values for other ns can be obtained from tables in most SPC texts.)
There are several variations of the [bar]X-R-chart. First, the procedure just described assumes the absence of trend in the sample means (for example, as might occur in monitoring the height and weight of a growing child, or a process that is improving), but an alternative procedure is available that does not make that assumption (Doty, 1996). Second, with larger sample sizes (for example, more than 10) the standard deviation should be used in place of the range, and this chart is known as an [bar]X-S-chart (X-bar-standard deviation-chart) (Doty, 1996; Ostle et al., 1996).
The use of [bar]X-charts to detect changes in the process mean over time is based on the assumption that the process variability is the same over time (for example, Pitt, 1994). The range (or R-) chart is used to detect changes in variability over time, and it should be monitored along with the [bar]X-R-chart (for example, Pitt, 1994).
The detection of changes in the process variability over time is important not only as a prerequisite to the interpretation of the [bar]X-R-chart, but also because such changes can be important in and of themselves. For example, it might be important to detect weekly changes in variability of weight, which might suggest "binging" and "purging" in a given week.
The R-chart is constructed as follows using the hypothetical data in Table 2 (for example, Ostle et al., 1996):
1. Compute the range for each sample (in this example, week).
2. Compute the mean sample range. Plot a horizontal line representing this value, as illustrated in Figure 4 (the CL). (The letter "m" represents the number of samples.)
[CL.sub.[bar]R] = [bar]R = [Sigma]R/m = 30/8 = 3.75
3. Compute the estimate of the population standard deviation of the mean range. When there are 2, 3, or 5 observations per sample [d.sub.2] equals 1.128, 1.693, and 2.326, respectively. Values for other ns can be obtained from tables in most SPC texts. The value of [d.sub.3] equals .8525, .8884, and .8641 for 2, 3, and 5 observations per sample, respectively. Values for other ns can be obtained from tables in most SPC texts.
[[Sigma].sub.[bar]R] = [d.sub.3][bar]R/[d.sub.2] = (.8798)(3.75)/2.059 = 1.60
4. Compute the UCL and LCL and plot these as illustrated in Figure 4.
[UCL.sub.[bar]R] = [bar]R + 3[[Sigma].sub.R] = 3.75 + (3)(1.60) = 8.55
[LCL.sub.[bar]R] = [bar]R - 3[[Sigma].sub.R] = 3.75 - (3)(1.60) = -1.05 (A range cannot be negative, so truncate the LCL to 0 if the computed value is negative.)
This method is derived from the normal distribution and is based on the assumption that the samples are independent.
An algebraically equivalent method for computing control limits oftentimes found in the SPC literature is as follows, using the data in Table 2 (for example, Pitt, 1994):
[UCL.sub.[bar]R] = [D.sub.4][bar]R = (2.282)(3.75) = 8.56
[LCL.sub.[bar]R] = [D.sub.3][bar]R= (0)(3.75) = 0
([D.sub.3] and [D.sub.4] vary according to the sample size. For n = 2, 3, and 5, [D.sub.3] equals 0. For n = 2, 3, and 5, [D.sub.4] equals 3.267, 2.575, and 2.114, respectively. Values for other ns can be obtained from tables in most SPC texts.)
There are several variations of the R-chart. First, it is possible to construct an R-chart using individual data, although reservations have been expressed concerning the accuracy of this chart (Doty, 1996; Ostle et al., 1996). Second, when sample sizes are larger than 10, the standard deviation is recommended in place of the range, and this is referred to as a standard deviation- (S-) chart (Doty, 1996). The S-chart should be monitored along with the [bar]X-S-chart, just as the R-chart should be monitored along with the [bar]X-R-chart.
ATTRIBUTES CONTROL CHARTS
The charts discussed so far were designed for use with outcomes that are measured on a continuous scale. It also is possible to construct SPC charts for discrete outcome variables, which have distinctive individual values that cannot be divided. For example, an event or characteristic may be present or absent, or the number of events or characteristics may be counted (Doty, 1996). In the SPC literature discrete outcomes are referred to as attributes and their associated charts as attributes charts (for example, Doty, 1996; Pitt, 1994; Wheeler & Chambers, 1992). In general, attribute charts are not as sensitive to change as variables charts because the type of measurement is less precise, and so it is necessary to collect a larger number of samples (Doty, 1997; Griffith, 1996; Wheeler & Chambers).
One basic type of attribute chart is the proportion- (p-) chart. It is used to detect changes in the proportion of "nonconforming" units (that is, products or services) (for example, Doty, 1996; Pitt, 1994; Wheeler & Chambers, 1992). Many outcomes of interest in SSDs involve events comparable to nonconforming units. These include, for example, whether group members attend group meetings, anorexic clients skip meals, clinic patients keep appointments, truant students attend school, and social workers complete home visits.
In an example of a p-chart, a social services agency needs to control the proportion of client records with missing information. Each week a random sample of five client records is reviewed. Hypothetical data for 10 weeks are shown in Table 3; Figure 5 shows the p-chart, and the p-chart is constructed as follows, using these data (Doty, 1996):
1. Compute the proportion nonconforming (proportion of records each week with missing information), shown in Table 3. Plot these proportions as illustrated in Figure 5. (The letter "n" represents the sample size, and "np" represents the number of nonconforming units per sample.)
p = np/n
2. Compute the mean proportion of nonconforming records. Plot a horizontal line representing this value as illustrated in Figure 5 (the CL).
[CL.sub.[bar]p] = [bar]p = [Sigma]np/[Sigma]n = 12/50 = .24
3. Compute the standard deviation of the mean proportion nonconforming.
[[Sigma].sub.[bar]p] = [square root of [bar]p(1 - [bar]p])/n] = [square root of .24(1 - .24)/5] = .19
4. Compute the LCL and UCL and plot horizontal lines representing these values as illustrated in Figure 5.
[UCL.sub.[bar]p] = [bar]p + 3[[Sigma].sub.[bar]p] = .24 + 3(.19) = .81
[LCL.sub.[bar]p] = [bar]p - 3[[Sigma].sub.[bar]p] = .24 - 3(.19) = -.33 (A proportion cannot be negative, so truncate the LCL to 0 if the computed value is negative.)
TABLE 3--Data for p-Chart Client Record Sample (Week) 1 2 3 4 5 p np n 1 0 0 0 1 0 .2 1 5 2 0 1 0 0 0 .2 1 5 3 0 0 1 0 1 .4 2 5 4 0 0 0 0 0 0 0 5 5 0 0 0 0 1 .2 1 5 6 0 1 1 0 0 .4 2 5 7 1 0 0 0 0 .2 1 5 8 0 1 0 0 1 .4 2 5 9 0 0 1 0 0 .2 1 5 10 0 1 0 0 0 .2 1 5 12 50 NOTE: For a client record 0 indicates no missing data and 1 indicates missing data.
This method is derived from the binomial distribution and has four prerequisites (Doty, 1996; Pitt, 1994; Wheeler & Chambers, 1992). First, the sample must consist of distinct items. Second, each distinct item must be classified as possessing or not possessing some attribute. Third, the probability that an item does or does not have the attribute under study (that is, p) must be constant in each sample. Finally, the likelihood of an item possessing the attribute must not be affected by whether or not the preceding item possessed the attribute.
There are several variations of the p-chart. First, the procedure just described assumes equal sample sizes, but a procedure is available that does not (for example, Doty, 1996). Second, it is possible to construct a chart in which the number of nonconforming units is plotted; this is known as an np-chart (for example, Doty, 1996). The np-chart is useful when the number of nonconforming units is most meaningful, but this chart should be used with equal sample sizes. Third, it is possible to construct a chart in which percentages are used instead of proportions for ease of understanding; this is known as a 100p-chart (The CL, UCL, and LCL are multiplied by 100.) (Doty, 1996). Finally, it is possible to construct a p-chart in which weights are used to adjust for the degree or importance of the unit nonconforming (for example, some types of missing data are more important than others); this is available for equal and unequal sample sizes (Doty, 1996).
A nonconforming unit can have more than one nonconformity. For example, the interest might be in counting the number of pieces of missing information in each record (for example, missing social security number, missing referral source, and so forth) instead of just whether information was missing.
A count- (c-) chart should be used when the interest is detecting and controlling changes in the number of nonconformities, and there is an equal opportunity of measurement (for example, equal number of units or equal time period, which is analogous to equal sample sizes) (for example, Doty, 1996; Pitt, 1994; Wheeler & Chambers, 1992). Many outcomes of interest in SSDs involve counts. These include, for example, the number of negative thoughts, aggressive outbursts, times a parent compliments a child, parent-child arguments, telephone contacts with clients, placement changes for a child in state custody, and visits to an elderly person by a caregiver.
In an example of a c-chart, an intervention is used successfully to reduce the daily number of negative thoughts. The social worker then conducts a six-week (that is, 42 days) follow-up to monitor whether this level is maintained, or whether booster shots are needed. During each phase, a two-hour period is selected randomly everyday, and the client records the number of negative thoughts during that time. Hypothetical data for this example are shown in Table 4, and Figure 6 illustrates the c-chart, and the c-chart is constructed as follows, using these data (Doty, 1996):
1. Compute the mean number of self-depreciating thoughts across observation periods (two hours each day). Plot a horizontal line representing this value, as illustrated in Figure 6 (the CL). (The letter "c" represents the count, the number of nonconformities, and the letter "n" represents the number of samples.)
[CL.sub.c] = [bar]c = [Sigma]c/n = 461/42 = 10.98
2. Compute the standard deviation of c.
[[Sigma].sub.c] = [square root of [bar]c] = [square root of 10.98] = 3.31
3. Compute the LCL and UCL and plot horizontal lines representing these values as illustrated in Figure 6.
[UCL.sub.c] = [bar]c + 3[[Sigma].sub.[bar]c] = 10.98 + 3(3.31) = 20.92
[LCL.sub.c] = [bra]c - 3[[Sigma].sub.[bar]c] = 10.98 - 3(3.31) = 1.04
TABLE 4--Data for c-Chart Number of Sample Negative (Day) Thoughts 1 12 2 11 3 7 4 15 5 8 6 11 7 9 8 10 9 10 10 13 11 11 12 17 13 12 14 14 15 10 16 16 17 12 18 9 19 10 20 6 21 7 22 8 23 11 24 11 25 10 26 14 27 15 28 8 29 11 30 13 31 9 32 8 33 12 34 10 35 16 36 11 37 10 38 11 39 13 40 9 41 7 42 14 461
This method is derived from the Poisson distribution and has four prerequisites (Doty, 1996; Pitt, 1994; Wheeler & Chambers, 1992). First, the counts must be of discrete events. Second, the counts must occur within a well-defined, finite region of space, time, or product. Third, the events must occur independently of each other. Fourth, the probability of an event must be constant across samples.
There is a variation of the c-chart in which the counts can be weighted (for example, if some types of self-depreciating thoughts are more important than others). In addition, if the observation times or other opportunity for measurement are unequal (for example, a different number of client records sampled in different weeks), a variation of the c-chart, known as the u-chart, should be used to monitor the average number of nonconformities per opportunity of measurement. Also, there is a variation of the u-chart in which differential weights can be applied (for example, Doty, 1996).
ALTERNATIVE DECISION RULES
Typically in the SPC literature, a single data point falling outside of the three-standard deviation limits is used to signal a change in the process parameter. A number of other decision rules also have been proposed, although they are too numerous to detail fully here, and there is not a set of universally accepted rules (for example, Griffith, 1996; Ostle et al., 1996; Pfadt & Wheeler, 1995; Wheeler & Chambers, 1992). However, several rules typically are proposed to signal a change in a process mean:
1. Nine (some say eight) consecutive data points fall on the same side of the CL (this is useful for detecting sustained shifts in the process mean).
2. Two of three consecutive data points fall between two and three standard deviations from the CL (this is useful for early detection of a change in the process mean).
3. Four of five consecutive data points fall between one and two standard deviations from the mean (this is useful for early detection of a change in the process mean).
4. Six or more consecutive data points exhibit an upward or downward pattern (this is useful for detecting a trend).
SELECTING AN SPC CHART
This article has discussed a number of basic SPC charts. Figure 7 shows a decision tree for use in selecting from among these. The first step is to decide whether the outcome is a continuous, binary, or count variable. If the outcome is continuous, an X-mR-chart or m [bar]X-mR-chart should be used if the sample sizes equal 1 (the latter with a slowly changing, more variable process), an [bar]X-R-chart and R-chart should be used if the sample sizes are equal and range from 2 through 10, and an [bar]X-S-chart and S-chart should be used if the sample sizes are equal and greater than 10. If the outcome is a binary variable, an np-chart can be used if the sample sizes are equal and the interest is in the number of nonconforming outcomes, and a p-chart or a 100p-chart can be used if the sample sizes are equal or unequal and the interest is in the proportion or percentage of nonconforming outcomes, respectively. Finally, if the outcome is a count variable, a c-chart should be used if the sample sizes are equal, and a u-chart should be used if the sample sizes are not equal.
A recent article in an industrial engineering magazine (Institute of Industrial Engineers Solutions, 1997) provides a fairly comprehensive list of SPC software. It lists 93 different software packages. Most of these software products are expensive, priced above $500 and up to $17,000. Because most of these products are designed for complex manufacturing environments, many of the high-end packages contain functions that are not needed in the analysis of SSD data and can overly complicate the analysis of such data.
There are some reasonably priced software packages under $500 with the functions needed to enter and analyze SSDs (see Table 5). From this list SPC XL seems particularly attractive because of its low price and the fact that it works within Microsoft Excel, which provides extensive graphics capabilities and the ability to export results easily to word processor packages such as Microsoft Word.
TABLE 5--SPC Software Company (Product) Price Notes Web Address Digital Computations, $199 Runs with MS www.sigmazone.com/ Inc. (SPC XL) Excel as option SPC_XL/spc_xl.html in Excel's main menu bar; provides graphing options and variety of control charts. User Solutions Inc. $149 Collection of 15 www.usersol.com/ (Spreadsheet QC) menu-driven products.sqc.html spreadsheet templates that work with MS Excel, Lotus 1-2-3, and Quattro Pro; provides ability to customize and enhance macros. NCSS (NCSS 2000) $300 Statistical www.amsquare.com/ software that ncss/ncsswin.html contains quality control functions and reports; imports and exports data from Excel, SAS, SPCC, and other formats; demo can be downloaded from Internet. Pister Group, Inc. $495 Enter data www.pister.com/ (QC-Pro SPC) through spc.html spreadsheet-like interface; data can be exported to spreadsheet, database, and word processor formats; variety of control charts. Sky Mark Corporation $395 Provides ability www.skymark.com (PathMaker 2) to enter data using spreadsheet and dynamically see charts and graphs on same screen; demo and tour of product available from Internet.
Because many social work academics have access to SPSS, it is notable that the SPSS-base product produces SPC charts. SPSS allows the user to select among four categories of SPC charts: (1) [bar]X-R-charts, [bar]X-S-chart, R-charts, and S-charts; (2) X-mR-charts and mR-charts; (3) p-and np-charts; and (4) c- and u-charts. In addition, SPSS has a stand-alone product called QI Analyst, which provides a wide array of SPC charts, statistics, and functions, although it is relatively expensive.
Another option is to construct spreadsheets (for example, Zimmerman & Icenogle, 1999). We developed Excel spreadsheets to enter data; calculate CL, UCL, and LCL; and dynamically display an SPC chart. Thus far, we have developed spreadsheets for the X-mR-chart, [bar]X-R-chart, R-chart, p-chart, and c-chart. These spreadsheets are available without charge at http://utcmhsrc2.csw.utk. edu/evaluatingpractice/course%20materials.htm.
Although SPC charts have a place in the analysis of SSD data, they have some potential limitations. First, SPC charts detect nonnormative outcomes based on specifications established by the inherent variation in a process, not by external specifications that define whether the outcome is acceptable. In SPC literature the ability of a process to meet external specifications is referred to as "process capability," and it is possible to determine this (for example, Doty, 1996, 1997; Griffith, 1996; Ostle et al., 1996; Pitt, 1994). However, the problem in social work and related areas is that most often specifications for acceptable outcomes are unavailable.
A second potential limitation of the SPC charts discussed in this article is that they assume the absence of autocorrelation. SPC charts are available that do not rely on this assumption (for example, Atienza, Tang, & Ang, 1998; Liu & Tang, 1996), although there is some disagreement about the extent to which autocorrelation distorts conclusions drawn from SPC charts (Wheeler & Chambers, 1992).
A third potential limitation of SPC charts is the criteria used to judge a process out of statistical control. With the use of the three-standard-deviation rule, alpha is set to .0027, two-tailed. Out-of-control data points command attention because they are so unlikely. However, the probability of correctly detecting an out-of-control process (that is, statistical power) is limited by this low alpha value. Less stringent decision criteria might be more appropriate when using SPC charts for the analysis of SSD data, especially in practice evaluation. This is an issue that needs further consideration (Blumenthal, 1993; Carey & Lloyd, 1995), although strong arguments have been advanced against a relaxation of the three-standard-deviation rule (for example, Wheeler & Chambers, 1992).
A fourth potential limitation of SPC charts is that often with SSDs, especially in practice evaluation, it is necessary to draw conclusions on the basis of a small number of observations. In general, the larger the sample size and the larger the number of samples, the easier it is to detect an out-of-control process. However, with small samples any method for analyzing SSD data, visual or statistical, is prone to error.
A final potential limitation is that SPC charts were designed for use by manufacturers to produce quality products, although these methods also have been applied, for example, to the delivery of health care (for example, Carey & Lloyd, 1995) and behavioral outcomes (for example, Pfadt et al., 1992; Pfadt & Wheeler, 1995; Sideridis & Greenwood, 1996). The delivery of high-quality products undoubtedly differs in important ways from achieving desirable social work outcomes. For example, manufacturing processes are different from processes underlying the outcomes of concern to social work. The implications of such differences need further consideration.
Finally, this article has not discussed all available SPC charts, or all aspects of SPC. For example, it has not discussed multivariate SPC methods (for example, Doty, 1996; Montgomery, 1996). Also, SPC encompasses more than charts, including methods for identifying the causes of processes that are out of control (for example, Pareto charts, cause and effect diagrams, process flowcharts) (for example, Albin, 1992; Brannen & Streeter, 1995; Carey & Lloyd, 1995; Doty, 1996, 1997; Pfadt & Wheeler, 1995), and methods for bringing processes into control. These additional charts and other aspects of SPC warrant more consideration.
The SPC literature is vast, dating back to Shewhart's 1931 work, and so entering this literature can be daunting. In addition to the literature cited in this article, one especially useful starting place for the interested reader is the Web page of the American Society for Quality (ASQ) (www.asq.org). Among other features, this site provides a glossary, a searchable database of relevant articles, and a catalogue of books. Also, the Journal of Quality Control, published by the ASQ, and the journal Technometrics, published jointly by the ASQ and the American Statistical Association, are good sources for SPC and related topics.
No single data analytic procedure suffices for the analysis of SSD data. SPC charts are no exception to this. However, SPC charts do have a useful place in the analysis of SSD data, and they merit further consideration in social work and related professions.
Albin, J. (1992). Quality improvement in employment and other human services: Managing for quality through change. Baltimore: Paul H. Brookes.
Atienza, O. O., Tang, L. C., & Ang, B. W. (1998). A SPC procedure for detecting level shifts of autocorrelated processes. Technometrics, 40, 340-351.
Barlow, D. H., Hayes, S. C., & Nelson, R. O. (1984). The scientist practitioner: Research and accountability in clinical and educational settings. New York: Pergamon Press.
Barlow, D. H., & Hersen, M. (1984). Single-case experimental designs: Strategies for studying behavior change (2nd ed.). New York: Pergamon Press.
Berman, E. M. (1995). Implementing TQM in state welfare agencies. Administration in Social Work, 19(1), 55-72.
Bloom, M., & Fischer, J. (1982). Evaluating practice: Guidelines for the accountable professional. Englewood Cliffs, NJ: Prentice Hall.
Bloom, M., Fischer, J., & Orme, J. G. (1995). Evaluating practice: Guidelines for the accountable professional (2nd ed.). Boston: Allyn & Bacon.
Bloom, M., Fischer, J., & Orme, J. G. (1999). Evaluating. practice: Guidelines for the accountable professional (3rd ed.). Boston: Allyn & Bacon.
Blumenthal, D. (1993). Total quality management and physicians' clinical decisions. JAMA, 269, 2775-2778.
Blythe, B. J., & Tripodi, T. (1989). Measurement in direct practice. Newbury Park, CA: Sage Publications.
Blythe, B., Tripodi, T., & Briar, S. (1994). Direct practice research in human service agencies. New York: Columbia University Press.
Boettcher, R. E. (1998). A study of quality-managed human service organizations. Administration in Social Work, 22(2), 41-56.
Brannen, S. J., & Streeter, C. L. (1995). Doing it with data: Total quality management and the evaluation of social services. In B. Bummer & P. McCallion (Eds.), Total quality management in the social services: Theory and practice (pp. 59-88). Albany, NY: Rockefeller College Press.
Carey, R., & Lloyd, R. C. (1995). Measuring quality improvement in healthcare: A guide to statistical process control applications. New York: Quality Resources.
Cohen, J., & Cohen, P. (1983). Applied multiple regression/ correlation analysis for the social sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum.
Doty, L. A. (1996). Statistical process control (2nd ed.). New York: Industrial Press.
Doty, L. A. (1997). SPC for short run manufacturing. Cincinnati: Hanser Gardner.
Franklin, R. D., Allison, D. B., & Gorman, B. S. (Eds.). (1997). Design and analysis of single-case research. Mahwah, NJ: Lawrence Erlbaum.
Gottman, J. M., & Leiblum, S. R. (1974). How to do psychotherapy and how to evaluate it. New York: Holt, Rinehart & Winston.
Griffith, G. K. (1996). Statistical process control methods for long and short runs (2nd ed.). Milwaukee: ASQC Quality Press.
Gummer, B. (1996). Total quality management: Organizational transformation or passing fancy? Administration in Social Work, 20(3), 75-95.
Hopkins, B. L. (1995). Applied behavior analysis and statistical process control? Journal of Applied Behavior Analysis, 28, 379-386.
Institute of Industrial Engineers Solutions. (1997, November). SPC/SQC software: Buyer's guide, 29, pp. 46-54.
Jayaratne, S., & Levy, R. L. (1979). Empirical clinical practice. New York: Columbia University Press.
Kazdin, A. E. (1982). Single-case research designs: Methods for clinical and applied settings. New York: Oxford University Press.
Kazi, M.A.F. (1998). Single-case evaluation by social workers. Brookfield, VT: Ashgate.
Kirk, S. A. (Ed.). (1996). Book forum on the scientist--practitioner (Special Issue). Social Work Research, 20(2).
Kratochwill, T. R. (Ed.). (1978). Single subject research: Strategies for evaluating change. New York: Academic Press.
Kratochwill, T. R., & Levin, J. R. (Eds.). (1992). Single-case research design and analysis: New directions for psychology and education. Hillsdale, NJ: Lawrence Erlbaum.
Liu, R. Y., & Tang, J. (1996). Control charts for dependent and independent measurements based on bootstrap methods. Journal of the American Statistical Association, 91(436), 1694-1700.
Martin, L. (1993). Total quality management: The new managerial wave. Administration in Social Work, 17(2), 1-16.
Mawhinney, T. C. (1988). Organizational behavior management and statistical process control. New York: Haworth Press.
Montgomery, D. C. (1996). Introduction to statistical quality control (3rd ed.). New York: John Wiley & Sons.
Moore, S. T., & Kelly, M. J. (1996). Quality now: Moving human services organizations toward a consumer orientation to service quality. Social Work, 41, 33-40.
Orme, J. G. (1991). Statistical conclusion validity for single-system designs. Social Service Review, 65, 468-491.
Ostle, B., Turner, K. V., Jr., Hicks, C. R., & McElrath, G. W. (1996). Engineering statistics. Belmont, CA: Duxbury Press.
Pfadt, A., Cohen, I. L., Sudhalter, V., Romanczyk, R. G., & Wheeler, D. J. (1992). Applying statistical process control to clinical data: An illustration. Journal of Applied Behavior Analysis, 25, 551-560.
Pfadt, A., & Wheeler, D. J. (1995). Using statistical process control to make data-based clinical decisions. Journal of Applied Behavior Analysis, 28, 349-370.
Pitt, H. (1994). SPC for the rest of us: A personal path to statistical process control. Reading, MA: Addison Wesley.
Posavac, E. J. (1995). Statistical process control in the practice of program evaluation. Evaluation Practice, 16(2), 121-130.
Rubin, A., & Babbie, E. (1989). Research methods for social work. Belmont, CA: Wadsworth.
Rubin, A., & Babbie, E. (1997). Research methods for social work (3rd ed.). Pacific Grove, CA: Brooks/Cole.
Rubin, A., & Knox, K. S. (1996). Data analysis problems in single-case evaluation: Issues for research on social work practice. Research on Social Work Practice, 6, 40-65.
Shewhart, W. A. (1931). Economic control of quality of manufactured product. Princeton, NJ: Reinhold.
Sideridis, G. D., & Greenwood, C. R. (1996). Evaluating treatment effects in single-subject behavioral experiments using quality-control charts. Journal of Behavioral Education, 6(2), 203-211.
Wheeler, D. J., & Chambers, D. S. (1992). Understanding statistical process control (2nd ed.). Knoxville, TN: SPC Press.
Zimmerman, S. M., & Icenogle, M. L. (1999). Statistical quality control using EXCEL. Milwaukee: ASQ Quality Press.
The authors thank Martin Bloom, Terri Combs-Orme, Joel Fischer, and Jim Seaberg for their suggestions on an earlier draft of this article, and Jim Post for help with Microsoft Excel. Correspondence concerning this article and requests for copies should be addressed to John G. Orme. Electronic mail may be sent to John G. Orme at firstname.lastname@example.org or to Mary Ellen Cox at email@example.com. An earlier version of this article was presented at the Council on Social Work Education Annual Program Meeting, March 1999, San Francisco, and at the meeting of the Society for Social Work and Research, January 1999, Austin, TX.
Original manuscript received June 26, 2000 Final revision received December 19, 2000 Accepted December 28, 2000
John G. Orme, PhD Associate Professor College of Social Work University of Tennessee Henson Hall Knoxville, TN 37996 e-mail: firstname.lastname@example.org
Mary Ellen Cox, PhD Research Associate College of Social Work University of Tennessee Henson Hall Knoxville, TN37996
|Printer friendly Cite/link Email Feedback|
|Author:||Orme, John G.; Cox, Mary Ellen|
|Publication:||Social Work Research|
|Article Type:||Statistical Data Included|
|Date:||Jun 1, 2001|
|Previous Article:||Client participation and outcomes of intensive family preservation services.|
|Next Article:||Social work and vulnerable families: Economic hardship, protective factors, and service success.|