Printer Friendly

New household survey and the CPS: a look at labor force differences.

In September 1984, the Bureau of the Census released initial statistics from a new household survey, the Survey of Income and Program Participation (SIPP). The survey, which was developed over many years, is expected to provide an indepth look at the incomes of Americans and the extent to which governmental assistance plays a part in their lives. This survey also includes selected information about labor force activity, because labor force activity and the receipt of certain types of income are closely related.

The Current Population Survey (CPS), a monthly survey conducted by the Bureau of the Census for the Bureau of Labor Statistics, is the source of the government's statistics on labor market activity. It has been in existence since 1940 and is the oldest continuous household survey in the world. All other household surveys conducted by the Census Bureau, including the SIPP, are modeled after the CPS. Periodically, the labor force concepts and survey procedures used in the CPS undergo reviews by presidentially appointed commissions to ensure that the data produced continue to be as accurate and as representative of national labor market trends as possible. Great care is taken to see that the measures of labor market activity are consistent over time. As a result, the CPS is a key source of data for both guiding economic policy and understanding the labor market.

Because SIPP data are now available on a regular basis, it is appropriate to examine the labor force estimates from the two surveys to determine the differences between them. And clearly, differences should be expected. Both surveys have different conceptual underpinnings, and methodological differences abound. Nevertheless, if the reasons for the differences between each survey's labor force estimates can be isolated, the SIPP data may prove to be an important complement to the CPS.

This article compares the labor force concepts and methods used in SIPP with those in CPS. The basic labor force estimates from both surveys are then discussed. Of the many possible reasons for the differences in labor force estimates, two are examined in detail: coverage and reference period differences. A concluding section discusses the future research required to understand more fully SIPP and CPS labor force differences.

Conceptual and methodological issues

The CPS came into existence as a result of a need for better information on the number of persons looking for work during the Great Depression. It is primarily concerned with measuring the current labor force activities of the Nation's population--that is, how many persons are working, how many are actively looking for work or are on layoff from jobs, and how many are not in the labor force. Among other things, the CPS is the source of the Nation's "official" unemployment rate. SIPP evolved in the 1970's when a need was recognized for better information on the incomes of persons and the extent to which they received income from government transfer programs. If thus is basically an income survey, but because it is important for policy purposes to look at all income, including that earned through work, questions are necessarily asked in the survey concerning labor market activities.

At the core of the CPS labor force data is the "activity concept." This amounts to identifying a person's major activity in relation to the labor market during a 1-week reference period. A battery of questions determines whether these activities fit the official concepts of employment and unemployment. The activity concept is also used in SIPP, but somewhat differently. The concept is also used in conjunction with information on some and possible program participation. It involves ascertaining whether a person had a job at any time during the period 4-month reference period. A sequence of questions in the SIPP questionnaire then establishes whether there were any periods of absence from a job or periods when one was looking for work or on layoff.

The concepts of "employment" and "unemployment" have very precise meanings in the CPS and use of these terms is avoided in SIPP. The comparable terms in the new survey--"with job" and "looking for work or on layoff"--are also carefully defined. Conceptual and methodological differences between the two surveys are summarized in exhibit 1 on pages 6 and 7.

SIPP. SIPP is a longitudinal survey of persons, that is, data are collected for the same person over time. The sample contained individuals in 20,000 households when the survey started in October 1983--January 1984 period; a second panel of 13,500 households was introduced in the February--May 1985 period. The entire sample is divided into four rotation groups, and one rotation group is in operation every month. This "staggered" sample design permits the full sample to be interviewed over a 4-month period; one complete interview sequence of the four rotation groups is referred to as a "wave."

Persons in the households of both panels remain in the survey for 2-1/2 years, that is, through eight interviews. For those persons who move out of a household, attempts are made to follow them and thus retain them in the survey. (In cases where household units are occupied by entirely new residents, no further interviews at that address are conducted.) Interviews are obtained, however, for persons moving into housing units containing SIPP sample members.

SIPP sample members are interviewed during the first 2 weeks of the month. Each person 16 years of age and over is interviewed if present in the household, and "proxy" interviews are obtained only from responsible household members. A typical interview lasts approximately 10 to 15 minutes per individual. The reference period for the labor force, income, and program participation questions is the previous 4 months, although the labor force questions are asked in reference to each week of the 4-month period (involving up to a 19-week recall period).

Persons with jobs are defined as those who have "... a definite arrangement for regular work for pay every week of every month." These persons may have been absent from their job because of illness, vacation, a labor dispute, bad weather, or other reasons, but still are considered as holding a job. As long as a job was held at some time during a week, a person is considered to have had a job the entire week. Individuals are classified as looking for work if they did not work in a week, but indicated that they looked for work and were available for work. They are classified as on layoff if they had been released from an employer due to a temporary lack of materials or slack work. In addition, persons who are to report to a new wage or salary job within 30 days are considered to be looking for work or on layoff.

Because of the length of the reference period, individuals can have more than one labor market status during the 4 months. For example, as depicted below, a person could have been out of the labor force (N), then started looking for a job (U), and then found one (E):

This is a key difference from the CPS concept of "current" labor force activity and accounts for some of the difference in the labor force statistics from the two surveys. This multiple-status possibility means that, basically, SIPP is measuring "work experience" of individuals during the 4-month reference period and thus is similar in this respect to the CP supplement conducted each March wehn information is collected on the labor force activities of the population in the previous calendar year.

Although SIPP data are collected using a 4-month reference period, all of the data--including the labor force estimates--are issued for a calendar quarter and are "monthly averages" for that quarter. This means that the labor force status estimates are averages of the various statuses individuals had in each of the 3 months of the quarter. In other words, monthly averages were calculated for each of the possible labor force statuses that occur in a month's time, and, just as multiple statuses are possible during the 4-month reference period, so too are they possible in a month.

The SIPP data in this article refer to persons 16 years of age and over from nonfarm households and members of the Armed Forces living with their families either on or off post. Data for persons from farm households are currently being examined for possible inclusion in future SIPP quarterly reports.

THE CPS. The CPS is a sample survey of 59,500 households that is conducted monthly. Households come into the sample for 4 months, are rotated out for 8 months, and then return for 4 months. This rotation pattern lends stability to the estimates of month-to-month and year-to-year changes, as the measurement of change over time is perhaps the most important aspect of the survey. The rotation pattern also yields a degree of longitudinality, in that the same households--actually, addresses--are tracked over a 16-month period.

Interviews are conducted initially in person--for the first and fifth month-in-sample households--and subsequently by telephone, after a relationship has been established between the enumerator and household members. On average, about 65 percent of a month's interviews are conducted by telephone. Generally, a responsible household member will answer questions on behalf of all members 16 years of age and older. Interviews typically last 5 to 7 minutes per person. The reference period for interviews is the week containing the 12th day (except in December, when it is often a week earlier). Enumeration is carried out in the subsequent week, the week of the 19th. Thus, questions on labor market status are framed in the context of one's activity "last week," a very short recall period. Data for the reference week are used to represent individuals' statuses for the entire month.

Classification of labor market status is carried out on a priority basis in which employment takes precedence over unemployment, and unemployment takes precedence over not being in the labor force at all. In the course of an interview, persons are asked if they performed any work at all--that is, an hour or more--"for pay or profit." A positive response results in their being counted as employed. In addition, persons who did not work at all in the week but were temporarily absent from jobs for personal reasons such as vacation and so forth (same as in SIPP) are also counted among the employed, regardless of whether they are paid for their time off. A comparatively small group of persons who work without pay on a family farm or business enterprise for 15 hours or more in the reference week are included in the employed counts.

To be counted as unemployed, a person must not, of course, have worked at all and must have actively looked for work some time in the prior 4-week period (for example, registered with a public or private employment office, applied directly to an employer, placed an ad, mailed a letter of application, and so forth) and be currently available for work. Persons who are on layoff (and expect to be recalled sometime in the future) and those who are waiting to start a new wage or salary job within 30 days need not be looking for work to be included among the unemployed. Those who are either employed or unemployed are in the labor force. Those doing neither are in the third major category: not in the labor force. In contrast to the illustration of multiple statuses in SIPP shown earlier, the circles for CPS would be tangential rather than possibly interlocking:

The differences

Qualitative. As noted earlier, differences in the operation of both surveys could easily affect any comparisons of the data, even if the surveys were conceptually identical (which, as we have seen, they are not). For example, an important qualitative difference we know little about concerns interviewers and their training. Given the different focus of both surveys, it is possible that SIPP interviewers have different attitudes and expectations concerning the labor force questions. Training differences, too, many have an effect because the SIPP questionnaire is longer and more complicated to administer than the CPS's. Processing and estimation procedures are another possible source of the labor force differences of which little is known at this time. While the CPS procedures have been in operation for many years, SIPP's have only recently been set in place. The complexity of these procedures would make them a possible candidate to which some labor force differences could be attributed. Differences in recall period, as will be seen, can affect the data. And last, but not least, are the differences involving the SIPP and CPS questionnaires (this is discussed in greater detail later). It is known from survey research that the wording of questions and their placement can affect respondents' answers.

Survey nonresponse is an important factor in assessing the qualitative differences. Although nonresponse to specific income questions in SIPP is less than that reported in the March CPS income supplement, the overall nonresponse rate to SIPP interviews is considerably higher than in the monthly CPS (see exhibit 1.) The reasons for this may stem from a variety of factors, including sensitivity over the questions being asked and the amount of time needed to complete a SIPP interview. In any event, differences in rates of nonresponse may account for part of the differences in the labor force dta from both surveys.

Unlike SIPP, the operation of the CPS has been scrutinized for many years, not only by statistical agencies responsible for it but also by independent researchers and presidential commissions. Topics such as questionnaire design and content, estimation procedures, nonresponse, and so on have been examined for their effect on the data. Because of this, the quality of the CPs labor force estimates is well known, while SIPP's awaits investigation. Consequently, qualitative differences between the two surveys cannot be ruled out in any accounting of the differing labor force estimates.

Data. The labor force categories used in SIPP are different from the traditional three used in the CPS. Whereas the CPS sorts the entire civilian noninstitutional population hierarchically into three mutually exclusive groups--employment, unemployment, and not in labor force--the SIPP produces more numerous and complicated categories. (See table 1.) The longer reference period in the SIPP concept permits multiple statuses (as noted earlier), a result of the design and intent of the survey. In SIPP, interest is in the strength of individuals' attachment to the job market over time in relation to their household income situation; in CPS, the interest is with measuring the labor force activities of the population at a single point in time.

Table 1 shows quarterly averages of SIPP labor force estimates for the third quarter 1983 through the second quarter 1984, and an average of these four quarters. According to the data, an average of 98.6 million persons had jobs for an entire month during this 1-year period, and 3.9 million had jobs for only part of the month. We have used averages of the four quarters in the SIPP and CPS labor force comparisons because they are more representative than are data for any single quarter. (Data differences between SIPP and CPS during the four-quarter period are discussed later.)

The SIPP estimates in table 1 can be collapsed into fewer categories so as to facilitate comparison with the CPS estimates. A monthly average of 102.6 million persons held jobs in the third quarter 1983 through second quarter 1984 period (98.6 million had jobs for the entire month and 3.9 million had jobs for part of the month). (See table 2.) Also an average of almost 12.0 million persons spent time looking for work or were on layoff (481,000 had jobs the entire month, but were on layoff for some of the time; 1.9 million ahd jobs for part of the month, but also spent time looking for work or were on layoff; and 9.5 million had no jobs during the month and spent varying amounts of time looking for work or were on layoff). Obviously, the two groups (those with jobs and those who spent time looking for jobs or were on layoff) are overlapping. Within each group there are small proportions of persons with jobs and of persons who looked for work or were on layoff.

One of the first SIPP--CPS differences that can be seen in table 2 that is not affected by overlapping groups involves the population covered. The civilian noninstitutional population age 16 and over (according to independently derived estimates) in the CPS was 175.3 million, while the estimate of the noninstitutional population age 16 and over (also an independent estimate) for SIPP was 171.3 million. The primary reason for the difference is that persons in farm households were excluded from the SIPP estimate. Operating in the opposite direction is the fact that military personnel living in households within the continental United States are surveyed in SIPP but not in the CPS. (Resident Armed Forces members are, however, combined with CPS estimates for counts of total employment and labor force, based on figures provided by the Department of Defense.)

In terms of employment, SIPP found 102.6 million persons with jobs for at least part of a month, while the CPS had 103.2 million employed persons. If persons with jobs living in farm households had been included in the SIPP estimate, the resulting number would have exceeded the CPS employment estimate. In addition, the SIPP data relate to any employment within a month and not just in 1 week as in the CPS. The longer reference period in SIPP permits a greater likelihood for some employment to be captured than does the CPS.

A proportionately larger difference involves persons looking for work or on layoff in SIPP versus unemployed persons in CPS. Table 2 shows that when the various looking for work/on layoff groups in SIPP are combined, there is a total of almost 12.0 million persons in this category. This figure greatly exceeds--by 28 percent--the CPS estimate of unemployment of 9.3 million. (This difference would be even greater if persons from farm households who were on layoff or looking for work were included in the SIPP estimate.) Again, because of the longer reference period in SIPP, more people who were jobseeking or on layoff over the course of an entire month are reflected in SIPP than in the CPS. For example, a person looking for a job in the first week of the month, but then employed in the second, would be counted as employed in the CPS, whereas in SIPP the weeks of unemployment and employment would be counted. Also, the longer recall period could easily result in some "telescoping" (memory bias with respect to time) of job search, either within the period or from a prior period; it could also affect the recollection of marginal jobs, which, having gone unreported, would result in increased unemployment reporting.

Persons outside the labor force totaled 3.5 million more in CPS than in SIPP. If farm households were included in SIPP, its estimate would be closer to the CPS estimate. The longer reference period in SIPP allows more people only mildly interested in the job market to be counted as having labor force activity because there is more time for them to manifest their job interests.

The SIPP and CPS labor force estimates for the average of the four quarters extending from the third quarter of 1983 through the second quarter 1984 were also compared by age-sex groups. The "with job-employment" comparison revealed that for most of the groups the estimates from both surveys probably were not different in a statistically significant sense. However, the "looking/layoff-employment" comparison indicated that significant differences existed and that the degree to which the SIPP estimates were higher varied by age-sex group. (See table 3.) Among men, for example, the amount of the disparity declined from about 38 percent for teenagers to 13 percent for those age 55 to 64. Among women there was no apparent pattern in the differences.

Accounting for the differences

To understand better how and why the labor force estimates from SIPP and CPS differ, three adjustments were made to the SIPP data, making them conceptually more similar to the CPS data. First, persons living in farm households were included in the SIPP estimates; second, members of the Armed Forces living in households on or off post were subtracted from the SIPP estimates; and third--and most importantly--only SIPP data relating to the CPS reference weeks were used.

The last adjustment was possible because labor force activities in SIPP are recorded weekly during the 4-month reference period, and thus the week containing the 12th of the month could be identified and the data tabulated. An important source of difference still remained: thelength of the recall period. In the CPS, of course, recall problems are at a minimum, as one is asked about activities in the previous week (or the previous 4 weeks in the case of jobseeking). In SIPP, where each interview covers 4 months, the recall period ranges from 1 week to 18 or 19 weeks. The recall bias problem, therefore, is potentially large in SIPP, particularly for identifying individual weeks where telescoping could easily occur.

These adjustments attempted to account for some of the conceptual differences between SIPP and CPS. Survey coverage became more similar. The labor force classification systems also became more similar in the sense that it was possible to fit SIPP respondents into CPS's mutually exclusive categories. And the SIPP data became averages of the same weeks as the CPS data.

Table 4 shows the results of these three adjustments. Differences continue to exist with the employment, unemployment, and not in the labor force estimates. Employment was 1.2 million lower in SIPP than in the CPS--102.1 versus 103.2 million. In contrast, the unemployment estimate was 1.5 million higher, or about 16 percent greater, in SIPP than in the CPS. (As discussed earlier, the SIPP count of persons looking for work or on layoff before these adjustments were made was 28 percent greater.) With respect to persons outside the labor force, SIPP found fewer persons than did the CPS (62.0 versus 62.7 million).

An examination of the adjusted SIPP labor force estimates for each quarter beginning with the third quarter of 1983 and ending with the second quarter of 1984 shows that for employment and unemployment, there appears to be a narrowing of differences, especially in the unemployment estimates. (See table 4.) In the third quarter of 1983, the unemployment estimates from both surveys differed by 20 percent, but then fell in each subsequent quarter. By the second quarter of 1984, the difference amounted to only 11 percent. The not-in-labor-force estimates from SIPP were below those of CPS throughout the period, although only the fourth quarter 1983 and first quarter 1984 estimates were significantly lower. The causes of this narrowing are not readily understood, but certainly the conditioning of SIPP respondents to the interview process has to be an important factor.

It should be noted that the population figures are not identical, as they should be theoretically, the SIPP figure being some 461,000 below the CPS level. This occurs because SIPP has a slightly different universe than the CPS, in that it includes persons in the military living in the households. Although these persons were removed for purposes of this comparison, because of some errors in coding of such persons and other reasons, a slightly different population estimates was produced. Clearly, the diffeence is spread proportionately across the civilian labor force, employment, unemployment, and not in the labor force categories and thus should be taken into account in examining the SIPP-CPS labor force difference.

Adjustments were made by age-sex groups in teh "looking/layoff-unemployment" category. (See table 5.) Although differences were reduced in all of the age-sex groups, SIPP estimates were still greater than those for the CPS. The differences were most evident among younger men and middle-aged women.

To understand why the adjusted SIPP and CPS estimates continue to differ, many other factors would have to be accounted for besides coverage and reference period differences. These would include, for example, sample designs, nonresponse rates, training of interviewers, experience of interviewers, mode of interview, processing and estimation, questionnaires, length of recall periods, variance differences, and so on. While a comparison of such factors is outside the scope of this article, we cann suggest how two of these--the questionnaires and recall periods--may account for some of the remaining difference.

The SIPP and CPS labor force questionnaires are very different. SIPP begins with a question about whether a person had a "job or business" at some time during the 4-month period. For persons who had jobs, subsequent questions are asked about how long they had their jobs in the reference period, whether they had been absent from them and why, and whether they looked for work or were on layoff wen they did not have jobs. For persons who did not have jobs during the entire period, questions are asked if they looked for work or had been on layoff and, if so, for how long. All persons who looked for work orwere on layoff are asked about their availability for work. (Exhibit 2 on page 11 contrasts relevant SIPP and CPS questions.)

In the CPS, the first question asked concerns activites during the reference week (working, looking for work, going to school, and so forth). If the respondent does not indicate "working" that week, a second question asks whether any work was performed at all. It is not until the third question, also concerning the existence of employment, that the word "job" or "business" appears. If the person had no job, the question then focus on whether or not the person was looking for work. The looking for work questions include a query on the search methods used so as to verify that job-seeking took place.

These questionnaire differences undoubtedly contribute to the statistical differences. As indicated above, the CPS obtains more information about the possibilities of performing any work at all and thus should be expected to show a greater employment count. It follows that, having identified fewer people with jobs than in the CPS, SIPP can be expected to find more persons looking for work, particularly those not reporting marginal work. Also, the SIPP questions about jobseeking are less probing than in the CPS in the sense that they do not verify job search and thus can overstate unemployment to a degree.

The SIPP and CPS recall periods also are very different. As indicated earlier, CPS respondents report activities in the prior week, whereas SIPP reporting can extend up to 19 weeks, opening up the possibility for recall bias. Some research has been done in the area of recall bias in retrospective labor force questionnaires. While the evidence is mixed as to the direction of the bias, one researcher found that respondents tend to overstate the amount of unemployment experienced in the immediate past and understate unemployment in the distant past (more than half a year). Spells of unemployment experienced in the distant past tend to be forgotten.

Questionnaire and recall period differences, therefore, could be working hand-in-hand to cause SIPP's lower measurement of work activity and higher measurement of jobseeking relative to the CPS. And their joint effect may be especially evident among those groups who are only marginal or casually attached to the labor market. We are suggesting that many of these individuals with such marginal attachments, when interviewed in SIPP and asked the sole question about a "job or business," may reply in the negative because they feel their work activities (in the prior 4 months) did not constitute a "real" job. They may perceive a job as involving fixed working hours, a work station, or an identifiable employer who, in addition to paying a wage or salary, also provides certain fringe benefits. Moreover, they are more likely to forget jobs of comparatively short duration, especially when a long recall is involved. In the CPS, on the other hand, such individuals would very likely be enumerated as employed because there are several questions that will pin down any employment, including marginal jobs, and there is a short recall period.

Having neglected to report marginal work in the SIPP interviews, these persons would most likely report that they were looking for work at some time in the prior 4-month period, especially because they had found odd jobs in their job search.

Future research

Clearly, this has only been a preliminary examination of the differences in the labor force estimates obtained from SIPP and CPS. Coverage and reference period differences in both surveys were examined as sources of the disparate estimates and did indeed help account for differences. In addition, questionnaire and recall period differences were considered as additional sources for the remaining discrepancies in the estimates. What direction should future research take?

As discussed, the possible sources of the SIPP-CPS labor force differences are numerous. A full fledged examination of each and every difference between the two surveys would encompass the subjects raised in a 1978 volume of the Department of Commerce, An Error Profile: Employment as Measured by the Current Population Survey. But such an examination may be along way off because SIPP is still in its infancy, and there is much yet to be learned about the survey itself. The full effect of the SIPP operation on the data has yet to be studied: How much bias stems from the long recall period? Does the amount of time taken to conduct an interview affect respondents' answers? To what extent is rotation group bias (or time-in-sample bias) present in the data? Although a survey research agenda hs been established, topics await investigation. In addition, as SIPP matures, fundamental changes may be made, for example, in the estimating procedure or the questionnaire (small changes are being planned for the labor force questions in the 1986 panel) which might affect the data. In other words, SIPP is in a "wetting-in" process during which changes in some labor force estimates may simply be a reflection of internal changes. Consequently, any rigorous examination of SIPP and CPS labor force estimates may be some years off.

Even after the SIPP is as fully integrated and stabilized as the CPS, differences in labor force estimates may still remain unexplained. Differences may result from the purposes of the two surveys. For example, given SIPP's emphasis on program participation, respondents who report the receipt of government transfer payments may be more likely to indicate job search to justify their participation, even when search did not take place.

All of this, of course, reemphasizes the preliminary nature of this study. In all likelihood, it will be many years before a better understanding of the relationship between SIPP and CPS labor force data can be achieved. In the meantime, the prudent use of these data should be as they were originally intended: SIPP's as information for interpreting income and program participation and CPS's as a measure of current labor market activity.
COPYRIGHT 1985 U.S. Bureau of Labor Statistics
No portion of this article can be reproduced without the express written permission from the copyright holder.
Copyright 1985 Gale, Cengage Learning. All rights reserved.

Article Details
Printer friendly Cite/link Email Feedback
Title Annotation:Current Population Survey
Author:Ryscavage, Paul M.; Bregger, John E.
Publication:Monthly Labor Review
Date:Sep 1, 1985
Previous Article:The National Longitudinal Surveys of Labor Market Experience: an annotated bibliography of research.
Next Article:Area wage surveys shed light on declines in unionization.

Related Articles
Foreign born in the U.S. labor market: the results of a special survey.
Reconciling divergent trends in real income.
Workers at the minimum wage or less: who they are and the jobs they hold.
Revisions of state and local area labor force statistics.
A year's work: labor force activity from a different perspective.
Work-time reduction in the U.S. and Western Europe.
Do more-educated workers fare better following job displacement?
A surge in growing income inequality?
Substantial changes in CPS. .
Estimating gross flows consistent with stocks in the CPS: the basic gross-flow table formerly used by the Bureau of Labor Statistics in examining...

Terms of use | Copyright © 2016 Farlex, Inc. | Feedback | For webmasters