Printer Friendly

Computer-aided telephone interviewing used in the Hours at Work Survey.

Computer-aided telephone interviewing used in the Hours at Work Survey

The Hours at Work Survey, a new component of U.S. productivity measures, represents the first use of Computer Assisted Telephone Interviewing (CATI) at the Bureau of Labor Statistics.1 CATI refers to the use of interactive computer systems by interviewers during telephone contacts with survey respondents.2 This relatively new process is in contrast to the traditional "batch' survey procedure, in which interviewing, data entry, data verification, and error correction constitute separate functions. This report describes the CATI process and presents the results of using the technique in the Hours at Work Survey.

An overview of the system

The Hours at Work CATI is an interactive system in which multiple users can access a mainframe computer data base using IBM 3270 terminals. CATI software is written in MANTIS, an application development system produced by CINCOM. Inc., and files are managed using VSAM (Virtual Storage Access Method), an IBM file management system. Communication with the data base is controlled by CICS (Customer Infomation Control System), also an IBM software package.

When a user logs onto the system and is properly identified, CATI displays a menu screen which contains six functions. By keying in the appropriate code, the user may select one of the following functions:

Function 1. Used to interview the survey respondent or to reconcile data failing edit criteria.

Function 2. Used to display the names of all respondents within the same primary company.

Function 3. Used to display previous year's data for a recurring respondent.

Function 4. Used to scan the data base for reports that have failed edit criteria.

Function 5. Used to scan the data base by interviewer identification code or by the date scheduled for contacting particular respondents.

Function 6. Used to print paper copies of status reports.

The following are the major features of the CATI system:

Form script. A facsimile of the report form is displayed on the screen for the interviewer, along with reported data and notations on edit failures.

Cursor control. The interviewer has complete cursor control and can move the cursor to any field on the facsimile report form to enter or change data.

Full screen editing. While still on the telephone with a respondent, the interviewer can enter data into the mainframe data base and be immediately informed of entries that fail the system's edit criteria.

After telephone contact is established with a respondent, the interviewer simply follows the edit prompts displayed by the CATI system. If the interviewer is unable to resolve an edit failure during the initial contact, CATI provides for entry of a date on which the respondent should be recontacted for clarification. This information is stored in the data base and the file can be searched later based on the scheduled contact date. If the respondent provides an explanation or a correction, the interviewer moves the cursor to the entry in question, and keys a correction or enters an explanation cobe beside the entry. The system instantaneously edits the record based on the new entry and displays appropriate messages if further clarification is necessary.

With CATI, the interviewer no longer uses a pencil, paper error listing, calculator, and correction form. The edit-reconciliation process is reduced to one task--namely, using a terminal and telephone to interact with a data base containing reported data.

The phasing-in of CATI

The first Hours at Work Survey was conducted in 1982 in order to collect data for the 1981 reference year. The survey collects quarterly and annual data for hours paid (including vacations, holidays, and so forth) and hours actually spent at work for production or nonsupervisory employees at business establishments in all nonagricultural sectors of the economy. The survey is based on a probability design to produce ratios of "hours at work' to "hours paid' for each 2-digit Standard Industrial Classification in manufacturing and 1-digit Standard Industrial Classification in nonmanufacturing. The ratios are then used in computing productivity measures for the major economic sectors.

The first annual survey was conducted entirely by mail. It was not until 1983, the second survey year, that CATI was used. Because it was not known how CATI would affect the survey response, the new system was implemented in stages over a period of several years.

1982: basic operations. The first-year operation consisted of an initial mailing of 1-page questionnaires to approximately 4,200 sample units in March, with two mail follow-ups to nonrespondents between April and June. Upon receipt, the completed reports were checked in and batched for key entry. The reported data were machine edited for validity, and data failing the edit criteria were printed on error listings for review. A report failing the edit was first checked for key entry errors, and then the respondent was contacted by mail or telephone to verify any remaining questionable entries. The resulting corrections were manually transcribed to yet another form which was sent to key entry for processing. Once again, data records with corrections would either pass or fail computer editing. If a record failed, it would go through the verification procedures again. This reiterative process continued until all data passed the prescribed edit criteria.

The data processing was labor-intensive and time-consuming. Approximately one-third of the respondents (1,200 sample units) had to be contacted for data verification. Many of these were contacted by telephone. This led the Bureau to conclude that telephone contact to carify information previously supplied by mail was feasible and cost-effective. Although 76 percent of the initial sample units returned the survey form, the response rate--the proportion of sample units providing usable data--was only 50.4 percent.3 The difference between the two rates occurred primarily because some reported data did not meet the survey definition of "hours at work,' rendering the report unusable. A response is considered usable when a reporting unit provides acceptable data for at least one quarter of the reference year.

1983: the second year. In 1983, the survey operation was modified to incorporate CATI into the edit-reconciliation function. The questionnaires were mailed to the sample units, returned reports were batched, and data were keyed and computer edited as in the previous year. Additionally, the name and telephone number of the respondent were entered into the data base. Using the edit-reconciliation function, interviewers searched the data base for records with error codes. No paper error listings were produced, however. Instead, data failing one or more edits were displayed on a terminal video screen. The respondent was then contacted by telephone and requested to verify the reported information.

CATI produced excellant results. A completion rate (sample units reporting data) of 90 percent yielded a total response rate of 80.1 percent, despite a 75-percent staff turnover from the previous year. Staff productivity was clearly enhanced through use of the new system. Although the improvement in the response rate was not entirely attributable to the implementation of CATI, it was apparent that the system enabled the staff to be more efficient in the edit-reconciliation process.

1984: initial solicitation of nonrespondents. In the third year, a prompting feature was added to CATI for use in conducting initial solicitation. The prompts display questions designed to guide the interviewer through the process of initiating solicitation by telephone. However, it was decided to use this new feature only after all efforts to obtain the data by mail had failed. This decision was based on the following considerations:

While data being requested are generally available from the payroll records of employers, the respondent probably would not be able to provide the information by telephone without some type of prenotification. Mail solicitation gives the respondent more time to research and prepare these data.

Before the Bureau could initiate a contact by telephone, time and resources would be needed to obtain the name and telephone number of the appropriate person to contact within each sample unit. Telephone numbers for sample units were accessible, but reaching an individual in the establishment who could provide the information was more difficult and time consuming.

After efforts to obtain data by mail ended, the CATI solicitation function was used to conduct follow-ups of the 1,079 nonrespondents. The sample units were equally and randomly assigned to each of five interviewers, who were given 10 workdays to complete the contacts and to obtain data using the CATI system.

Completion rates, measured as usable responses divided by eligible assigned units,4 varied by interviewer from a low of 21.5 percent to a high of 78.6 percent. The interviewer with the highest completion rate had a rate that was 24.7 percent higher than the second most successful interviewer, who had a rate of 53.9 percent. The averae completion rate for all interviewers using CATI was 34 percent.

Both clerical employees and junior level professionals were used in the CATI data collection procedure. All interviewers received training on the IBM 3270 terminal. In addition, recommended interviewing techniques were demonstrated using mock interviews. Although there was a standard script for introducing oneself to the respondent, interviewers were given approval to develop their own introductory statements using established guidelines.

A thorough review of the CATI solicitation procedures used by each interviewer was conducted in order to identify any cause for the large differences in the completion rates. The review revealed that certain personal characteristics of the interviewers, such as a professional demeanor and a good grasp of the subject matter, contributed significantly to effectiveness on the telephone. The results of the review were documented and used in staff selection and in training interviewers.

Collecting "hard data' by telephone generally required several telephone calls to a sample unit: first, to establish a contact who could provide the data or schedule a date when the data would be available, and later, to actually collect the data. The final survey results were comparable to those in the previous year. The response rate was 78.2 percent, of which 9 percent was obtained using CATI solicitation.

1985: monitoring continues. As in previous years, the initial survey contact was done by mail. A certified follow-up mailing was used to contact the nonrespondents. The signature on the certified return receipt was entered into the CATI data base and used to establish an initial contact person within the nonresponding unit. For program analysis, CATI was modified to keep a record of each call, including the duration of the call.

The initial response rate by mail was 32 percent. The response rates for two mail follow-ups were 23.6 percent and 12.5 percent. After mail solicitation ended, there were 1,413 sample units with eligibility not determined, of which 1,120 were contacted using the CATI system. Data were collected from two-thirds of the eligible sample units.

An average of 2.1 telephone calls was made to each sample unit, and the average time spent on each telephone contact was 5.9 minutes. CATI solicitation contributed 17.1 percent to the total response rate, which was 85.2 percent for the year.

Summary on using CATI

The BLS experience with using CATI to improve survey response has been positive. We started the first year with a response rate of 50.4 percent, and three surveys later, attained a response rate of 85.2 percent. This improvement was not wholly attributable to CATI but, rather to a combination of CATI and a more experienced staff. Even so, evidence of the benefits of CATI has been so persuasive that plans have been developed for modifying the Hours at Work Survey CATI to run on a microcomputer. Switching the system from a mainframe computer to a microcomputer will significantly reduce system operating costs and improve online computer response time. The use of CATI is also being tested for other BLS survey programs, in hopes of similarly cutting data collection time and operating costs, boosting response rates, and improving data quality.


1 See Kent Kunze, "A new BLS survey measures the ratio of hours worked to hours paid,' Monthly Labor Review, June 1984, pp. 3-7, for a description of the Hours at Work Survey.

2 Robert M. Groves and Nancy A. Mathiowetz, "Computer Assisted Telephone Interviewing: Effects on Interviewers and Respondents,' Public Opinion Quarterly, Spring 1984, pp. 356-69; J. Merrill Shanks, "The Current Status of Computer-Assisted Telephone Interviewing: Recent Progress and Future Prospects,' Sociological Methods and Research, November 1983, pp. 119-42; and The Role of Telephone Data Collection in Federal Statistics, Statistical Policy Working Paper 12, Prepared by Subcommittee on the Role of Telephone, Mail, and Personal Interviews in Federal Statistics, Federal Committee on Statistical Methodology, Statistical Policy Office (Washington, Office of Management and Budget, 1984).

3 Response rate = Number of units providing usable reports/(Number of eligible sample units Number of sample units with eligibility not determined).

4 A sample unit is deemed "ineligible' if, by the nature of its activity, it does not meet the definitions established for the Hours at Work Survey or the unit is out of business and no longer exists.

Table: 1. Response rates for the BLS Hours at Work Survey over the CATI phase-in period, 1982-85
COPYRIGHT 1986 U.S. Bureau of Labor Statistics
No portion of this article can be reproduced without the express written permission from the copyright holder.
Copyright 1986 Gale, Cengage Learning. All rights reserved.

Article Details
Printer friendly Cite/link Email Feedback
Author:Toscano, Guy A.
Publication:Monthly Labor Review
Date:May 1, 1986
Previous Article:Union response to changes in printing technology: another view.
Next Article:Work-related deaths in 1984: BLS survey findings.

Related Articles
Polling without pressure; research firm finds high-tech niche with interactive touchscreen computers.
Bureau highlights.
Measuring intrahousehold allocation of time: response to Anne E. Winkler; measuring the use of time by more than one individual in a household,...
Emergency survey methods in acute cryptosporidiosis outbreak.

Terms of use | Copyright © 2017 Farlex, Inc. | Feedback | For webmasters