A comparison of patient self-administered and investigator-administered measurement of quality of recovery using the QoR-40.
A 40-item questionnaire (the QoR-40) had been previously developed to measure five dimensions of quality of recovery after anaesthesia and surgery. Each of the 40 items is rated on a scale of 1 to 5, with a maximum score of 200. In this study we compared patient self-administered with investigator-administered QoR-40. We studied 62 postoperative patients within 48 hours of their surgery. Agreement between the two methods was analysed using the intraclass correlation coefficient, bias and limits of agreement. There was strong correlation between the investigator-administered and patient self-administered QoR-40 scores, intraclass correlation coefficient 0.86 (95% CL 0.77 to 0.92), P<0.001. The bias and limits of agreement were 3.1 and -22 to 28, respectively. There were 10 (16%) patient self-administered questionnaires that were not completed at first attempt. The time to complete the questionnaire when investigator-administered was 253 (16) s [mean, (SD)], and on first attempt for patients was 362 (19) s, P<0.001. The QoR-40 is as valid measure of postoperative recovery when administered with the assistance of an investigator as compared with the patient self-administered version. Investigator-administered measurement of the QoR-40 is a more efficient use of resources, as complete and more timely data are collected.
Key Words: anaesthesia, recovery, measurement, quality, psychometric testing
Measurement of the quality of healthcare has gained increasing importance over the last 20 years. This has been driven, at least in part, by an interest in providing efficient and appropriate services and the need to undertake objective and yet, patient-focused research in order that these aims can be achieved. The service nature and safety of modern anaesthesia practice has led to the need for additional approaches to measuring quality in anaesthesia.
Some groups have developed measures of patient satisfaction and/or quality of recovery after anaesthesia and surgery (1-9). One of these, a 40-item questionnaire (the QoR 40), has been found to be a valid, reliable and responsive measure of quality of anaesthesia care (5,6,8,10). However, despite these features, there remains a small but significant proportion of patients who have some difficulty completing the questionnaire, typically manifesting as incomplete data. This necessitates checking the QoR-40 for completeness, identifying internally incongruent responses and in some cases directly assisting the patient with the completion of the questionnaire. We therefore were interested in evaluating the validity of clinician or investigator-assisted use of the QoR-40 because this should improve its accuracy and timely completion. Our study hypothesis was that this would provide equivalent and therefore valid measurement of the QoR-40 in a typical surgical setting.
Institutional ethics committee approval was obtained for this study. Patients who had undergone elective surgery in the previous 48 hours were identified from the theatre surgical list postoperatively and were approached by a member of the research team who gave a brief explanation of the study and obtained informed written consent. Patients were excluded if they were less than 18 years of age, had poor English language comprehension, drug or alcohol dependence, a psychiatric disorder or a medical condition impairing accurate and objective completion of the questionnaire (e.g. critically ill, tracheally intubated, uncontrolled pain). Baseline demographic data were collected for descriptive purposes.
The measurement tool used in this study, the QoR 40, is a 40-item questionnaire, developed for the purpose of measuring quality of recovery following anaesthesia and surgery (6,8,10). It consists of the following dimensions, with the number of items for each dimension in parentheses: (i) physical comfort (12 items), (ii) emotional state (9 items), (iii) physical independence (5 items), (iv) psychological support (7 items) and (v) pain (7 items). Each item is rated on a scale of 1 to 5, providing a minimum score of 40 and maximum of 200.
The QoR-40 questionnaire was completed twice, once by the patient (self-administered) and once with direct assistance from one of the investigators (investigator-administered). The investigators included two experienced research nurses (both female) and two intermediate level anaesthesia residents (one male, one female). The order of administration was allocated randomly in blocks of 10 using a table of random numbers. Each was administered within 30 minutes of the other.
For the investigator-administered questionnaire, the investigator asked the patient to rate each of the items on the QoR-40 by reading each item in turn, asking the patient to provide a rating and then proceeding on to the next item until all 40 items had been addressed. For the patient self-administered questionnaire, each patient was given a brief explanation on how to complete the questionnaire and after confirmation of their understanding of these requirements, the investigator then left the room. The patient was left to complete the questionnaire and asked to note the time taken to do this. A small desk clock was provided for this purpose. The investigator returned after 15 to 30 minutes to collect the completed questionnaire. The questionnaire was checked for completeness by the researcher, and if incomplete, the patient was asked to answer the missing items. Whether the questionnaire was completed at first attempt was noted. The questionnaire was then placed in an opaque envelope and sealed. In both cases, the time taken to complete the QoR-40 was recorded. Completed questionnaire data were coded and entered into a database by an investigator blinded to group identity.
Our chosen sample size was guided by previous studies (6,8,10), aiming to detect a 20% reduction in correlation (0.80 [strong agreement' 6] versus 0.64 [unacceptable agreement (5,6))], using a one-sided test with type I error 0.05 and 80% power. The required number of patients to complete the questionnaire was calculated at 57 patients. To account for possible ineligible or uncompleted questionnaires we further increased the sample size to at least 60 patients.
Demographic and perioperative data are presented as mean (standard deviation [SD]), median, range or number (%). Selected data are presented as mean (95% confidence intervals [CI]). Agreement between patient self-rated and investigator-administered QoR-40 scores was measured using the intraclass correlation coefficient (ICC) (11) and the Bland-Altman method (12). Unlike simple correlation, such as that done by the Pearson or Spearman method which estimate the association between two distinct variables without consideration of their agreement, the calculation of ICC incorporates replication and so is a measure of reproducibility. An ICC greater than 0.75 indicates excellent reproducibility.
Other exploratory analyses of association were done using Pearson and Spearman rank correlation. We used multivariate linear regression to identify factors associated with the extent of the discrepancy between investigator-assisted and patient self-administered QoR-40 scores. All statistical analyses were performed using SPSS for Windows V10.0 (SPSS Ltd., Chicago, IL, U.S.A.). A P value of < 0.05 was considered significant.
We enrolled 62 patients and obtained complete data from all patients. Our study population included a diverse range of patients and surgical procedures (Table 1). Patients were assessed at a mean (SD) 21(12)h, range 1 to 48h, after surgery.
There were ten (16.2%) patient self-administered questionnaires that were not completed at first attempt, requiring the investigator to assist the patient to complete the missing items. The time to complete the questionnaire when investigator-administered was 253 (16)s and on first attempt for patients was 362 (19)s; mean difference 109s (95% CI: 53-164s), P<0.001.
There was excellent correlation between the investigator-administered and patient self-administered QoR-40 scores, ICC=0.86 (95% CI: 0.77 to 0.92), P<0.001 (Figure 1). There was no significant difference between investigator-administered and patient self-administered QoR 40 scores, 174 (16.2) vs 171 (19.5), respectively, P=0.056.
The bias and limits of agreement were 3.1 and -22 to 28, respectively. A Bland-Altmann plot demonstrates agreement between the two methods of measuring the QoR-40 (Figure 2).
The patient self-administered QoR-40 scores for minor, intermediate and major surgery were 176, 174 and 162, respectively; median test P=0.06. The investigator-assisted QoR-40 scores for minor, intermediate and major surgery were 182, 174 and 171, respectively; median test P=0.06. There were no patient or surgical factors significantly associated with a larger difference between investigator-administered and patient-administered QoR-40 scores; factors tested were: patient age, patient gender, ASA status, education level, occupation, English-speaking at home, extent of surgery, type of investigator (research nurse or resident), investigator gender and patient-investigator gender interaction (all P>0.05).
[FIGURE 1 OMITTED]
[FIGURE 2 OMITTED]
We have studied a heterogenous adult postoperative patient population and demonstrated a strong correlation and agreement between patient self-administered and investigator-administered QoR 40 scores as a measure of quality of recovery. Because our study included a diverse range of surgical patients, typical of our institution's casemix and that of many other hospitals, we believe our findings can be applied to many surgical settings. The time required to complete the questionnaire, completion rate and ability to discriminate recovery from minor, intermediate and major surgery, support the preferential use of the investigator-assisted method. These findings are supported by our unit's extensive experience in use of the QoR-40 in over 1,000 patients recruited to our previous trials.
We measured the QoR-40 using two approaches, each within 30 minutes of the other. In this short time-frame the patient's quality of recovery after anaesthesia was unlikely to have changed. To balance any effect this may have had, we randomly allocated the order of administration to avoid this source of bias. Repeat administration of a measurement tool introduces the possibility of patients recalling responses from the first administration and thus increasing the agreement between the two measurements. It is not possible to avoid this effect but the extensive number (n=40) of varied items would reduce such a bias. Whilst our study demonstrated a strong correlation between these two methods, other covariates could have influenced the results. However, we did not identify any patient or surgical factors that confound the agreement between methods, adding to the value and generalizability of our results.
Previous studies have shown that the QoR-40 can be completed reliably and accurately by most patients in less than ten minutes (6). The present study found that it could be given to patients in the postoperative period, without prior explanation or practice, and satisfactory results could be obtained. We have also demonstrated that when conducted by a young clinician or experienced research nurse, the time taken to complete the questionnaire is significantly less than when patients self-administered the questionnaire. We also showed that patients did not complete the questionnaire at the first attempt in about one in six (16%) occasions. This is to be expected as patients can find completion of a questionnaire in the postoperative phase time-consuming and possibly burdensome. This has implications for the QoR-40 as a measure in future studies or as a quality assurance tool.
We have demonstrated that the QoR-40 is an equally reliable and valid measure of postoperative recovery when administered by an investigator as compared with patient self-administration, but requires less time and produces more complete data. This is a more efficient use of resources, as complete and more timely data are collected and the responses require no further cross-checking.
Any measurement tool must accurately measure what it sets out to measure and not be influenced by other factors (2,13). Measuring quality or satisfaction with anaesthesia services is no different and must not be confounded by surgical and nursing care, interpersonal skills of health care workers, psychological state of the patient and timing of use of the tool (2). This study was designed to confirm that these factors do not influence the measurement of quality of recovery when using the QoR-40. This is a further test of its validity and clinical utility. However, it must be recognised that we did not have sufficient power to identify all of these factors. This study had patients with QoR-40 scores in the range 120 to 200 and so there is no information from patients with an extremely poor recovery; it is our experience that such scores are rare (<0.05%).
In this study we have shown that in the early (<48h) postoperative period, in a heterogenous adult surgical population, investigator-administered use of the QoR-40 questionnaire is a valid and reliable tool for measurement of postoperative recovery. When compared with patient self-administration, investigator-administered measurement of the QoR-40 ensures completeness and more timely data collection in many circumstances, and this can lead to more efficient use of resources.
Professor Myles was supported by an Australian National Health and Medical Research Council Practitioner's Fellowship.
(1.) Gill TM, Feinstein AR. A critical appraisal of the quality of quality-of-life measurements. JAMA 1994; 272:619-626.
(2.) Fung D, Cohen M. Measuring patient satisfaction with anaesthesia care: A review of current methodology. Anesth Analg 1998; 87:1089-1098.
(3.) Eagle CJ, Davies JM. Current models of "quality"-an introduction for anaesthetists. Can J Anaesth 1993; 40:851-862.
(4.) Tong D, Chung F, Wong D. Predictive factors in global and anesthesia satisfaction in ambulatory surgical patients. Anesthesiology 1997; 87:856-864.
(5.) Myles PS, Hunt JO, Nightingale CE et al. Development and psychometric testing of a quality of recovery score after general anesthesia and surgery in adults. Anesth Analg 1999; 88:83-90.
(6.) Myles PS, Weitkamp B, Jones K, Melick J, Hensen S. Validity and reliability of a postoperative quality of recovery score: the QoR 40. Br J Anaesth 2000; 84:11-15.
(7.) Dexter F, Aker J, Wright WA. Development of a measure of patient satisfaction with monitored anesthesia care: the Iowa satisfaction with anesthesia scale. Anesthesiology 1997; 87:865873.
(8.) Myles PS, Hunt JO, Fletcher H et al. Relation between quality of recovery in hospital and quality of life at 3 months after cardiac surgery. Anesthesiology 2001; 95:862-867.
(9.) Hogue SL, Reese PR, Colopy M et al. Assessing a tool to measure patient functional ability after outpatient surgery. Anesth Analg 2000; 91:97-106.
(10.) Leslie K, Troedel S, Irwin K et al. Quality of recovery from anesthesia in neurosurgical patients. Anesthesiology 2003; 99:1158-1165.
(11.) Morton AP, Dobson AJ. Assessing agreement. Med J Aust 1989;150:384-387.
(12.) Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1986; 1:307-310.
(13.) Kirshner B, Guyatt G. A methodological framework for assessing health indices. J Chron Dis 1985; 38:27-36.
S. T GOWER *, C. A. QUIGG *, J. O. HUNT [dagger], S. K. WALLACE [double dagger], P. S. MYLES [section]
Department of Anaesthesia and Perioperative Medicine, Alfred Hospital, Melbourne, Victoria, Australia
* M.B., B.S., Anaesthesia Registrar, Department of Anaesthesia and Perioperative Medicine, Alfred Hospital, Melbourne, Victoria.
[dagger] R.N., R.M., B.Hlth.Sc., Grad Cert Hlth Info, Clinical Trial Coordinator, Research Unit, Department of Anaesthesia and Perioperative Medicine, Alfred Hospital, Melbourne, Victoria.
[double dagger] R.N., Research Assistant, Research Unit, Department of Anaesthesia and Perioperative Medicine, Alfred Hospital, Melbourne, Victoria.
[section] M.B., B.S., M.P.H., M.D., F.C.A.R.C.S.I., F.A.N.Z.C.A., Director,
Department of Anaesthesia and Perioperative Medicine, Alfred Hospital, Melbourne, Victoria, Professor and Chair, Academic Board of Anaesthesia and Perioperative Medicine, Monash University and NHMRC Practitioner Fellow, Centre for Clinical Research, Canberra.
Address for reprints: Professor P S. Myles, Department of Anaesthesia and Perioperative Medicine, The Alfred Hospital, PO Box 315, Melbourne, Vic. 3004.
TABLE 1 Patient demographic and surgical data (n=62). Number (%) unless otherwise stated. Factor Age, y mean (SD) 50 (17) range 18-87 Male gender 40 (64.5) English speaking at home Always 51 (82.3) Mostly 6 (9.7) Sometimes 3 (4.8) Rarely 2 (3.2) Education level Primary 3 (4.8) Secondary 41 (66.1) Tertiary 11 (17.7) Postgraduate 7 (11.3) Occupation Professional 12 (19.4) Office worker 16 (25.8) Labourer 26 (41.9) Home duties 5 (8.1) Unemployed 3 (4.8) ASA status I 19 (30.6) II 26 (41.9) III 14 (22.6) IV 3 (4.8) Extent of surgery Minor 15 (24.2) Intermediate 31 (50.0) Major 16 (24.2) Surgical specialty Orthopaedic 18 (29.0) General 10 (16.1) Plastics 8 (12.9) Cardiothoracic 7 (11.3) Vascular 6 (9.7) Urology 3 (4.8) Ear, nose and throat 3 (4.8) Other 7 (11.3) Hospital length of stay, days mean (SD) 6.2 (7.7) range 1-34