A Systematic Review of Values-Based Psychometric Tools Within Acceptance and Commitment Therapy (ACT).
The ACT model encompasses six processes (acceptance, cognitive defusion, being present, self-as-context, values, and committed action) which together contribute to psychological flexibility (Hayes et al., 2011). Acceptance and cognitive defusion are the primary processes involved in becoming open to direct personal experiences, without seeking to escape or ameliorate such internal events. ACT processes also cultivate a "centered response style," using mindfulness and self-as-context components to focus on the present moment (Hayes et al., 2011). Finally, values and committed action encourage individuals to resolve to act in accordance with their personal values, in spite of potential barriers or ongoing difficulties.
Theories of values have been posited by psychologists, researchers, and philosophers in the past, though these will not be the focus of this article (see Emmons, 2003; Rogers, 1964; Rokeach, 1973, for examples of alternative conceptualizations). Within ACT, values have been conceptualized as guiding principles for living which provide direction for actions. A distinction is made from goals, because values cannot be attained, but rather provide intrinsic motivation and conviction for behavioral pursuits over a long-term period (Hayes et al., 2011; Wilson et al., 2010). Hence, it is proposed that living in line with values generates a sense of purpose, meaning and vitality in our lives (Hayes et al., 2011). Wilson and Dufrene (2009) state that values are "freely chosen, verbally constructed consequences of ongoing, dynamic, evolving patterns of activity, which establish predominant reinforcers for that activity that are intrinsic in engagement in the valued behavioral pattern itself' (p. 66). Therefore the salient aspects of values are that they are personal, self-selected (i.e., not out of obligation or due to avoidance or fear of reprimand), flexible but enduring, related to motivation and impact on both behavior and well-being.
A previous review by Serowik, Khan, LoCurto, and Orsillo (2018) collapsed the content of values into four descriptors. Values were conceptualized as being intrinsic and due to personal preference, being distinct from goals, providing motives for behavior, while also being positively related to well-being outcomes when behaviors were consistent with identified values. However, aspects of values identified by Smout et al. (2014) also noted themes around the clarity of values, the relative importance of values, and the segregation of values from emotions. Although it is not necessary for all measures to include each of the above elements, because this may depend on the function of the values-assessment (e.g., whether this is for clinical goal-setting, versus an assessment of the process of valued-living, or how this construct relates to other outcomes), the utility of each of these characteristics should be considered in the development of values-based questionnaires.
Values are a critical component of ACT; however, to date limited research has examined the unique contribution of values work within interventions. In order to investigate the role of values at a process-level within research, it is critical to ensure that the measures employed within values-assessment are empirically sound. A number of assessment tools have been developed to facilitate such process-level examinations within ACT. Psychological Flexibility has been measured using the Acceptance and Action Questionnaire (AAQ-II; Bond et al., 2011), whereas measures of values such as the Valued Living Questionnaire, Valuing Questionnaire and Chronic Pain Values Inventory (Wilson et al., 2010; Smout et al., 2014; and McCracken & Yang, 2006, respectively) have also emerged as a consequence of ACT, and the need for process-level analyses of its constructs. This review, therefore, aims to identify value-based measures currently in use, and examine what existing studies would suggest about their ability to reliably and validly measure the ACT concept of values.
Aims and Objectives
The aim of this study is to conduct a systematic review of the empirical literature related to values assessment tools within ACT. All available ACT values-measurement instruments will be evaluated on the basis of their measurement properties using published criteria (Prinsen et al., 2018; Mokkink et al., 2018; Terwee et al., 2018), namely, their validity and reliability. Although many ACT randomized controlled trials (RCTs) have examined the effects of interventions on valued living (Lundgren et al., 2012; Pinto et al., 2017; Tyrberg, Carlbring, & Lundgren, 2017), and two reviews (Reilly et al., 2019; Serowik et al., 2018) have examined the content and properties of values questionnaires in a broader sense, no prior review has included ACT-consistent measures that include values as a subscale.
Initial searches for this review were conducted on March 15, 2018 and updated on February 22,2019 (Prospero registration ID: CRD42018099435), and included all available published studies, not subject to publication year. Three databases (PubMed, PsycINFO, and Web of Science) were used in completing the search for existing values-based tools. The search terms used for each database can be seen in Table 1.
All search terms contained the phrases "Acceptance and Commitment Therapy" and "Valu*." This was done in order to ensure that the majority of ACT research relevant to values were included. Search strings did not include the acronym "ACT" given the generality of the word and numerous results yielded. Therefore, all searches that were screened within the title, abstract, and main body of papers (where possible), and additional hand searches were conducted in order to ensure that no appropriate papers or instruments were excluded. The reference lists of included studies, as well as all studies that reported on aspects of reliability or validity of measures (e.g., internal consistency) were examined in order to ensure all relevant, published studies were included.
Studies and reference lists included in existing reviews (e.g., Reilly et al., 2019; Serowik et al., 2018) were also screened in order to ensure no articles were overlooked. Finally, the Association for Contextual Behavioral Science (ACBS) website was consulted for all information relevant to values and values assessment within ACT, and members of the ACBS community were also contacted via the ACT list serv (https://contextualscience.org/ emailing_lists#ACTPROFESSIONALS) in order to ensure that no additional values-measures or validation studies were inadvertently omitted. Database search parameters returned a total of 1,015 results prior to the removal of duplicate papers: PubMed (196 results), PsycINFO (347), and Web of Science (472).
Studies that (1) were not published in journals, or not in English; (2) were not developed for ACT, or are not consistent with the ACT conceptualization of values; (3) were case studies, single-subject research, pilot studies or protocol papers; (4) were qualitative or theoretical in nature; or (5) did not reference the use of any values-based psychometric instrument were not included in the current review. Papers included for analysis were all required to utilize a values-based tool, and focus on aspects of psychometric reliability or validity. A full list of inclusion and exclusion criteria can be seen in Table 2.
Abstracts of all relevant papers retrieved from searches were uploaded to Rayyan, an online application developed for the completion of systematic reviews (Ouzzani, Hammady, Fedorowicz, & Elmagarmid, 2016). Potentially relevant studies (n = 634) underwent the process of abstract screening, and were separately coded by two reviewers (the first author and a PhD candidate), such that only papers that potentially utilized ACT values-based measures were included for full-text review. Agreement of 89.7% was achieved in relation to articles to be retained. Where disagreements occurred, these were resolved through discussion until a consensus was reached. In papers where it was unclear if a values-based measure was included, articles were retained for the subsequent stage of screening.
The remaining 137 papers were subjected to full-text screening by first and third authors, and independently coded so that information regarding the properties of each instrument could be extracted. Each reviewer assessed the method and results sections of all articles for information related to the reliability and validity of the ACT values-measures used, and subsequently labeled the characteristics reported (e.g., internal consistency). Discrepancies in initial coding occurred for 11 papers, and were resolved by consultation and consensus among the authors. Articles that reported on aspects of the psychometric properties of any values-based measure were retained (n = 54), and the reference lists of these articles screened for any further relevant papers (n = 7 potentially relevant papers were identified, and full-texts accessed). A total of 21 papers that focused on the validation of such values-based instruments were subsequently identified. Figure 1 displays the screening and selection processes, with reasons for exclusion of papers.
Data from all selected studies were retrieved from papers by the first and second authors using tables modeled on those provided by the COnsensus-based Standards for the selection of health status Measurement of INstruments (COSMIN) manual for systematic reviews of Patient-Reported Outcome Measures (PROMs; Mokkink et al., 2018; Prinsen et al., 2018; Terwee et al., 2018). As such, the sample size and composition, names of the instruments utilized and their authors, characteristics of the measure (including rating scale, ease of administration, subscale composition, and scoring), and available translations and adaptations of measures were recorded based on the information provided in articles.
Additional information pertaining specifically to the reliability and validity of each measure were extracted as variables of interest. The psychometrically relevant information within articles was evaluated in line with the criteria for good measurement properties and using a Grading of Recommendations Assessment, Development, and Evaluation (GRADE) approach for systematic reviews of clinical trials (Prinsen et al., 2016; Schiinemann, 2013; Terwee et al., 2007), and subsequently pooled so that an overall rating could be applied to the psychometric properties of each ACT values-measure included. Where the information provided in articles was insufficient, the authors were contacted with requests for further details.
Assessment of Bias
The updated COSMIN checklist (Mokkink et al., 2018; Prinsen et al., 2018; Terwee et al., 2018) was utilized to determine the methodological quality of the validation papers. This checklist comprises standards for design requirements and preferred statistical analyses within studies that examine the measurement properties of psychometric tools (Mokkink et al., 2018). Overall, the properties considered within this framework relate to the reliability (including internal consistency, test-retest reliability and measurement error), validity (content validity, construct validity, criterion validity, hypothesis testing, and cross-cultural validity), and responsiveness of the psychometric tools.
These criteria use a four-point rating system, such that each standard can be rated as very good, adequate, doubtful, or inadequate (e.g., for methods of calculating and reporting internal consistency). The rating of the study quality is determined by taking the lowest rating of any standard within the box (i.e., the worst scores counts principle), and can therefore be used to grade the standard of the existing empirical evidence. This ensures that results of less methodologically robust studies will temper the inferences and conclusions made about the properties of a given measure.
Synthesis of Results
The COSMIN checklist allows for researchers to tailor the outcomes according to the focus of the review. Accordingly, each measure was rated on four psychometric properties (construct validity, including structural validity and hypothesis testing, convergent and discriminant validity, internal consistency and test-retest reliability), first using the COSMIN risk of bias checklist, and subsequently the criteria for good measurement properties and the GRADE approach for systematic reviews of clinical trials (Prinsen et al., 2016; Schunemann, 2013; Terwee et al., 2007). It should be noted that the unidimensionality expectation should not be applied to several of the measures because values was considered to be one factor by design (e.g., the Values Wheel and Bull's-Eye Values Survey). Criterion validity was not considered in the data synthesis because there is currently no universally agreed "gold standard" measure of values within ACT. Instead, where existing measures of values were used as comparators with novel measures as a means of validation, this was considered as an aspect of convergent validity.
Content validity was qualitatively assessed. This was done as the COSMIN criteria evaluated content validity in the context of qualitative and quantitative studies of clients' and multidisciplinary professionals' perceptions of relevance, comprehensiveness, and comprehensibility of the measures. As such, the criteria outlined within the COSMIN checklist were utilized as a guide in the discussion of content validity; however, this is not the core focus of the review. Additional factors such as discriminant and incremental validity will also be considered in light of available evidence, though not featured within the COSMIN guidelines. The criteria for good measurement properties were also edited to determine a study as demonstrating sufficient evidence for temporal stability in cases where Pearson's r [greater than or equal to] .70. The adapted criteria for good measurement properties utilized in the current review can be seen in Table 3.
Table 4 presents an overview of the key characteristics of the 24 values-based measurement instruments identified by the current review. This table includes all instruments identified throughout the search that referred to the ACT conceptualization and construct of values, though not all of these measures had published validation studies. Thirteen measures had been previously psychometrically validated in one empirical paper, whereas four had been validated in more than one study, and hence have multiple reports of their psychometric properties. Seven measures (i.e., Survey of Life Principles, Social Values Survey, Academic Values Questionnaire, Daily Valued Action Questionnaire, Work Values Questionnaire, Personal Values Questionnaire, and a novel values measure) did not have journal articles published in order to examine and validate their psychometric properties, and hence were not evaluated in the pooled findings. Measures of values that were either not designed specifically for ACT, or that have since been adapted to be more ACT-consistent were also excluded from the review. Examples of these include the Full Portrait Value Questionnaire (PVQ-40; Schwartz et al., 2001), Personal Strivings Questionnaire (Emmons & McAdams, 1991), and the Sense of Coherence Scale (Antonovsky, 1979).
The overall rating of the quality of the evidence for the 17 instruments that had been the subject of psychometric validation studies was synthesized based on both the methodological quality of the papers and evidence for the robustness of instruments, with results displayed in Table 5. The properties of each values-measure will be discussed in detail below, focusing on their content, construct, structural, convergent, discriminant, and incremental validity.
Bull's Eye Values Survey (BEVS; Lundgren et al., 2012)
The BEVS is an idiographic measure of values, developed in clinical practice. It focuses on Values Attainment across four life domains: work/education, leisure, relationships, health/ personal growth, and Values Persistence, in spite of barriers to valued actions.
Content validity The BEVS addresses values clarification, values-based action, and potential barriers to engaging in values-congruent behaviors. Hence, the foundations are consistent with the theoretical constructs of ACT. Potential value domains were piloted with clients such that the most frequently endorsed items were included in the final measure, as agreed by clinicians familiar with ACT. This suggests comprehensibility and comprehension of the BEVS were both acceptable.
Further Measures of Validity and Reliability All psychometric investigations within this study demonstrated adequate methodological rigor. Results of the first substudy support the bifactor structural validity of the subscales of the BEVS; however, because this analysis was undertaken primarily with the purpose of examining the underlying relationships among the various measures included, it was therefore decided not to examine structural validity as outlined in the COSMIN criteria. Construct validity was supported by virtue of hypothesis testing, with values attainment being positively correlated with psychological flexibility and life satisfaction, and negatively related to depression, anxiety, and stress. The inverse was also true for persistence with barriers and outcomes. Good testretest reliability was exhibited over three time points. The discriminative validity of the BEVS was assessed by comparing scores obtained postintervention across ACT-intervention and control groups. Results found significant differences in outcomes on values attainment ([F.sub.(1,3)] = 93.17, p < .001) and persistence with barriers ([F.sub.(1,3)] = 61.45, p < .001) for those in the ACT intervention group compared to those in the control group; however, this is likely more relevant to the responsiveness and sensitivity of the BEVS than its discriminant validity. Convergent validity with other values-measures was not assessed.
Values Wheel (VW; O'Connor, Tennyson, Timmons, & McHugh, 2019)
The VW is a self-rated idiographic tool, measuring both the relative importance of and behavioral consistency with values using a hand-held disc, with five distinct, movable segments corresponding to each of a person's chosen values.
Content Validity Development of the VW was guided by ACT theory and existing measures such as the BEVS and Schedule for the Evaluation of Individual Quality of Life direct weighting (Hickey et al., 1996). Its idiographic nature allows for values-clarification. The VW considers ratings of both the relative importance of values, as well as behavioral consistency with values, and is consistent with the ACT conceptualization of values. No reference was made to consultation with other researchers or clinicians during its development.
Further Measures of Reliability and Validity At least adequate methodological quality was demonstrated in the methods used to assess the psychometric properties of the VW. Principal Component Analysis (PCA) was conducted, suggesting a two-factor structure; however because no Confirmatory Factor Analysis (CFA) was completed, this did not meet the criteria for good measurement properties. Construct validity of the VW was partially supported, because VW scores positively related to mental health, openness to experience, and behavioral awareness, and negatively correlated with depression, anxiety and stress, but did not meet criteria for good measurement properties (<75% hypotheses supported). Although significant convergence with alternative values-measures (rs = 0.26, 95% BCa Cl [0.12, 0.40], p = .001 with the VLQ, and rs = 0.16, 95% BCa Cl [0.00, 0.30], p = .046 for the valued action subscale of the CompACT) and adequate temporal stability (Spearman's r = .57, .57 and .65 over 3 time points) were demonstrated, these were also insufficient, according to COSMIN criteria.
Valuing Questionnaire (VQ; Carvalho, Pamliera, Pinto-Gouveia, Gillanders, & Castilho, 2018; Smout et al., 2014)
The VQ is a 10-item questionnaire that examines the extent to which individuals have enacted personal values. It contains two subscales: Values Progress and Values Obstruction.
Content Validity The authors of the VQ consulted previous literature in relation to values within ACT, and identified seven key themes through which questions were derived. Clinicians and researchers within the area, as well as individuals with no experience of ACT were consulted in order to rate the relevance, content, and readability of proposed items. Overall, it demonstrates acceptable content validity in relation to the ACT conceptualization of valued living.
Further Measures of Validity and Reliability All studies in the Smout et al. (2014) and Carvalho et al. (2018) validation papers demonstrated at least adequate methodological rigor for the properties assessed. Results supported a two-factor model, with both subscales demonstrating high levels of internal consistency. Construct validity was supported by virtue of positive outcomes on hypothesized correlations, such that Values Progress was positively related to life satisfaction, quality of life, self-compassion, psychological well-being, psychological flexibility, positive affect, and mindfulness, and negatively correlated with depression, anxiety, stress, and negative affect. Furthermore, the inverse was true for the Values Obstruction subscale. However, convergent validity with other values-measures (the Valued Living Questionnaire; Wilson et al., 2010) was not evident, though this was anticipated by the authors of the VQ due to the low-magnitude correlations reported in the original VLQ validation paper. Incremental validity was also examined, with results suggesting the VQ adds predictive validity in a variety of well-being outcomes (mastery, personal growth, social relationships, depression, anxiety, stress, purpose, self-acceptance, and life satisfaction) over the VLQ and ELS. Test-retest reliability of the VQ did not meet the criteria for good measurement properties.
The Valued Living Questionnaire (VLQ; VanBuskirk et al., 2012; Wilson et al., 2010)
The VLQ examines an individual's values across 10 distinct life domains: family relations, intimate relations, parenting, social relations, employment, education/training, recreation, spirituality, community life, and physical well-being. The first 10 items ask clients to rate the importance of domains; the subsequent 10 items ask for ratings of consistency for living in line with values in the previous week.
Content Validity The VLQ was originally developed for use in clinical interventions. Professionals trained in ACT aimed to identify value-domains that would incite motivation within interventions; hence the domains featured within the VLQ were abstracted based on the values most commonly reported by clients, and in light of clinician experience. The VLQ provides both clarification of personal values and information regarding values-based action. Overall, it demonstrates acceptable content validity in relation to the ACT conceptualization of valued living.
Further Measures of Validity and Reliability Substudies in the Wilson et al. (2010) and VanBuskirk et al. (2012) validation papers demonstrated adequate to very good methodological rigor for properties measured. Exploratory Factor Analysis (EFA) supports the structural validity of the valued living composite score in the original validation study; however, in a cross-cultural sample, CFA suggests that the VLQ provides greater utility when interpreted as two separate subscales (Importance and Consistency, VanBuskirk et al., 2012). It is not clear whether this is due to differences in cross-cultural samples, or whether the VLQ composite can be reliably interpreted in a unidimensional manner.
Results were in line with hypotheses, such that valued living scores were positively correlated with outcomes on quality of life, social functioning and mental health, and negatively correlated with experiential avoidance, depression, anxiety, somatization, relationship difficulties, and general pathology (Wilson et al., 2010). Both Values Importance and Values Consistency scores were positively correlated with racial identity attitudes, and negatively correlated with depression, anxiety, and stress (VanBuskirk et al., 2012). Adequate discriminant validity was also demonstrated within the VanBuskirk et al. (2012) study, because no significant correlations were identified between values scores and scores on unrelated demographic variables (all ps > .05). Evidence suggests the VLQ demonstrates good reliability (Wilson et al., 2010). Convergent and incremental validity of the VLQ compared to other related constructs or instruments was not assessed by either validation study.
Valued Living Questionnaire for Alcohol Use (VLQ-A; Miller et al., 2016)
As in the VLQ, the VLQ-A asks participants to rank 21 valued life domains on both Importance and Behavioral Consistency, as well as assessing the extent to which alcohol use is consistent with each valued domain (Alcohol Consistency). It is made up of 63 items.
Content validity The majority of value-domains listed on the VLQ-A were in line with those of the VLQ; however authors included a number of additional domains (e.g., social image, self-identity, mental health), and participants were encouraged to include relevant domains. As with the VLQ, although this scale is consistent with the majority of the ACT conceptualization of values, it does not explore the motives and "freely chosen" nature of values.
Further measures of validity and reliability At least adequate methodological rigor was present for all psychometric properties assessed. Outcomes of EFAs support a unidimensional structure for the VLQ-A; however, as no CFA was conducted, this did not fulfill the criteria for good measurement properties. Construct validity was assessed using hypothesis testing, and results were consistent with expectations, such that scores on Valued Drinking were positively correlated with alcohol consumption, and negatively correlated with alcohol problems and readiness to change. The VLQ-A was found to have indeterminate test-retest reliability, as the Alcohol Consistency subscale did not meet criteria. The Valued Drinking composite scale demonstrated good internal consistency across studies; however, quality of evidence for internal consistency cannot exceed the quality of evidence for structural validity, because this is contingent on unidimensionality (Mokkink et al., 2018; Prinsen et al., 2018; Terwee et al., 2018). Therefore, evidence for internal consistency on subscales of the VLQ-A could not be examined. Convergent validity of the VLQ-A compared to similar instruments was not reported.
Valued Living Questionnaire Adapted for Dementia Caregiving (VLQAC; Romero-Moreno,
Gallego-Alberto, Marquez-Gonzalez, & Losada, 2017)
The VLQAC is an amended version of the VLQ, for caregivers of individuals with dementia. As in the VLQ, the VLQAC instructs participants to rank 10 life domains in terms of both Importance and Consistency. An additional two items were included to reflect the domains of caring for relatives and self-care in caregivers.
Content validity Two new items were included in the VLQAC due to the feedback of participants in a pilot study of an ACT intervention for dementia caregivers. Although the VLQAC demonstrates acceptable content validity in terms of the ACT conceptualization of values, again, it may inadvertently overlook the processes of pliance versus tracking and competing motives in caregiving behavior, though it allows for examinations of both qualitative importance of domains, and behavioral congruence with these.
Further measures of validity and reliability Adequate methodological quality was demonstrated in all aspects of the validation paper. EFA suggests a bidimensional structure, including Commitment to Own Values and Commitment to Family Values subscales; however, as no CFA was completed, criteria for good measurement properties were not met. Good construct validity was demonstrated, such that both subscales were positively correlated with life satisfaction and acceptance, and negatively correlated with behavioral problems, cognitive fusion, depression, and anxiety, though the hypothesis (of unidimensionality) was not supported. Both subscales also demonstrated incremental validity in predicting depression, anxiety, and life satisfaction when entered after demographic variables, as well as acceptance and cognitive fusion. Internal consistency ratings were indeterminate across subscales, and could not be reliably interpreted due to a lack of available evidence for structural validity. Test-retest reliability and convergent validity of the VLQAC with other values measures was not examined.
Chronic Pain Values Inventory (CPVI; Akerblom, Perrin, Rivano Fischer, & McCracken, 2017; McCracken & Yang, 2006)
The CPVI is a 12-item measure that examines the importance of value areas such as family, intimate relationships, friendships, work, health and personal growth, and learning, as well as the amount of success each individual has experienced in acting consistently with these.
Content validity The CPVI was developed based on existing ACT literature and interventions for chronic pain. As with the VLQ and its adaptations, the CPVI does not assess the underlying motivations for chosen values. However, given its theoretical foundation and consistency with the ACT conceptualization of values, the content appears to demonstrate adequate validity. In addition, a number of both experts and patients were consulted in the translation of the CPVI to Swedish, and necessary edits made accordingly (Akerblom et al., 2017).
Further measures of validity and reliability Both studies examining the validity of the CPVI demonstrated very good methodological rigor for the properties examined. No factor analyses were conducted to assess the construct validity of the CPVI in either validation study; however, construct validity was assessed using hypothesis-testing. Correlations between both values success and discrepancy subscales and measures of acceptance, avoidance, committed action, psychological flexibility, disability, depression, and anxiety were all in line with predictions. Both subscales demonstrated good internal consistency across studies; however, given that evidence of unidimensionality was not provided, internal consistency ratings could not be reliably assessed. Incremental validity was also explored, with results indicating that the CPVI added predictive validity in psychosocial disability, other disability, depression, and depression-related interference in functioning (McCracken & Yang, 2006), as well as adding incremental validity over other ACT process measures in predicting depression, anxiety, pain interference, mental health, vitality, and physical and social functioning (Akerblom et al., 2017). Test-retest reliability of the CPVI was not reported.
Valued Time and Difficulty Questionnaire (VTDQ; Drake et al., 2019)
The VTDQ is a 30-item measure developed from the VLQ, and contains the same 10 value domains, each rated on three separate scales: Importance of Engagement with Value, Amount of Time Engaged with Value, and Difficulty Engaging with Values.
Content validity The VTDQ was developed from the original VLQ, and as such the domains examined are derived from previous ACT research; however, the VTDQ focuses on the importance of engaging with each valued domain, the amount of time engaging in valued activities, and difficulties in engaging with values. Thus, this measure seeks to concurrently examine values and value-based action, while also exploring the influence of experiential avoidance on values-based action. Overall, the VTDQ appears to be consistent with the ACT construct of values. No information was provided around the comprehensibility of the items presented to participants, or any refinements made.
Further measures of validity and reliability This study demonstrated very good methodological quality. Construct validity was assessed using hypothesis testing, with results of correlations consistent with expectations (i.e., Difficulty scores for both U.S. and Japanese samples were positively associated with both experiential avoidance and psychological distress, whereas scores on the Importance and Time subscales were negatively correlated with both distress and avoidance). Differences in responses were, however, observed among the two cultural subgroups, in line with authors' expectations. In terms of internal consistency, Cronbach's a for both Importance and Difficulty subscales of the VTDQ were acceptable, though scores on the Time subscale did not demonstrate good reliability. However, as no evidence of unidimensionality was provided, the quality of evidence for internal consistency of scales could not be interpreted in this review. No assessments of test-retest reliability were conducted.
Engaged Living Scale (ELS-16, Trompetter et al., 2013; ELS-9, Trindade, Ferreira, Pinto-Gouveia, & Nooren, 2016)
The ELS has been validated in two studies: one focusing on the original 16-item version (Trompetter et al., 2013) and a second validating a 9-item version (Trindade et al., 2016). The ELS was designed as a brief, process-oriented tool to assess engaged living.
Content validity ACT literature and experts were consulted for information regarding values and engaged living. The facets extracted from this were: values, committed action, and evaluation, such that these encompassed aspects of the flexibility, motives, behavioral responses, and congruence with values. Items were piloted on a sample of university students, and clinicians/researchers working with ACT (Trompetter et al., 2013). The ELS appears to demonstrate good content validity.
Further measures of validity and reliability Based on the COSMIN assessment of bias checklist, both studies demonstrate adequate or very good methodological quality for the properties examined. In terms of structural validity, both exploratory and confirmatory factor analyses support a bifactor model, though scores can also be interpreted as a unitary index (Trompetter et al., 2013); however, criteria for good structural validity were not consistently met. This may indicate that the ELS-16 is suitable for clinical populations (as in Trompetter et al., 2013), but has not yet been demonstrated as valid for university students (Trindade et al., 2016).
Construct validity was examined using hypothesis testing within both studies, with correlations between ELS scale scores and psychological flexibility, cognitive fusion, mindfulness, values, mental health, personality, quality of life, and pain all in expected directions, bar the "openness to experience" facet of the NEO 5-factor personality inventory. Convergent validity was demonstrated between the ELS-16 and the ELS-9; however, convergence with the VLQ did not meet the criteria for good measurement properties (Trindade et al., 2016). Though, as the ELS does not purport to examine solely the construct of values, this was in line with authors' predictions. The ELS also demonstrated incremental validity over psychological flexibility and mindfulness measures in predicting outcomes on both pain and mental health. Good evidence for the temporal stability of scores on the ELS was provided.
Comprehensive Assessment of Acceptance and Commitment Therapy Processes (CompACT; Francis, Dawson, & Golijani-Moghaddam, 2016)
The CompACT is a 23-item measure that aims to examine the six core ACT processes (comprising PF), and contains Openness to Experience, Behavioral Awareness, and Valued Action subscales.
Content validity Existing ACT-relevant questionnaires were consulted during initial item development. Ten ACT experts subsequently rated the face and content validity of items relating to each ACT process, in line with Delphi methodology.
Item comprehensibility was assessed using standardized tests and 10 adults unfamiliar with ACT also completed a pilot and feedback session for the CompACT. It is suggested, therefore, that the measure demonstrates adequate face and content validity, though does not allow for values clarification.
Further measures of validity and reliability All properties assessed were found to demonstrate adequate or very good methodological quality. The 3-factor structure of the CompACT was supported by an EFA; however, the criteria for structural validity and as a result internal consistency were insufficient, according to the criteria for good measurement properties. Construct validity was supported by virtue of hypothesis testing, such that significant positive correlations were found between higher valued action scores and depression, anxiety, stress, and psychological inflexibility (rs = .23 .41 rs <.01), with significant negative correlations being seen between valued action subscale scores and positive physical health and mental health outcomes (rs ranging from -.17 to -.39, rs < .01). Discriminant validity was also demonstrated for all subscales, because no significant correlations were found with an unrelated construct: social desirability. Although the CompACT demonstrated incremental validity above the AAQ-II, the Valued Action subscale only accounted for a significant proportion of variance on outcomes relating to depression ([beta] = .14, p < .01), and not anxiety or stress ([beta] = -.01, -.08, respectively).
Multidimensional Psychological Flexibility Inventory (MPFI; Rolffs et al., 2018)
The MPFI is a comprehensive measure, assessing the six core ACT processes comprising PF, as well as their inverse processes (i.e., psychological inflexibility). It contains 12 subscales (Acceptance, Present Moment Awareness, Self-as-Context, Defusion, Values, and Committed Action, and their inverse processes), each containing five items, and two composite scales (Psychological Flexibility and Psychological Inflexibility; 30 items each).
Content validity Existing measures of ACT processes and consultation with the ACT literature were utilized in developing an initial item pool. Additional items were generated by authors who were familiar with ACT and designed with client comprehensibility being considered. Although such a process measure cannot assist in the identification of personal values, adequate content validity for values as a process was demonstrated.
Further measures of validity and reliability The amalgamation of second and third substudies demonstrated very good methodological quality across all properties assessed. Structural validity of the MPFI was demonstrated via CFA, such that each of the 12 subscales loaded onto two higher order factors, and met the criteria for good measurement properties. Good internal consistency was additionally demonstrated for both Values and Lack of Contact with Values subscales. Construct validity of the MPFI was supported by hypothesis testing, with all correlations between values-relevant subscales and other outcome variables being in the expected directions. The low to moderate magnitude correlations between MPFI values subscale scores and unrelated constructs additionally attested to their discriminant validity.
Pain Flexibility Scale for Parents (PFS-P; Cederberg, Weineland, Strandskov, Dahl, & Ljungman, 2017)
The PFS-P is a 24-item instrument designed to measure pain acceptance in parents of children with chronic pain. It measures both values and cognitive defusion, and contains three distinct subscales: Pain resistance, Valued Action, and Pain Fusion.
Content validity Psychologists familiar with ACT were consulted in the initial development of items. Items were piloted on parents both of physically healthy children and parents of children with cancer in order to ascertain comprehensiveness and comprehensibility. Although the Valued Action subscale does not provide clarity for parents' personal values, it is theoretically consistent with the ACT construct of values.
Further measures of validity and reliability Adequate methodological quality was found for properties examined. Results of EFA suggest a three-factor solution, with Pain Resistance, Valued Action, and Pain Fusion subscales, though this did not meet the criteria for good measurement properties, because CFA was not conducted. Good construct validity was also demonstrated, with all four hypotheses being supported in the results (of relevance, the Valued Action subscale was negatively correlated with both pain catastrophizing and psychological inflexibility). Furthermore, the test-retest reliability of the total scale and Valued Action subscale met the threshold for good measurement properties. Although good internal consistency was found for the values-relevant subscale, this could not be reliably assessed due to the criteria for structural validity not being met. No information on incremental, discriminant, or convergent validity with other similar measures was reported.
Values Tracker (VT; Pielech et al., 2015)
The VT is a self-rated 2-item measure of values engagement for individuals with chronic pain. It asks participants to respond to 1 item measuring Vital Actions, and 1 item measuring Values Progress.
Content validity The two items of the VT were taken from an existing daily diary (see Vowles, Fink, & Cohen, 2014) for monitoring pain intensity, pain control, and engagement in valued activities for adults completing ACT interventions for chronic pain. Items were selected to assess aspects of both engagement in valued activities, and actions taken in accordance with values. Although items included are derived from existing ACT literature, no information in relation to the comprehensiveness and patient-rated comprehensibility was provided.
Further measures of validity and reliability The study demonstrated very good methodological rigor for properties examined. In terms of structural validity, no factor analyses were conducted; however, construct validity was assessed using hypothesis testing. Results were in line with hypotheses, such that outcomes on the VT were negatively correlated with pain intensity, distress, pain-related anxiety, depression, and disability outcomes, and positively correlated with acceptance, values, psychological flexibility, self-compassion, and pain coping. Convergence was also found between the VT and the CPVI, suggesting both are measuring similar processes; however, the strength of correlation did not meet the criteria for good measurement properties. Incremental validity for values along with pain duration, intensity, and distress in predicting outcomes in physical and psychosocial disabilities, depression, anxiety, ACT constructs, self-compassion, and pain coping behaviors was also demonstrated. The article did not report on test-retest reliability or any aspects of discriminative validity.
Acceptance and Action Questionnaire: Substance Abuse (AAQ-SA; Luoma, Drake, Kohlenberg, & Hayes, 2011)
The AAQ-SA is an 18-item measure designed to assess psychological flexibility, values, cognitive fusion, and acceptance in individuals with a history of substance abuse issues. It contains two subscales: Values Commitment and Defused Acceptance.
Content validity An initial pool of items was based on items in the development of the original AAQ; this was based on contributions from 12 ACT clinicians and researchers. The content of the AAQ-SA was also validated by consulting ACT experts to provide ratings regarding the consistency of items with ACT theory, as well as the comprehensibility and generalizability of items. The AAQ-SA appears to demonstrate good content validity; however, the comprehensibility of items for clients was not assessed in its development.
Further measures of validity and reliability Based on the COSMIN checklist, the study demonstrates adequate methodological quality for all properties measured. In terms of structural validity, exploratory factor analyses support a bifactor model; however, this did not meet the threshold for good measurement properties by virtue of it being subjected to EFA rather than CFA, and hence internal consistency ratings could not be reliably interpreted. Construct validity was assessed using hypothesis testing, with correlations between subscale and total scale scores and substance use, depression, internalized shame, internalized stigma, social support, self-esteem, self-concealment, and active coping with stigma all in expected directions, though only three were significantly correlated to the Values Commitment subscale; however, theoretically values-committed behavior would not necessarily be expected to be correlated with such constructs. Incremental, discriminant, and test-retest reliability were not examined.
Acceptance and Action Questionnaire for Obsessions and Compulsions (AAQ-OC; Jacoby et al., 2018)
The AAQ-OC is a 13-item adaptation of the AAQ-II for measuring psychological inflexibility in clients with obsessive-compulsive (OC) symptoms. It is made up of two subscales: Valued Action and Willingness, with higher scores indicating greater psychological inflexibility, lower ability to act in line with values, and lower levels of willingness.
Content validity An initial item pool was generated based on the pool used in the development of the AAQ-II, and adapted for intrusive thoughts. A description of intrusive thoughts was also included for comprehensibility. Four experts were consulted in order to ensure the comprehensiveness of the items related to ACT processes. Although no values-clarification is featured, and no client comprehensibility was reported, the AAQ-OC demonstrates acceptable content validity.
Further measures of reliability and validity The methodological standards of all psychometrics reported were of adequate or very good quality. CFA confirmed a 2-factor structure of the AAQ-OC, and met the criteria for good measurement properties. Good internal consistency was also reported for the Valued Action subscale. Construct validity was attested to by virtue of higher scores on the AAQ-OC (composite and Valued Action subscale) demonstrating positive correlations with outcomes of psychological inflexibility, depression, anxiety, stress, obsessive beliefs, and thought suppression. Convergence with others' values-measures was not examined. Adequate discriminant validity was demonstrated as correlations with related measures were significantly larger in magnitude (rs = .55 - .62) than correlations with unrelated measures (rs =.30 - .43). Incremental validity for the AAQ-OC over the AAQ-II in predicting OC symptoms was found; however, no reference was made to the impact of the values subscales within such analyses.
Values, Acceptance, Mindfulness Scale (VAMS; Lundgren et al., 2018)
The VAMS is an 11-item measure developed to assess psychological flexibility processes in athletes in sports settings, in particular ice hockey players. It is made up of three subscales: Values, Acceptance, and Mindfulness, as well as a PF composite.
Content validity Authors considered mindfulness, acceptance, and values processes to be of particular relevance to sports populations. The authors generated a pool of items, based on ACT theory and sports literature. Questions were piloted with a sample of former players to obtain qualitative feedback. No further information regarding the comprehensiveness or comprehensibility of items was included. The measure does not allow for values clarification processes. Nonetheless, the VAMS appears to demonstrate acceptable content validity.
Further measures of validity and reliability Although PCA was conducted to attest to the structural validity of the VAMS, the methodological quality was found to be doubtful given the sample size included; therefore, it did not meet the criteria for good measurement properties for either structural validity or internal consistency (though Cronbach's a for the Values subscale was also insufficient). Construct validity was demonstrated through hypothesis-testing, with correlations in expected directions for outcomes on mindfulness, acceptance, depression, anxiety, stress, and life satisfaction.
Valued Living Scale (Jensen, Vowles, Johnson, & Gertz, 2015)
This 24-item scale assesses perceived Importance, Confidence, and Success in achieving value-related goals across eight separate value domains in chronic pain populations.
Content validity Values were conceptualized as the perceived importance of specific goals. Social cognitive theory was referenced, positing that both perceptions about outcomes (expectations) and importance of outcomes (values) together influence behavior. Hence discrepancies between scores are assumed to be reflective of the extent to which a client feels they are living life in line with values-consistent goals. Items were based on clinical experiences of authors, as well as drawn from the CPVI and VLQ, but were not assessed for comprehensibility among clinical populations. Because this questionnaire does not appear to distinguish values from goals, and does not assess the motives for values or values-related goals, the content does not appear to be consistent with the ACT conceptualization of values.
Further measures of validity and reliability Based on the COSMIN assessment of bias checklist, the study was found to demonstrate adequate methodological quality for properties examined. In terms of structural validity, exploratory and confirmatory factor analyses were conducted on Success and Confidence subscales, though not the Importance subscale, due to kurtosis. Outcomes suggested a bifactor structure (social and relational, and health and productivity), on both Success and Confidence subscales. Construct validity was also assessed, with all outcomes being in line with the two stated hypotheses (i.e., Success and Confidence subscales of the VLS were negatively correlated with pain intensity, pain interference, and depression, whereas Difference scores were positively correlated with the same). Internal consistency was evident for the Success and Confidence subscales of the health and productivity factor, and the Confidence dimension of the social and relational factor. Convergent validity with existing values-measures for chronic pain was not assessed. No analysis of the test-retest reliability of the measure was reported.
Although previous articles (Reilly et al., 2019; Serowik et ah, 2018) conducted systematic reviews of values measures, this is the first known review of ACT-consistent values instruments containing values subscales. This review therefore provides a comprehensive and up-to-date review of the area of values-measurement. A total of 24 psychometric instruments were identified; however, only 17 of these had been the subject of published validation studies. Tools differed considerably in their characteristics, such as length, administration, content, structure, and focus. Overall, the evidence suggests that the VQ, ELS, VLQ, and MPFI demonstrate the strongest psychometric properties among the instruments evaluated. This is due to the fact that each of these measures provide evidence of both acceptable content validity and internal consistency. However, adequate evidence for convergent validity was not found among the listed values-measures. It should be noted that the BEVS and VW are not based on reflective models, therefore internal consistency could not be assessed, though the BEVS provided evidence of its structural, construct, temporal stability, and responsiveness, and hence can also be reliably used. Although the AAQ-OC was found to demonstrate excellent psychometric properties, this tool had been adapted for populations experiencing intrusive thoughts specifically, and may therefore not provide utility for all populations. Although the tools listed above are the measures recommended for use at present, it should also be noted that no psychometric instrument met all the criteria for good measurement properties.
Of all included scales, the VLS is the sole measure that does not appear to possess the requisite content validity to be consistent with ACT (due to a lack of distinction between values and goals), and is therefore not currently recommended for use in values-measurement. Outcomes of studies will be discussed both qualitatively, as well as in light of the quantitative evidence for their utility, as demonstrated by empirical research.
Summary of Findings
Content validity As recommended in the COSMIN checklist, content validity is the foremost aspect that should be considered when evaluating the utility and validity of PROMS (Mokkink et al., 2018; Prinsen et al., 2018; Terwee et al., 2018). Though an in-depth analysis was beyond the scope of this review, the overall comprehensiveness of ACT-consistent content, and the comprehensibility of items within each questionnaire will be discussed.
The CPVI, VT, and VLQ, as well as adaptations such as the VLQ-A and VLQAC all reference the relative importance of valued domains and behavioral consistency with chosen values; however, the motivations and reasons for selecting values and the distinction between values and goals are not addressed. One subscale of the VTDQ addresses the perceived difficulty in engaging with values (as well as importance and behavioral success), and may therefore include another salient aspect of the construct. The BEVS includes barriers to engaging with values, and both the BEVS and VW focus on idiographic values clarification and behavioral consistency with values. In contrast, the VQ, PFS-P, ELS, CompACT, MPFI, AAQ-SA, AAQ-OC, and VAMS do not allow for clarification or prioritization of values, though do note aspects of clarity, coherence, consistency, and commitment. The VQ and ELS also encompass elements addressing behavior, obstruction, and the impact of values on well-being.
Consultation with clinicians and ACT experts were noted in the development of nine measures (BEVS, VQ, VLQ, ELS, CompACT, MPFI, PFS-P, AAQ-SA, and AAQ-OC). However, additional measures including the VLQ-A, VLQAC, and VTDQ were derived from the VLQ and can therefore be viewed as comprehensive. Although the CPVI, VW, VT, VAMS, and VLS were developed from ACT theory or adapted from preexisting measures, no reference to the consultation of experts was made. In terms of the client comprehensibility and comprehensiveness of items, the BEVS, VQ, VLQ, VLQ-A, VLQAC, ELS, CompACT, PFS-P, and VAMS validation studies conducted either a pilot study or generated/ amended existing items with the assistance of target groups, which suggests adequate levels of patient comprehensibility and content validity, though the number of participants included was not in accordance with COSMIN guidelines.
Overall, all measures excepting the VLS adhere to the ACT conceptualization of values. However the VLS, CPVI, and VT articles all failed to reference consultation with a number of clinicians, researchers, or potential participants regarding the content of the measures.
Structural, construct, convergent, and discriminant validity
Though the structural validity of measures was reported in the majority of articles reviewed, CFA was only performed with the VQ, VLQ, ELS, MPFI, AAQ-OC, and the VLS. High-quality evidence was found for the structural validity of the VQ, MPFI, and AAQ-OC, whereas moderate quality evidence was demonstrated for both the VLQ and the ELS, in spite of inconsistencies across studies (Trindade et al., 2016; Trompetter et al., 2013). Evidence suggests the factor structure of the ELS is valid for use with chronic pain populations, but has yet to be demonstrated with alternative cohorts. The quality of evidence for structural validity of the VLS was deemed to be high, though it cannot be recommended due to content validity. EFAs were also conducted with the BEVS, VLQ-A, VLQ-AC, CompACT, PFS-P, AAQ-SA, and VAMS; however, CFA were not conducted to further validate the measures. The CPVI, VTDQ, and VT did not report on structural validity.
Outcomes suggest that the quality of evidence for construct validity (hypothesis-testing) was high for all measures, except the VW. The VLQ, CompACT, MPFI, and AAQ-OC also demonstrated high-quality evidence for their discriminant validity, and the BEVS reported evidence for sensitivity and responsiveness. When comparing outcomes with other values measures, the VQ, VW, ELS, and VT were assessed, but did not meet the criteria for convergent validity.
Internal consistency and test-retest reliability In terms of internal consistency, all measures reported outcomes bar the BEVS, VW, and VT; however, only studies with adequate structural validity could be evaluated. Therefore, high-quality evidence was found for the internal consistency of the VQ, ELS, MPFI, AAQ-OC, and VLS, with the VLQ also demonstrating moderate evidence, due to inconsistencies in findings. In addition, test-retest reliability was assessed for the BEVS, VW, VQ, VLQ, VLQ-A, ELS, and PFS-P, with moderate quality evidence found for the temporal stability of the BEVS, ELS, and PFS-P.
Evaluation of ACT Values Measures
The evidence for values measures within ACT, although growing, requires significantly more research; hence, future studies should continue to explore the properties of existing instruments. It should be noted that convergence with other measures of values was rarely examined. Furthermore, outcomes of the studies that did examine this (Jacoby et al., 2018; O'Connor, Tennyson, Timmons, & McHugh, 2019; Pielech et al., 2016; Smout et al., 2014; Trompetter et al., 2013) did not demonstrate adequate levels of convergence with respective comparator measures. Although this was anticipated within the validation of the ELS (because it purports to measure both values and committed action), no justification was provided within the other articles. This may suggest that values assessment tools are not in fact measuring the same construct, due to current differences in both content and focus (Mokkink et al., 2018; Prinsen et al., 2018; Terwee et al., 2018).
Measures of values may focus on clarity, coherence, behaviors, motivations, persistence, and perceived barriers or obstacles to living in line with values, therefore the precise components and targets of each instrument appear somewhat disparate. It is posited that values selected according to processes of pliance (e.g., "If I am generous, people will like me") or tracking (seeking immediate reinforcement) are likely to be inflexible and ultimately ineffective, because they are under aversive control (Levin, Hayes, & Vilardaga, 2012). This is perhaps of particular relevance for the VLQ AC, which focuses on both family and personal values within subscales, and could potentially result in participants endorsing values relating to caring for family due to fears of negative consequences (i.e., pliance; Levin et al, 2012). Hence, it is important for values-measurement instruments to attend to the reasons for which values are chosen, as augmentals are the only ACT-consistent form of values ; however, none of the included measures monitored this within the instruments, which suggests that no measure is comprehensive in its assessment of values.
Furthermore, although ratings for test-retest reliability were identified for several measures (the BEVS, ELS, and PFS-P), the reasons for assessing this aspect of values psychometrics is somewhat complex. Given that values are conceptualized as ongoing, dynamic patterns of activity, with one goal of ACT interventions being to increase engagement and behavioral consistency with values, fluctuations in behavioral-consistency ratings are expected (Wilson et al., 2010), in particular prior to intervention. Therefore, it may be more useful to compare whether test-retest scores achieved postintervention are more stable than those prior to intervention, as a consequence of an increased capacity for values-based living. This is in line with expectations posited within the ACT literature (Plumb, Stewart, Dahl, & Lundgren, 2009; Wilson et al., 2010).
Research versus Clinical Utility
A prime consideration in selecting an appropriate values-measure is the purpose for which it is being used. Because certain questionnaires seek to focus on values-clarification (in line with ACT-based interventions), such instruments may not be suited to quick, or process-oriented data collection for research. Instruments such as the BEVS, VW, VLQ, VLQ-A, VLQAC, CPVI, VTDQ, VT, and VLS may provide good clinical insights as to the relative importance of certain valued domains, in particular when discussions can be conducted within sessions around reasons for valuing. This would also ensure that perceived barriers such as experiential avoidance and practical solutions to this could be discussed (Wilson & Murrell, 2004).
Conversely, measures such as the VQ and ELS may provide more utility in evaluations of values and committed action processes within research, because such measures have a shorter administration time, and do not require values clarification (Trompetter et al., 2013). These measures also provide a platform for comparing scores between individuals, which is potentially clearer than scores provided by discrepancy analyses (e.g., in the CPVI, VLQ). Given the need to assess multiple components of the theoretical model within intervention studies, combined process measures such as the CompACT and MPFI may also be more suitable for a time-limited, broader process-based assessment (Francis et al., 2016; Rolfifsetal., 2018).
Limitations of Values Measures
Of the 33 substudies included, 21 utilized general adult or university samples, and 4 sought to assess forms of cross-cultural validity (Akerblom et al., 2017; Carvalho et al., 2018; Drake et al., 2019; VanBuskirk et al, 2012). Results of cross-cultural validation studies suggest that outcomes either varied across cultural samples (Drake et al., 2019), or the structure of the measure required adjustments (VanBuskirk et al., 2012). Akerblom et al. (2017) found the properties of the CPVI to be equivalent across Swedish and U.S. samples, though the structural validity of the measure was not assessed in either study. Given the variation in the conceptualizations and prioritization of values within different populations and cultures (Schwartz, 1992, 1999), it may be necessary to explore the content validity of prescribed values domains versus the applicability of idiographic nature of values (Drake et al., 2019). Therefore, additional research is required in order to ascertain the cross-cultural validity of measures.
Furthermore, the surplus of validation studies with college-aged students may not allow for generalizations to be made about the reliability and validity of measures across age groups, or for clinical populations. Only 11 substudies included clinical populations for measure validation (BEVS, VQ, AAQ-SA, CPVI, ELS, VT, VLS), as well as caregivers for children with cancer (PFS-P) or relatives with dementia (VLQAC); however, the majority of individuals attending services for ACT interventions will be experiencing some form of mental or physical health issues. Therefore, further validation studies are required with clinical samples in order to assure the appropriateness and psychometric reliability of measures used for such populations versus their validity for the general public (Prinsen et al., 2016).
Another limitation of the values measurement instruments identified are the rating scales employed. Because some questionnaires use Likert scales with even-ratings (e.g., VLQ items 1-10), it is not possible for participants to provide a "neutral" response to items, which may affect the quality of the data due to forced responses (Johns, 2010), though, as other psychometricians advocate for the utility of forced responses, this may not be regarded as problematic for all researchers (Cohen, Manion, & Morrison, 2002). Measures also vary significantly in scoring and manner of administration, such that some contain self-generated, idiographic responses for values (e.g., BEVS, VW, PVQ), whereas others offer predefined valued domains (e.g., VLQ, VLQ-A, CPVI), with participants being asked to rank each area. This may result in difficulties in interpreting and comparing scores, as well as assessing for convergent validity among values measures. However, this lack of convergent validity between existing measures may also be due to discrepancies in the content of instruments (Prinsen et al., 2016).
Composite scores achieved on certain measures (e.g., VLQ, VLQ-A, VLQAC, CPVI) may also not provide meaningful information about values and values-based action, because they do not allow for differentiations between individuals who endorsed all items to a moderate level versus those who provided varied or extreme responses on some items (Serowik et al., 2018; Trompetter et al., 2013; Wilson et al, 2010). Furthermore, many measures featured do not provide a definition or concept of values within their administration, which could result in participants misinterpreting the meaning of values or responding in a manner that is not consistent with the content, which is likely to affect responses; therefore, it may be useful to provide a brief conceptualization of values prior to research-based administrations (Demetriou, Ozar, & Eassau, 2015).
Although many instruments were subjected to CFAs (i.e., the VQ, VLQ, ELS, MPFI, AAQ-OC, and VLS), EFA was commonly employed in order to examine the underlying structure of measures, which resulted in structural validity being reported as indeterminate. Some evidence suggests that CFA is required for a robust assessment of the structural properties of psychometric tools (Henson & Roberts, 2006) and is required within the COSMIN guidelines; therefore, future research should seek to expand on existing evidence through the use of CFA.
Lastly, all included measures of values and behavioral consistency with values rely on self-report, which may result in more socially desirable responses due to fears of negative evaluation (Van de Mortel, 2008). Although this can be addressed within clinical interventions, it may give rise to inflated or inaccurate scores within research and hence should be considered in the development and administration of values-measures (Carey, Maisto, Carey, & Pumine, 2001; e.g., in the VLQ-A validation study, many reported that drinking behavior was congruent with values, which may be hue for values such as "having fun" or "engaging with friends," whereas drinking behavior may also directly conflict with other personal values).
Furthermore, individuals may be less inclined to endorse the importance of values they are not living consistently with, as this could result in frustration and discomfort due to cognitive dissonance (Festinger, 1962; Jones, 1990). For example, a person with agoraphobia may endorse family-based values rather than values that require more interaction with the external world (such as career or community values) and hence may not be reflective of levels of value-based living, because such values are under aversive control (Levin et al., 2012). Again, as such processes can be discussed within ACT interventions, it may prove a more pressing issue for research-based studies.
Limitations of the Current Review
One limitation of the current review was that the COSMIN checklist was not used to comprehensively assess the content or cross-cultural validity of the measures featured, because this was not within the scope of the review. It should be noted that had the COSMIN criteria been applied for content validity, studies would likely have been deemed of inadequate quality due to the insufficient numbers of participants and experts consulted during item development. Hence, future research around ACT instrument development should include more researchers, clinicians, and service users within their creation to ensure adequate content validity (Mokkink et al., 2018; Prinsen et al., 2018; Terwee et al., 2018). Qualitative research regarding perceptions of values may also be useful in order to determine whether participants' own perceptions of values (and the extent to which these affect behavior and well-being) are consistent with the ACT conceptualization of values.
Another limitation of the current study is the fact that only published journal articles were included in the qualitative synthesis. Therefore, it was not possible to evaluate a number of instruments (i.e., the PVQ, SVS, GLP, AVQ, NVM, WVQ, and DVAQ) that could demonstrate utility in values-measurement. In addition, any validation study identified that was not available in English was excluded from the review. The exclusion of ACT intervention articles is another potential limitation of the current research, because such articles are useful in determining the responsiveness and discriminant validity of ACT values-instruments; however, the review by Reilly et al. (2019) provides more insight on process-specific mechanisms of change within ACT (Bond & Bunce, 2003; Kemani, Hesser, Olsson, Lekander, & Wicksell, 2016; Varra, Hayes, Roget, & Fisher, 2008; Zettle, Rains, & Hayes, 2011).
In line with the above limitations, one avenue for future research may be to gather cross-cultural data in order to ensure greater generalizability of findings in relation to measurement properties, as well as the content validity of measures for different clinical populations, cultures, and nationalities. Furthermore, conducting validation studies of existing validated or unvalidated measures of values would provide further assessment of their potential utility. In addition, research on clients' perceptions of values and the impact of values on well-being would be useful in refining the content of values-measures. Finally, research should continue to explore the processes through which change is elicited within interventions, in particular using the data obtained from pre-post measures of values in ACT interventions.
Compliance with Ethical Standards
Conflict of Interest On behalf of all authors, the corresponding author states that there is no conflict of interest
Research Involving Human Participants and/or Animals This article does not contain any studies with human participants or animals performed by any of the authors.
Informed Consent As this article is a review of previous psychometric validation studies, data was not collected from participants by the authors; therefore informed consent was not required.
References marked with an "t" indicate psychometric validation studies included
[dagger] Akerblom, S., Perrin, S., Fischer, M. R., & McCracken, L. M. (2017). Further validation of the Chronic Pain Values Inventory in a Swedish chronic pain sample. Journal of Contextual Behavioral Science, (5(3), 261-267. https://doi.org/10.1016/jjcbs.2017.06.001
Antonovsky, A. (1979). Health, stress, and coping. San Francisco, CA: Jossey-Bass.
Blackledge, J. T., Ciarrochi, J., & Bailey, A. (2006). Personal values questionnaire. Unpublished manuscript. Australia: University of Wollongong.
Blackledge, J. T., Spencer, R., & Ciarrochi, J. (2007). Initial validation of the personal values questionnaire. Paper presented at the 33rd Annual Convention of the Association for Behavior Analysis International, San Diego, CA.
Bond, F. W., & Bunce, D. (2003). The role of acceptance and job control in mental health, job satisfaction and work performance. Journal of Applied Psychology, 88, 1057-1067. https://doi.org/10.1037/00219010.88.6.1057.
Bond, F. W., Hayes, S. C., Baer, R. A., Carpenter, K. M., Guenole, N., Orcutt, H. K., et al. (2011). Preliminary psychometric properties of the Acceptance and Action Questionnaire-II: A revised measure of psychological inflexibility and experiential avoidance. Behavior Therapy, 42, 676-688. https://doi.Org/10.1016/j.beth.2011.03.007.
Carey, K. B., Maisto, S. A., Carey, M. P., & Pumine, D. M. (2001). Measuring readiness-to-change substance misuse among psychiatric outpatients: I. Reliability and validity of self-report measures. Journal of Studies on Alcohol, 62(1), 79-88. https://doi.org/10.
[dagger] Carvalho, S. A., Palmeira, L., Pinto-Gouveia, J., Gillanders, D., & Castilho, P. (2018). The utility of the valuing questionnaire in chronic pain. Journal of Contextual Behavioral Science, 9,21-29. https:// doi.org/10.1016/j.jcbs.2018.06.002
[dagger] Cederberg, J. T., Strandskov, S. W., Dahl, J., & Ljungman, G. (2017). Parents' relationship to pain during children's cancer treatment: A preliminary validation of the Pain Flexibility Scale for parents. Journal of Pain Research, 10, 507-514. https://doi.org/10.2147/ JPR.S127019
Ciarrochi, J., & Bailey, A. (2008). A CBT-practitioner's guide to ACT: How to bridge the gap between Cognitive Behavioral Therapy and Acceptance and Commitment Therapy. Oakland, CA: New Harbinger.
Cohen, L., Manion, L., & Morrison, K. (2002). Research methods in education. New York, NY: Routledge.
Demetriou, C., Ozer, B. U., & Essau, C. A. (2015). Self-report questionnaires. In The encyclopedia of clinical psychology (pp. 1-6). Wiley Online Library, https://doi.oig/10.1002/9781118625392.wbecp507.
Doi, S., Yokomitsu, K., & Sakano, Y. (2014). Personal Values Questionnaire-II: Internal consistency and validity. Japanese Journal of Behavior Therapy, 40, 45-55 [in Japanese with English abstract].
[dagger] Drake, C. E., Masuda, A., Dalsky, D., Stevens, K. T., Kramer, S., Primeaux, S. J., ... & Mitamura, T. (2019). Examining US and Japanese college students' differences in psychological distress: The mediating roles of valued action and experiential avoidance. International Journal for the Advancement of Counselling, 41, 25-40. https://doi.org/10.1007/s 10447-018-9342-2
Eisenbeck, N., Scheitz, K., & Szekeres, B. (2016). A brief Acceptance and Commitment Therapy-based intervention among violenceprone male inmates delivered by novice therapists. Psychology, Society & Education, [section](3), 187-199. https://doi.org/10.25115/ psye.v8i3.173.
Emmons, R. A. (2003). Personal goals, life meaning, and virtue: Wellsprings of a positive life. In Flourishing: Positive psychology and the life well-lived (pp. 105-128). Washington, DC; American Psychological Association.
Emmons, R. A., & McAdams, D. (1991). Personal strivings and motive dispositions: Exploring the links. Personality & Social Psychology Bulletin, 17(6), 648-654. https://doi.org/10.1177/ 0146167291176007.
Fernandes, C., Castilho, P., & Pinto-Gouveia, J. (2012). Validagdo da versao Portuguesa do Questionario de Valores [Portuguese Validation of the Valued Living Questionnaire]. Unpublished manuscript.
Festinger, L. (1962). A theory of cognitive dissonance (Vol. 2). Stanford, CA: Stanford University Press.
Forman, E. M., Herbert, J. D., Moitra, E., Yeomans, P. D., & Geller, P. A. (2007). A randomized controlled effectiveness trial of acceptance and commitment therapy and cognitive therapy for anxiety and depression. Behavior Modification, 31(6), 772-799. https://doi.org/10. 1177/0145445507302202.
[dagger] Francis, A. W., Dawson, D. L., & Golijani-Moghaddam, N. (2016). The development and validation of the Comprehensive assessment of Acceptance and Commitment Therapy processes (CompACT). Journal of Contextual Behavioral Science, 5(3), 134-145. https:// doi.org/10.1016/j.jcbs.2016.05.003
Gaudiano, B. A., & Herbert, J. D. (2006). Acute treatment of inpatients with psychotic symptoms using Acceptance and Commitment Therapy: Pilot results. Behavior Research & Therapy, 44(3), 415-437. https://doi.Org/10.1016/j.brat.2005.02.007.
Gifford, E. V., Kohlenberg, B. S., Hayes, S. C., Antonuccio, D. O., Piasecki, M. M., Rasmussen-Hall, M. L., & Palm, K. M. (2004). Acceptance-based treatment for smoking cessation. Behavior Therapy, 35(4), 689-705. https://doi.org/10.1016/S0005-7894(04) 80015-7.
Glick, D M & Orsillo, S. M. (2009). Academic Values Questionnaire (AVQ). [5-item Likert-type scale assessing the extent to which students value academics]. Unpublished instrument.
Gregoire, S., Lachance, L., Bouffard, T., & Dionne, F. (2018). The use of acceptance and commitment therapy to promote mental health and school engagement in university students: A multisite randomized controlled trial. Behavior Therapy, 49(3), 360-372. https://doi.org/ 10.1016/j.beth.2017.10.003.
Hayes, S. C., Bames-Holmes, D., & Roche, B. (2001). Relational frame theory: A post-Skinnerian approach to language and cognition. New York, NY: Plenum Press.
Hayes, S. C., Strosahl, K. D., & Wilson, K. G. (2011). Acceptance and commitment therapy: The process and practice of mindful change. New York, NY: Guilford Press.
Henson, R. K., & Roberts, J. K. (2006). Use of exploratory factor analysis in published research: Common errors and some comment on improved practice. Educational & Psychological Measurement, 66, 393-416. https://d0i.0rg/l 0.1177/0013164405282485.
Hickey, A. M., Bury, G., O'Boyle, C. A., Bradley, F., O'Kelly, F. D., & Shannon, W. (1996). A new short form individual quality of life measure (SEIQoL-DW): Application in a cohort of individuals with HIV/AIDS. BMJ, 373(7048), 29-33. https://doi.oig/10.1136/bmj. 313.7048.29.
[dagger] Jacoby, R. J., Abramowitz, J. S., Buchholz, J., Reuman, L., & Blakey, S. M. (2018). Experiential avoidance in the context of obsessions: Development and validation of the Acceptance and Action Questionnaire for Obsessions and Compulsions. Journal of Obsessive-Compulsive & Related Disorders, 19, 34-43. https:// doi.org/10.1016/j.jocrd.2018.07.003
[dagger] Jensen, M. P., Vowles, K. E., Johnson, L. E., & Gertz, K. J. (2015). Living well with pain: Development and preliminary evaluation of the Valued Living Scale. Pain Medicine, 76(11), 2109-2120. https:// doi.org/10.1111/pme. 12802
Johns, R. (2010). Likert items and scales. Survey Question Bank: Methods Fact Sheet, 1, 1-11.
Jones, E. E. (1990). Interpersonal perception. WH Freeman/Times Books/Henry Holt & Co.
Kemani, M. K., Hesser, H., Olsson, G. L., Lekander, M., & Wicksell, R. K. (2016). Processes of change in Acceptance and Commitment Therapy and Applied Relaxation for long-standing pain. European Journal of Pain, 20(4), 521-531. https://doi.org/10.1002/ejp.754.
Levin, M. E., Hayes, S. C., & Vilardaga, R. (2012). Acceptance and commitment therapy: applying an iterative translational research strategy in behavior analysis. In G. J. Madden (Ed.), APA handbook of behavior analysis (Vol. 2, pp. 455-479). Washington DC: American Psychological Association.
Levitt, J. T., Brown, T. A., Orsillo, S. M., & Barlow, D. H. (2004). The effects of acceptance versus suppression of emotion on subjective and psychophysiological response to carbon dioxide challenge in patients with panic disorder. Behavior Therapy, 35(4), 747-766. https://doi.org/10.1016/S0005-7894(04)80018-2.
[dagger] Lundgren, T., Luoma, J. B., Dahl, J., Strosahl, K., & Melin, L. (2012). The bull's-eye values survey: A psychometric evaluation. Cognitive & Behavioral Practice, 9(4), 518-526. https://doi.org/10.1016/j. cbpra.2012.01.004
[dagger] Lundgren, T., Reinebo, G., Lof, P. O., Naslund, M., Svartvadet, R, & Parting, T. (2018). The Values, Acceptance, and Mindfulness Scale for ice hockey: A psychometric evaluation. Frontiers in Psychology, 9, 1794. https://doi.org/10.3389/fpsyg.2018.01794
[dagger] Luoma, J., Drake, C. E., Kohlenberg, B. S., & Hayes, S. C. (2011). Substance abuse and psychological flexibility: The development of a new measure. Addiction Research & Theory, 19(1), 3-13. https:// doi.org/10.3109/16066359.2010.524956
[dagger] McCracken, L. M., & Yang, S. Y. (2006). The role of values in a contextual cognitive-behavioral approach to chronic pain. Pain, 123(12), 137-145. https://doi.Org/10.1016/j.pain.2006.02.021
[dagger] Miller, M. B., Meier, E., Lombardi, N., Leavens, E. L., Grant, D. M., & Leffingwell, T. R. (2016). The Valued Living Questionnaire for Alcohol Use: Measuring value-behavior discrepancy in college student drinking. Psychological Assessment, 28(9), 1051-1060. https:// doi.org/10.1037/pas0000235
Mokkink, L. B., De Vet, H. C., Prinsen, C. A., Patrick, D. L., Alonso, J., Bouter, L. M., & Terwee, C. B. (2018). COSMIN risk of Bias checklist for systematic reviews of patient-reported outcome measures. Quality of Life Research, 27(5), 1171-1179. https://doi.org/ 10.1007/s11136-017-1765-4.
[dagger] O'Connor, M., Tennyson, A., Timmons, M. & McHugh, L. (2019). The development and preliminary psychometric properties of the Values Wheel. Journal of Contextual Behavioral Science, 12, 39-46. https ://doi.org/10.1016/j.jcbs.2019.01.005
Ouzzani, M., Hammady, H., Fedorowicz, Z., & Elmagarmid, A. (2016). Rayyan: A web and mobile app for systematic reviews. Systematic Reviews, 5, 210. https://doi.org/10.1186/sl3643-016-0384-4.
[dagger] Pielech, M., Bailey, R. W., McEntee, M. L., Ashworth, J., Levell, J., Sowden, G., & Vowles, K. E. (2016). Preliminary evaluation of the values tracker: A two-item measure of engagement in valued activities in those with chronic pain. Behavior Modification, 40(1-2), 239-256. https://doi.Org/l 0.1177/0145445515616911
Pinto, R. A., Kienhuis, M., Slevison, M., Chester, A., Sloss, A., & Yap, K. (2017). The effectiveness of an outpatient Acceptance and Commitment Therapy Group programme for a transdiagnostic population. Clinical Psychologist, 21(1), 33-43. https://doi.org/10. 1111/cp. 12057.
Plumb, J. C., Stewart, I., Dahl, J., & Lundgren, T. (2009). In search of meaning: Values in modern clinical behavior analysis. The Behavior Analyst, 32, 85-104. https://doi.org/10.1007/BF03392177.
Prinsen, C. A. C., Mokkink, L. B., Bouter, L. M., Alonso, J., Patrick, D. L., de Vet H. C. W., & Terwee, C. B. (2018). COSMIN guideline for systematic reviews of patient-reported outcome measures. Quality of Life Research, 27(5), 1147-1157. https://doi.org/10.1007/s11136018-1798-3.
Prinsen, C. A., Vohra, S., Rose, M. R., Boers, M., Tugwell, P., Clarke, M., et al. (2016). How to select outcome measurement instruments for outcomes included in a "Core Outcome Set": A practical guideline. Trials, 17,449. https://doi.org/10.1186/sl3063-016-1555-2.
Reilly, E. D., Ritzerh T. R., Scoglio, A. A. J., Mote, J., Fukuda, S. D., Ahem, M. E., & Kelly, M. M. (2019). A systematic review of values measures in acceptance and commitment therapy research. Journal of Contextual Behavioral Science, 12, 290-304. https://doi.org/10. 1016/j.jcbs.2018.10.004.
Rogers, C. R. (1964). Toward a modern approach to values: The valuing process in the mature person. Journal of Abnormal & Social Psychology, 68(2), 160-167. https://doi.org/10.1037/h0046419.
Rokeach, M. (1973). The nature of human values. New York, NY: Free Press.
[dagger] Rolffs, J. L., Rogge, R. D., & Wilson, K. G. (2018). Disentangling components of flexibility via the hexaflex model: Development and validation of the Multidimensional Psychological Flexibility Inventory (MPFI). Assessment, 25(4), 458-482. https://doi.org/10. 1177/1073191116645905
[dagger] Romero-Moreno, R., Gallego-Alberto, L., Marquez-Gonzalez, M., & Losada, A. (2017). Psychometric properties of the valued living questionnaire adapted to dementia caregiving. Aging & Mental Health, 21(9), 983-990. https://doi.org/10.1080/13607863.2016. 1191055
Schunemann, R. (2013). In H. Schiinemann, J. Brozek, G. Guyatt, & A. Oxman (Eds.)., The GRADE Working Group. GRADE Handbook for Grading Quality of Evidence and Strength of Recommendations. Available from gdt.guidelinedevelopment.org/app/handbook/ handbook.html Updated October 2013.
Schwartz, S. H. (1992). Universals in the content and structure of values: Theoretical advances and empirical tests in 20 countries. Advances in Experimental Social Psychology, 25, 1-65. https://doi.org/10. 1016/S0065-2601 (08)60281-6.
Schwartz, S. H. (1999). A theory of cultural values and some implications for work. Applied Psychology, 48(1), 23-47. https://doi.org/10. 1111/j. 1464-0597.1999.tb00047.x.
Schwartz, S. H., Melech, G., Lehmann, A., Burgess, S., Harris, M., & Owens, V. (2001). Extending the cross-cultural validity of the theory of basic human values with a different method of measurement. Journal of Cross-Cultural Psychology, 32(5), 519-542. https://doi. org/10.1177/0022022101032005001.
Serowik, K. L., Khan, A. J., LoCurto, J., & Orsillo, S. M. (2018). The conceptualization and measurement of values: A review of the psychometric properties of measures developed to inform values work with adults. Journal of Psychopathology & Behavioral Assessment, 40(4), 615-635. https://doi.org/10.1007/s10862-018-9679-1.
[dagger] Smout, M., Davies, M., Bums, N., & Christie, A. (2014). Development of the valuing questionnaire (VQ). Journal of Contextual Behavioral Science, 5(3), 164-172. https://doi.org/10.1016/jjcbs. 2014.06.001.
Terwee, C. B., Bot, S. D., de Boer, M. R., van der Windt, D. A., Knol, D. L., Dekker, J., et al. (2007). Quality criteria were proposed for measurement properties of health status questionnaires. Journal of Clinical Epidemiology, 60(1), 34-42. https://doi.org/10.1016/j. jclinepi.2006.03.012.
Terwee, C. B., Prinsen, C. A. C., Chiarotto, A., Westerman, M. J., Patrick, D. L., Alonso, J., et al. (2018). COSMIN methodology for evaluating the content validity of patient-reported outcome measures: A Delphi study. Quality of Life Research, 27(5), 1159-1170. https:// doi.org/10.1007/s 11136-018-1829-0.
[dagger] Trindade, I. A., Ferreira, C., Pinto-Gouveia, J., & Nooren, L. (2016). Clarity of personal values and committed action: Development of a shorter engaged living scale. Journal of Psychopathology & Behavioral Assessment, 38(2), 258-265. https://doi.org/10.1007/ s10862-015-9509-7.
[dagger] Trompetter, H. R., ten Klooster, P. M., Schreurs, K. M., Fledderus, M., Westerhof, G. J., & Bohlmeijer, E. T. (2013). Measuring values and committed action with the Engaged Living Scale (ELS): Psychometric evaluation in a nonclinical sample and a chronic pain sample. Psychological Assessment, 25(4), 1235-1246. https://doi. org/10.1037/a0033813.
Twohig, M. P., Hayes, S. C., & Masuda, A. (2006). Increasing willingness to experience obsessions: Acceptance and commitment therapy as a treatment for obsessive-compulsive disorder. Behavior Therapy, 37(1), 3-13. https://doi.org/10.1016/j.beth.2005.02.001.
Tyrberg, M. J., Carlbring, P., & Lundgren, T. (2017). Brief acceptance and commitment therapy for psychotic inpatients: A randomized controlled feasibility trial in Sweden. Nordic Psychology, 69(2), 110-125. https://doi.org/10.1080/19012276.2016.119827.
[dagger] Van Buskirk, K., West, L., Malcame, V., Afari, N., Liu, L., Petkus, A., & Wetherell, J. L. (2012). Confirmatory factor analysis of the valued living questionnaire in a black American sample: Implications for cognitive research and practice. Cognitive Therapy & Research, 36(6), 796-805. https://doi.org/10.1007/s10608-011-9405-8.
Van de Mortel, T. F. (2008). Faking it: Social desirability response bias in self-report research. Australian Journal of Advanced Nursing, 25(4), 40-48.
Varra, A. A., Hayes, S. C., Roget, N., & Fisher, G. (2008). A randomized control trial examining the effect of acceptance and commitment training on clinician willingness to use evidence-based pharmacotherapy. Journal of Consulting & Clinical Psychology, 76(3), 449460. https://doi.org/10.1037/0022-006X.76.3.449.
Veehof, M. M., Oskam, M. J., Schreurs, K. M. G., & Bohlmeijer, E. T. (2011). Acceptance-based interventions for the treatment of chronic pain: A systematic review and meta-analysis. Pain, 152, 533-542. https://doi.org/10.1016/j.pain.2010.11.002.
Vowles, K. E., Fink, B., & Cohen, L. (2014). Acceptance and Commitment Therapy for chronic pain: A diary study of treatment process in relation to reliable change in disability. Journal of Contextual Behavioral Science, 3, 74-80. https://doi.org/10.1016/5jcbs.2014.04.003.
Wilson, K. G., & Duffene, T. (2009). Mindfulness for two; An acceptance and commitment therapy approach to mindfulness in psychotherapy. Oakland, CA: New Harbinger.
Wilson, K. G., & Murrell, A. R. (2004). Values work in Acceptance and Commitment Therapy: Setting a course for behavioral treatment. In S. C. Hayes, V. M. Follette, & M. M. Linehan (Eds.), Mindfulness and acceptance: Expanding the cognitive-behavioral tradition (pp. 120-151). New Yoric, NY: Guilford Press.
[dagger] Wilson, K. G., Sandoz, E. K., Kitchens, J., & Roberts, M. (2010). The Valued Living Questionnaire: Defining and measuring valued action within a behavioral framework. The Psychological Record, 60(2), 249-272. https://doi.org/10.1007/BF03395706.
Zettle, R. D., Rains, J. C., & Hayes, S. C. (2011). Processes of change in acceptance and commitment therapy and cognitive therapy for depression: A mediation reanalysis of Zettle and Rains. Behavior Modification, 35(3), 265-283. https://doi.org/10.1177/ 0145445511398344.
Publisher's Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
K. Barrett (1)(iD), M. O'Connor (1), L. McHugh (1)
[mail] El K. Barrett
(1) School of Psychology, University College Dublin, Newman Building, Belfield, Dublin 4, Ireland
Published online: 12 August 2019
Caption: Fig. 1 PRISMA flow diagram of article screening and selection
Table 1 Search terms according to database Database Search String PubMed ((Acceptance and Commitment Therapy)) AND Valu * PsycINFO (Acceptance and Commitment Therapy) AND Valu * Web of Science TS = (Acceptance and Commitment Therapy AND Valu *) Table 2 Inclusion and exclusion criteria for studies Inclusion Criteria Exclusion Criteria * Published journal articles * Nonjoumal articles available in English. (e.g., books, theses etc.). * Articles that include a * Theoretical or background measure of values, relevant to articles. ACT. * Psychometric validation * Case studies, protocol papers, papers of ACT values-based qualitative studies, pilot tools. studies or single subject designs. Table 3 Adapted criteria for good measurement properties (Relevant constructs only) Measurement Property Rating Criteria Structural Validity + CTT: CFA: CFI or TLI or comparable measure > 0.95 OR RMSEA < 0.06 OR SRMR < 0.08 IRT/Rasch: No violation of unidimensionality: CFI or TLI or comparable measure > 0.95 OR RMSEA < 0.06 OR SRMR < 0.08 AND no violation of local independence: residual correlations among the items after controlling for the dominant factor < 0.20 OR Q3's < 0.37 AND no violation of monotonicity: adequate looking graphs OR item scalability > 0.30 AND adequate model fit: IRT: [x.sup.2] > 0.01 Rasch: infit and outfit mean squares [greater than or equal to] 0.5 and [less than or equal to] 1.5 OR Z-standardized values > -2 and < 2 ? CTT: Not all information for '+' reported IRT/Rasch: Model fit not reported - Criteria for *+' not met Internal Consistency + At least low evidence for sufficient structural validity AND Cronbach's alpha(s) [greater than or equal to] 0.70 for each unidimensional scale or subscale ? Criteria for "At least low evidence for sufficient structural validity" not met - At least low evidence for sufficient structural validity AND Cronbach's alpha(s) < 0.70 for each unidimensional scale or Subscale Reliability + ICC or weighted Kappa [greater than or equal to] 0.70 OR Pearson's r [greater than or equal to] 0.7 ? ICC or weighted Kappa not reported - ICC or weighted Kappa < 0.70 OR Pearson's r [greater than or equal to] 0.7 Construct Validity + The result is in accordance with the (Hypothesis Testing) hypothesis (> 75%) ? No hypothesis defined (by the review team) - The result is not in accordance with the hypothesis Convergent Validity + Correlation with gold standard (Adapted from Criterion [greater than or equal to] 0.70 OR AUC Validity criteria) [greater than or equal to] 0.70 ? Not all information for '+' reported - Correlation with gold standard < 0.70 OR AUC < 0.70 Note. Based on criteria outlined in Prinsen et al. (2016) and Terwee et al. (2007). AUC = area under the curve, CFA = confirmatory factor analysis, CFI = comparative fit index, CTT = classical test theory, ICC = intraclass correlation coefficient, IRT = item response theory, RMSEA: Root Mean Square Error of Approximation, SEM = Standard Error of Measurement, SRMR: Standardized Root Mean Residuals, TLI = Tucker-Lewis index Table 4 Overview of the characteristics of the values-assessment tools identified Instrument Author Constructs Measured Bulls Eye Values Lundgren et al. 1. Values Survey (BEVS) (2012). Values Wheel O'Connor et al. 1. Values-directed (VW) (In press) behavior Valuing Smout et al. 1. Values Questionnaire (2014). (VQ) Wilson et al, 1. Values (2010). Valued Living Questionnaire (VLQ) Valued Living Miller et al. (2016) Questionnaire 1. Values for Alcohol Use (VLQ-A) Valued Living Romero-Moreno 1. Values Questionnaire et al. (2016). Adapted for Dementia Caregiving (VLQAC) Chronic Pain McCracken & 1. Values. Values Yang (2006). Inventory (CPVI) Valued Time and Drake et al., 1. Values Difficulty (2018) Questionnaire (VTDQ) Engaged Living Trompetter et al. 1. Values. Scale (ELS) (2013). 2. Committed Action. Comprehensive Francis et al. 1. Psychological Assessment of (2016) Flexibility. Acceptance 2. Cognitive and Delusion. Commitment 3. Acceptance. Therapy 4. Contact with Processes the Present Moment. (CompACT) 5. Values. 6. Committed Action. Multidimensional Rolffs et al. 1. Psychological Psychological (2018) Flexibility. Flexibility 2. Cognitive Inventory Defusion. (MPFI) 3. Acceptance. 4. Contact with the Present Moment 5. Self-as-context. 6. Values. 7. Committed Action. (And each of their inverse processes) Pain Flexibility Cedetberg et al. 1. Values. Scale for (2017). 2. Cognitive Parents Defusion. (PFS P) Values Tracker Pielech et al. 1. Values (VT) (2016). 2. Committed Action Acceptance and Luoma et al. 1. Psychological Action (2011). Flexibility Questionnaire 2. Values (Substance 3. Cognitive Abuse; Fusion AAQ-SA) 4. Acceptance Acceptance and Jacoby et al. 1. Psychological Action (2018) Flexibility Questionnaire 2. Valued (Obsessions Action and 3. Willingness Compulsions; AAQ-OC) Values, Lundgren et al. 1. Psychological Acceptance, (2018) Flexibility Mindfulness 2. Values Scale (VAMS) 3. Acceptance 4. Mindfulness Valued Living Jensen et al. 1. Values-consistent Scale (VLS) (2015) goals. Personal Values Blackledge, 1. Values. Questionnaire Ciarrochi, & II (PVQ) Bailey (2006). Social Values Blackledge, 1. Social and Survey (SVS) Ciarrochi, & Relationship Bailey (2006). Values Academic Values Glick & Orsillo 1. Values Questionnaire (2009). (ACQ) Ciarrochi & Bailey (2008) Survey of Life 1. Life Principles (SLP) principles or Values 2. Goals Novel Values Gregoire, 2. Goals Measure (NVM) Lachance, Bouffard, & 2. Committed Action Dionne, (2018). Work Values Blackledge, 1. Work-related Questionnaire Spencer, & Ciarrochi 1. Work Values (WVQ; adapted (2007). Values from PVQ) Daily Valued Berghoff, 1. Value-guided Action Forsyth, Actions Questionnaire Ritzert, Eifert, & (DVAQ) Anderson (2018) Instrument Subscales Ease of Administration and Scoring (Number of Items in scale) Bulls Eye Values 1. Values Attainment Administration Survey (BEVS) (4 idiographic somewhat items; time-consuming, Work/Hducation, scoring easy (5) Leisure, Relationships, Health/Personal Growth) 2. Values Persistence (1 item) Values Wheel 1. Values-directed Administration (VW) behavior somewhat time-consuming, scoring relatively easy(5) Valuing 1. Values Progress Administration Questionnaire (5 items) quick and scoring (VQ) 2. Values Obstruction easy (10) (5 items) 1. Values Importance Administration (10 items) relatively quick, Valued Living 2. Values Consistency scoring somewhat Questionnaire (10 items) complex (20) (VLQ) Valued Living 1. Values Importance Administration Questionnaire (21 items) time-consuming for Alcohol 2. Values Consistency and scoring Use (VLQ-A) (21 items) somewhat 3. Alcohol complex (63) consistency with Values (21 items) Valued Living 1. Values Importance Administration Questionnaire (12 items) relatively quick, Adapted for 2. Values Consistency scoring somewhat Dementia (12 items) complex (24) Caregiving 3. Commitment to (VLQAC) Own Values (Composite scores of 8 items) 4. Commitment to Family Values (Composite scores of 4 items) Chronic Pain 1. Values Importance Administration Values (6 items) quick and scoring easy(12) Inventory 2. Values Success (CPVI) (6 items) Valued Time and 1. Importance of Administration Difficulty engagement with relatively quick Questionnaire values (10 items) and scoring (VTDQ) 2. Time engaged relatively easy with values (10 items) (30) 3. Difficulty in engaging with Values (10 items) Engaged Living 1. Valued Living Administration Scale (ELS) (10 items) quick and scoring 2. Life Fulfilment easy(16) (6 items) Comprehensive 1. Openness to Administration Assessment of Experience quick and scoring Acceptance (10 items). easy (23) and 2. Behavioral Awareness Commitment (5 items) Therapy 3. Valued Action Processes (8 items) (CompACT) Multidimensional All subscales Administration Psychological comprising 5 relatively quick Flexibility items and scoring easy Inventory 1. Acceptance. (60) (MPFI) 2. Present Moment Awareness. 3. Self as Context 4. Defusion. 5. Values. 6. Committed Action. 7. Flexibility Composite (30 items). 8. Experiential Avoidance. 9. Lack of Contact with the Present Moment 10. Self-as- Content. 11. Fusion. 12. Lack of contact with Values. 13. Inaction. 14. Inflexibility Composite (30 items) Pain Flexibility 1. Pain Resistance Administration Scale for (9 items) quick and scoring Parents 2. Valued Action easy (24) (PFS P) (9 items) 3. Pain Fusion (6 items) Values Tracker 1. Vital Actions Administration (VT) (1 item) quick and scoring 2. Values Progress easy (2) (1 item) Acceptance and 1. Values Administration Action Commitment quick and scoring Questionnaire (9 items) easy(18) (Substance 2. Defused Acceptance Abuse; (9 items) AAQ-SA) Acceptance and 1. Valued Action Administration Action (8 items) quick and scoring Questionnaire 2. Willingness easy(13) (Obsessions (5 items) and Compulsions; AAQ-OC) Values, 1. Acceptance Administration Acceptance, (5 items) quick and scoring Mindfulness 2. Values (3 items[R] easy(11) Scale (VAMS) 3. Mindfulness (3 items) Valued Living 1. Importance Administration Scale (VLS) 2. Success relatively quick 3. Confidence and scoring Each rated across 8 somewhat valued-domains complex (24 + 26 value-related goals) Personal Values 1. Values Administration Questionnaire importance / time-consuming II (PVQ) Commitment / and scoring easy Progress / (81) Success (4 items, each across 9 separate domains). 2. Reasons for Valuing (5 items, each across 9 separate domains) Social Values 1. Values Administration Survey (SVS) importance / relatively quick Commitment / and scoring Progress / relatively easy Success (36) (4 items, each across 4 separate social value domains). 2. Reasons for Valuing (5 items, each across 4 separate social value domains) Academic Values 1. Academic or Administration Questionnaire Educational quick and scoring (ACQ) Values (5 items) easy(5) 1. Importance Administration (58 items)H2. time-consuming Survey of Life Pressure (58 items) and scoring Principles (SLP) 3. Activity (58 items) complex (232) 4. Success (58 items) Novel Values 1 .Values Clarity Administration Measure (NVM) (5 items) quick and scoring 2. Values Coherence easy (10) (5 items) Work Values Administration Questionnaire (9 items) relatively quick (WVQ; adapted and scoring easy (9) from PVQ) Daily Valued 1. Value-guided Administration Action Behaviors quick and scoring Questionnaire (4 items) easy (4) (DVAQ) Instrument Scale type Response Options (Range) Bulls Eye Values Self-rated idiographic 1-7 (for each Survey (BEVS) scale of Values Attainment, of the Values scored by placing an 'X' Attainment within 4 distinct subscales) segments on an image of 1-7 (Values a dartboard. Persistence) Values Persistence subscale also idiographic in nature, and self-rated on an 8-point Likert scale, where 1 = Doesn't prevent me at all, and 7 = Prevents me completely. Values Wheel Self-rated idiographic 0-1 (Relative (VW) scale of values-directed weighting of each behavior. Individuals value converted to write 5 values on 5 decimals; e.g. 20 = separate colored discs. .20) Subsequently, 1-100 (Behavioral behavioral consistency consistency with with each value over the each value) previous week is rated Composite score from 0-100. Finally, the derived by size of the 5 discs are multiplying adjusted to reflect the weighting x relative importance of consistency. each value. Valuing Self-rated 7-point Likert 0-30 (Progress) Questionnaire scale where 0 = Not at 0-30 (Obstruction) (VQ) All True and 6 = Completely True. Self-rated scales ranging 10-100 from 1-10, where 1 = Valued Living Not at all Important / (Range for each: Questionnaire Not at all Consistent, Importance, (VLQ) and 10 = Extremely Consistency and Important / Completely Composite score) Consistent. Valued Living Values Importance and 10-100 (Range for Questionnaire Values Consistency are Importance and for Alcohol scored on self-rated Consistency Use (VLQ-A) scales ranging from subscales) 1-10, where 1 = Not at Value-drinking all Important /Not at all discrepancy scores Consistent, and 10 = (Importance x Extremely Important / Alcohol Completely Consistent. Consistency) range Alcohol consistency with -50 to +50. Values self-rated on a Valued drinking scale ranging from -5 Composite score, (does not fit at all), to ranging from -1,050 +5 (fits extremely well), to +1,050. with no zero option. Valued Living Self-rated scales ranging 10- 100 Questionnaire from 1-10, where 1 = (Range for each: Adapted for Not at all Important / Importance, Dementia Actions not at all Consistency, Caregiving Consistent, and 10 = Composite score, (VLQAC) Extremely Important / Commitment to Own Actions Completely Values, and Consistent. Commitment to Family Values) Chronic Pain Self-rated, 6-point Likert 0-5 (Success) Values scale where 0=Not at all 0-5 (Discrepancy) Inventory Important and 5 = (CPVI) Extremely Important. Valued Time and Self-rated, 11-point scales 0-100 Difficulty asking about agreement (Range for each Questionnaire with statements, where 0 subscale) (VTDQ) = Not at all and 10 = Completely Engaged Living Self-rated, 5-point Likert 10-50 (Valued Scale (ELS) scale where 1 = Living Subscale) Completely Disagree 6-30 (Life and 5 = Completely Fulfilment Agree. Subscale) 16-80 (Total Score) Comprehensive Self-rated on a 7-point 0-60 (Openness to Assessment of Likert scale, ranging Experience) Acceptance from 0 = Never True, 0-30 (Behavioral and to 6 = Always True. Awareness) Commitment 0-48 (Valued Therapy Action) 0-138 Processes (CompACT (CompACT) Composite) Multidimensional Self-rated on a 6-point Not reported. Psychological Likert scale, from Never Flexibility to Always or from Never Inventory True to Always True. (MPFI) Pain Flexibility Parent-rated 7-point 0- 144 Scale for Likert scale, where 0 = (Composite score) Parents Completely Disagree 0-54 (Valued Action (PFS P) and 6 = Entirely Agree. Subscale) Values Tracker Self-rated scale hum 1-10, 0-20 (Composite (VT) where 1 = Not at All, and score) 10 = Most Possible. Acceptance and Self-rated 8-point Likert 7-126 Action scale where 1 = Never (Composite score) Questionnaire True and 7 = Always (Substance True. Abuse; AAQ-SA) Acceptance and Self-rated 7-point Likert 13-91 Action scale where 1 = Never (Composite Score) Questionnaire True and 7 = Always 5-35 (Willingness) (Obsessions True. 8-56 (Valued Action and subscalc) Compulsions; AAQ-OC) Values, Items are self-rated on a 11-77 Acceptance, 7-point Likert scale (Composite score) Mindfulness where 1 = Never True Scale (VAMS) and 7 = Always True. Valued Living Self-rated 11-point Likert Totals for Scale (VLS) scale where 0 = Not at all Importance, Success Important / Successful/ and Confidence, Confident, and 10 = and discrepancies Extremely between Important/Successful / Confidence- Confident. Importance, and Success-Importance can be computed. Personal Values Self-rated, 5-point Likert 36-180 Questionnaire scale for subscale 1, (first subscale) II (PVQ) where 1 = Not at all/Not 45-225 at all Important / (second subscale) Committed / Successful and 5 = Extremely so / Extremely Important / Committed / Successful. Self-rated, 5-point Likert scale for subscale 2, where 1 = Strongly Disagree and 5 = Strongly Agree. Social Values Self-rated, 5-point Likert 16-80 Survey (SVS) scale for subscale 1, (first subscale) where 1 = Not at all / Not 20-100 at all Important / (second subscale) Committed / Successful and 5 = Extremely so / Extremely Important / Committed / Successfiil. Self-rated, 5-point Likert scale for subscale 2, where 1 = Strongly Disagree and 5 = Strongly Agree. Academic Values Self-rated 5-point Likert 5-25 (Total Score) Questionnaire scale where 1 = Strongly (ACQ) Agree and 5--Strongly Disagree. Self-rated on 9-point 1-9 (Importance and scale for both Importance Pressure subscales) and Survey of Life Pressure subscales on 58 1-5 (Success Principles (SLP) distinct life principles, subscale)1-58 where 1 = Unimportant (Activity subscale) To Me / Ifeel no pressure and 9 = Extremely Important to me / I feel extreme pressure. Activity subscale responses are either Yes/No, with Success responses ranging from 1 (Not at all Successful) to 5 (Highly Successful). Novel Values Self-rated 7-point Likert 5-30 (Composite Measure (NVM) scale where 1 = Strongly score) Disagree and 6 = Strongly Agree. Work Values Participants write about 1-20 (first Questionnaire work values, and subscale: (WVQ; adapted subsequently respond to Importance/ from PVQ) 9 statements about these Commitment) 1-25 values. Response options (second subscale: range from 1 (0% Progress/Success) successful), to 5 (81-100% successful) in the first section, and 1 (Strongly Disagree) and 5 (Strongly Agree) in the second. Daily Valued Self-rated 7-point Likert 1-7 (Total Action scale, where 1 = Score--Mean of Questionnaire Disagree Strongly and 7 summed items) (DVAQ) = Agree Strongly. Instrument Intended Age Range Population Bulls Eye Values Clinical and Validated Survey (BEVS) non-clinical with adults adult with populations. epilepsy, age range 21-55 years, and university students (mean age = 26.4 years) Values Wheel Clinical and Validated (VW) non-clinical with adults, adult age 18+ (M = populations. 25.89, SD = 8.55). Valuing Clinical and Validated Questionnaire non-clinical with young (VQ) adult adults, populations. mean age = 20.4 (SD = 4.5; Smout et al., 2014). And with adults with chronic pain (Carvalho et al., 2018) Clinical and Validated with non-clinical young Valued Living adult adults, Questionnaire populations. mean ages (VLQ) = 20/22.6 years. Valued Living University Validated with Questionnaire students. university for Alcohol students Use (VLQ-A) (Af = 19.5 years, SD = 1.76 years). Valued Living Caregivers of Validated Questionnaire relatives with with adults Adapted for dementia. aged 18+, Dementia mean age = Caregiving 60.97 years (VLQAC) (SD = 14.34). Chronic Pain Adults with Validated for Values chronic pain. adults. Mean age Inventory 47.6 years (CPVI) old (5D = 11.7 years). Valued Time and Non-clinical Validated with Difficulty adult university Questionnaire populations. students (VTDQ) aged 19.3 (SD = 3.3). Engaged Living Non-clinical Validated for Scale (ELS) and clinical adults. adult Mean age populations 52.78 years with chronic (SD = pain. 12.37 years). Comprehensive Non-clinical Validated for Assessment of and clinical adults. Acceptance adult Mean age and populations. 31.34 years Commitment (SD = Therapy 11.12). Processes (CompACT) Multidimensional Non-clinical Validated for Psychological and clinical adults. Flexibility adult Mean age Inventory populations. 33.5 years (MPFI) (SD = 12.4). Pain Flexibility Parents of Validated for Scale for children with parents of Parents chronic pain. children (PFS P) aged 0-18. Values Tracker Adults with Validated for (VT) chronic pain. adults based on sample where mean age was 54.7 years (SD = 13.4 years) Acceptance and Adults with a Validated Action history of with adults Questionnaire substance aged 18-63. (Substance abuse or Abuse; misuse. AAQ-SA) Acceptance and Adults with Validated Action Obsessive with adults Questionnaire Compulsive aged 17+ (Obsessions Disorder years. and (OCD) Compulsions; symptoms. AAQ-OC) Values, Ice hockey Validated Acceptance, players. with Senior Mindfulness Ice hockey Scale (VAMS) players in Sweden. Valued Living Adults with Validated with Scale (VLS) chronic pain adults aged 23-91. Personal Values Clinical and Preliminary Questionnaire non-clinical validation II (PVQ) adult studies populations. completed with university undergradu- ate students, but no validation paper identified. Social Values Young adult Preliminary Survey (SVS) populations. validation study conducted with children, but no validation paper identified. Academic Values Students No validation Questionnaire enrolled in paper (ACQ) third level identified. education. Students enrolled in Survey of Life third level No validation Principles (SLP) education. paper identified. Novel Values None specified. No validation Measure (NVM) paper identified. Work Values Students No validation Questionnaire enrolled in paper (WVQ; adapted third level identified. from PVQ) education. Daily Valued Adults No validation Action (aged 18+) paper Questionnaire experiencing identified. (DVAQ) anxiety symptoms. Instrument Available Languages and Adaptations Bulls Eye Values English. The BEVS Survey (BEVS) has also been translated into Spanish and Norwegian. Values Wheel English. Can be (VW) adapted using translations of values cards. Valuing English. The VQ has Questionnaire also been (VQ) translated into Portuguese (Carvalho et al., 2018) English. The VLQ has also been Valued Living translated into Questionnaire Italian (VLQ-I) (VLQ) and Portuguese (see Fernandes, Castilho, & Pinto-Gouveia, 2012, for validation). Eisenbeck, Scheitz, & Szekeres, (2016) reported a Hungarian version of the VLQ was under preparation. Valued Living English. No known Questionnaire adaptations. for Alcohol Use (VLQ-A) Valued Living English and Spanish. Questionnaire Adapted for Dementia Caregiving (VLQAC) Chronic Pain English. A Swedish Values version of the CPVI is also Inventory available for (CPVI) adults with chronic pain (validated in Akerblom et al. 2017). Valued Time and English and Difficulty Japanese. Questionnaire (VTDQ) Engaged Living English and Dutch. Scale (ELS) Adapted to a 9-item version (ELS-9) in Portuguese and validated with young adults (Trindade et al., 2016). Comprehensive English. No known Assessment of adaptations or Acceptance translations. and Commitment Therapy Processes (CompACT) Multidimensional English. No known Psychological adaptations or Flexibility translations. Inventory (MPFI) Pain Flexibility English and Swedish. Scale for Parents (PFS P) Values Tracker English. No known (VT) adaptations. Acceptance and English. No known Action adaptations. Questionnaire (Substance Abuse; AAQ-SA) Acceptance and English. No known Action adaptations. Questionnaire (Obsessions and Compulsions; AAQ-OC) Values, English and Swedish. Acceptance, Mindfulness Scale (VAMS) Valued Living English. No known Scale (VLS) adaptations. Personal Values English. The Questionnaire PVQ-II has also II (PVQ) been translated into German, and translated and validated with a Japanese sample (See Doi, Yokomitsu, & Sakano, 2014). Social Values English. No known Survey (SVS) adaptations. Academic Values English. No known Questionnaire adaptations. (ACQ) English. No known adaptations. Survey of Life Principles (SLP) Novel Values French. No known Measure (NVM) adaptations. Work Values English. No known Questionnaire adaptations. (WVQ; adapted from PVQ) Daily Valued English. No know Action adaptations. Questionnaire (DVAQ) Note. Descriptors such as "Administration quick" implies measures can be completed with minimal instruction and/or feedback, and likely in less than 5-10 minutes. "Administration time-consuming" may be due to the need for support during administration, or due to the length of the questionnaire. Descriptors such as "Scoring easy" indicate that a computation of scores can be done by simple addition, while "Scoring complex" may indicate the need for multiple calculations (e.g. addition plus multiplication and subtraction). Table 5 Results of studies on measurement properties of ACT-based values measures Instrument Country Population N(% (Language) female) Bulls-Eye Values South Africa Adults with 27 (52%) Survey (Lundgren et (English) Epilepsy al., 2012) Sweden University 147 (67%) (Swedish) Students Values Wheel Ireland University 150(66%) (O'Connor et al., (English) Students in press) and Adult non- clinical populations. Valuing Australia University 338 (69%) Questionnaire (Smout (English) Students et al 2014) Australia University 292 (69%) (English) Students Australia University 630 (69%) (English) Students Australia Clinical Sample 285 (65%) (English) Carvalho et al. Portugal Adults with 231 (100%) (2018) (Portuguese) Chronic Pain Portugal University 340 (79%) (Portuguese) students and adult non-clinical populations. Portugal Non-clinical 169 (66%) (Portuguese) adult sample. Valued Living USA University 57 (76%) Questionnaire (English). Students (Wilson et al. 2010) USA University 253 (80%) (English). Students VanBuskirk et al. USA University 128(72%) 2012) (English). Students Valued Living USA University 146 (68%) Questionnaire for (English). Students Alcohol Use (Miller et al. 2016) USA University 222 (55%) (English). Students Valued Living Spain Adult 253 (77%) Questionnaire (Spanish). adapted Caregivers for Dementia of relatives with Dementia Caregiving (Romero-Moreno et al., 2016) Chronic Pain Values United Adults with 140(68%) Inventory (McCracken Kingdom Chronic & Yang, 2006) (English) Pain Akerblom et al. Sweden Adults with 232 (85%) -2017 (Swedish) Chronic Pain Valued Time and Japan and USA University 411 (66%) Difficulty (Japanese Students Questionnaire (Drake and English) et al, 2018) Engaged Living Scale Netherlands University 439 (58%) (Trompetter et al, (Dutch) students. 2013) their parents and grandparents Netherlands Adults with 238 (76%) (Dutch) Chronic Pain Trindade el al, 2016 Portugal University 893 (58%) (Portuguese) Students Portugal University 52 (75%) (Portuguese) Students Comprehensive United Non-clinical 352 (74%) Assessment of Kingdom Adult Acceptance and (English) Sample Commitment Therapy Processes (Francis et al, 2016) Multidimensional USA (English) Non-clinical 372 (52%) Psychological Adult Flexibility Sample Inventory (Rolffs et al, 2018) USA (English) Non-clinical 2150 (60%) Adult + 518 Sample (59%) Pain Flexibility Sweden Parents of 243 (60%) Scale in Parents (Swedish) children (Cederberg et al, with cancer 2017) Values Tracker United Adults with 302 (65%) (Pielech et al, Kingdom Chronic 2015) (English) Pain Acceptance and USA (English) Adults 352 (40%) Action Questionnaire receiving for Substance Abuse treatment (Luoma et al, 2011) for substance abuse issues Acceptance and USA (English) University 511(67%) Action Questionnaire Students for Obsessions and Compulsions (Jacoby et al, 2018) USA (English) University 313 (64%) Students Values, Acceptance, Sweden Senior Ice 93 (22%) Mindfulness Scale (Swedish) Hockey (Lundgrcn et al, Players 2018) Valued Living Scale USA (English) Adults 144 (83%) (Jensen et al, 2015) receiving treatment for chronic pain Instrument Structural Validity Meth. Result Rating Quality Bulls-Eye Values n/a Survey (Lundgren et al., 2012) Values Wheel n/a (O'Connor et al., in press) Valuing Adequate EFA conducted, all Questionnaire (Smout items standardized et al 2014) loadings > .30 (?) Very good CFA conducted--CFI = -99 (+) Very good CFA conducted--CFI = .97 (+) Carvalho et al. Very good CFA conducted--CFI = (2018) .96 (+) Valued Living Questionnaire (Wilson et al. 2010) Adequate PCA, one factor 35.04% variance, ratio 4:1.6 and standardized loadings .39-.65 (?) VanBuskirk et al. Adequate CFA conducted--CFI = 2012) .96. (Importance), CFI = .96 (Consistency) (+) Valued Living Adequate EFA, one factor 60% Questionnaire for variance (ratio Alcohol Use (Miller 8.57:1) and et al. 2016) standardized loadings J9-.73 (?) Adequate EFA, first factor 55% variance (ratio 9.16:1) and standardized loadings 25-.58 (?) Valued Living Adequate EFA, first factor 29% Questionnaire variance. adapted standardized for Dementia loadings all > .4 (?) Caregiving (Romero-Moreno et al., 2016) Chronic Pain Values Inventory (McCracken & Yang, 2006) Akerblom et al. -2017 Valued Time and Difficulty Questionnaire (Drake et al, 2018) Engaged Living Scale Adequate EFA, one factor 36.95% (Trompetter et al, variance and ratio of 2013) first to second factor greater than 4. SRMR = .039, but CFI = .936 (?) Very good CFA conducted--CFI = .98 for bifactor model and CFI = .97 correlated two factor (+) Trindade el al, 2016 Very good CFA conducted--CFI = .91 for bifactor model and RMSEA = .10(-) Doubtful CFA conducted--CFI = .94 for bifactor model and RMSEA = -09 (-) Comprehensive Adequate EFA conducted, 3 Assessment of factor model Acceptance and accounting for 59.6% Commitment Therapy of variance (?) Processes (Francis et al, 2016) Multidimensional Adequate EFA conducted, no Psychological outputs given (?) Flexibility Inventory (Rolffs et al, 2018) Very good IRT model fit not reported (?) CFA 2 factors, CFI = .946, SRMR = .06, RMSEA = .04 (+) Pain Flexibility Adequate EFA, one factor 39% Scale in Parents variance and ratio of (Cederberg et al, first to second factor 2017) greater than 4 (?) Values Tracker (Pielech et al, 2015) Acceptance and Adequate EFA, standardized Action Questionnaire loadings on common for Substance Abuse factor >.30, and item (Luoma et al, 2011) total correlations above 0.4 (?) Acceptance and Adequate EFA one factor 40.41% Action Questionnaire variance and ratio of for Obsessions and first to second factor Compulsions (Jacoby <4 (?) et al, 2018) Very good CFA conducted. CFI = .91, SRMR = 0.06, RMSEA = .11 (+) Values, Acceptance, Doubtful PCA one factor 30.47% Mindfulness Scale variance and ratio of (Lundgrcn et al, first to second factor 2018) <4 (?) Valued Living Scale Very good EFA (Success)--SRMR (Jensen et al, 2015) = 0.03 (Confidence) -RMSEA = 0.01 CFA (Success) RMSEA = 0.05, (Confidence) SRMR = 0.07 (+) Instrument Internal Consistency Meth. Result Rating Quality Bulls-Eye Values n/a Survey (Lundgren et al., 2012) Values Wheel n/a (O'Connor et al., in press) Valuing Questionnaire (Smout et al 2014) Very good Cronbach's a = .87 both subscales (+) Very good Cronbach's [alpha] = .81 Progress. .79 Obstruction (+) Carvalho et al. Very good Cronbach's (2018) [alpha] = .86 Progress, .83 Obstruction (+) Valued Living Very good Cronbach's Questionnaire [alpha] = .79-.83 (Wilson et al. 2010) Importance (+ +) [alpha] = .58-.60, Consistency (--) Cronbach's [alpha] = .65-.74, Composite (+ -) Very good Cronbach's [alpha] = .77 Importance (+) Cronbach's [alpha] = .75 Consistency (+) Cronbach's [alpha] = .77 (+) VanBuskirk et al. Very good Cronbach's 2012) [alpha] = .91 Composite (+) Cronbach's [alpha] = .90 Importance (+) Cronbach's [alpha] = .82 Consistency (+) Valued Living Very good Cronbach's Questionnaire for [alpha] = .97 Alcohol Use (Miller Valued Drinking et al. 2016) Composite (+) Very good Cronbach's [alpha] = .96 Valued Drinking Composite (+) Valued Living Very good Cronbach's Questionnaire [alpha] = .76 adapted Commitment to own for Dementia Values (+) Cronbach's [alpha] = .61 Caregiving Commitment to (Romero-Moreno Family Values (-) et al., 2016) Cronbach's [alpha] = .75 Commitment to Values composite (+) Chronic Pain Values Very good Cronbach's Inventory (McCracken [alpha] = .82 for & Yang, 2006) both Success and Discrepancy subscales (?) Akerblom et al. Very good Cronbach's -2017 [alpha] = .84 for both Success and Discrepancy subscales (?) Valued Time and Very good Cronbach's [alpha] Difficulty = .73 Importance Questionnaire (Drake (?) et al, 2018) Cronbach's [alpha] = .86 Difficulty (?) Cronbach's [alpha] = .65 Time (?) Engaged Living Scale Very good Cronbach's (Trompetter et al, [alpha] = .86 2013) Valued Living and Life Fulfilment (++) Cronbach's [alpha] = .90 Total Scale (+) Very good Cronbach's [alpha] = .89, .87, .91 (Valued Living, Life Fulfilment, Total Scale, respectively) (+++) Trindade el al, 2016 Very good Cronbach's [alpha] = .88, .90, .92 (Valued Living, Life Fulfilment, Total Scale, respectively) (+++) Very good Cronbach's [alpha] =.76, .89, .88 (Valued Living, Life Fulfilment, Total Scale, respectively)(+++) Comprehensive Very good Cronbach's [alpha] Assessment of = .91, .90, .87 Acceptance and for overall Commitment Therapy CompACT score. Processes (Francis Openness to Experience and et al, 2016) Behavioral Awareness scales, respectively. Cronbach's [alpha] = .90 Valued Living subscale(+) Multidimensional Psychological Flexibility Inventory (Rolffs et al, 2018) Very good Cronbach's [alpha] = .91 for values subscale (+) Cronbach's [alpha] = .87 for lack of contact with values subscale. (+) Pain Flexibility Very good Cronbach's [alpha] Scale in Parents = .93 (Total (Cederberg et al, scale) 2017) Cronbach's [alpha] = .89 (Valued Action Subscale (+) Values Tracker (Pielech et al, 2015) Acceptance and Very good Cronbach's [alpha] Action Questionnaire = .84, .85 for Substance Abuse (defused (Luoma et al, 2011) acceptance and total scale score, respectively) Cronbach's [alpha] = .82 for Values Commitment (+) Acceptance and Very good Cronbach's [alpha] Action Questionnaire = .78 and .89 for for Obsessions and Willingness and Compulsions (Jacoby total scale et al, 2018) scores, respectively) Cronbach's [alpha] = .91 for Values (+) Very good Cronbach's [alpha] = .82 and .93 for Willingness and total scale scores, respectively) Cronbach's [alpha] = .92 for Values (+) Values, Acceptance, Very good Cronbach's [alpha] Mindfulness Scale = .76. .75, .82 (Lundgrcn et al, for VAMS total, 2018) Acceptance and Mindfulness scales, respectively. Cronbach's [alpha] = 0.63 Values factor (-) Valued Living Scale Very good Cronbach's [alpha] (Jensen et al, 2015) ranged from .75 - .89, Confidence. Success and Confidence (+++) Instrument Hypothesis Testing Meth. Result Rating Quality Bulls-Eye Values Survey (Lundgren et al., 2012) Very Results in line with good 6 Hypo (+ 6) Values Wheel Very Correlations in (O'Connor et al., good expected in press) directions, though not significant in 2 instances (+4) (-2) Valuing Questionnaire (Smout et al 2014) Very Results in line with good 7 hypos (+7) Carvalho et al. (2018) Very Results in line with good 7 hypos (+7) Valued Living Questionnaire (Wilson et al. 2010) Very good Results in line with 3 Hypo (+3) VanBuskirk et al. Very good No hypotheses 2012) defined in article (?); but consistent with review team's hypotheses (+8) Valued Living Questionnaire for Alcohol Use (Miller et al. 2016) Very good Results in line with 3 Hypo (+3) Valued Living Very good Results in line with Questionnaire 2 Hypo (+2) adapted for Dementia Caregiving (Romero-Moreno et al., 2016) Chronic Pain Values Very good Results in line with Inventory (McCracken 5 Hypo (+5) & Yang, 2006) Akerblom et al. Very good Results in line with -2017 4 Hypo (+4) Valued Time and Very good Results in line with Difficulty 2 Hypos (+2) Questionnaire (Drake et al, 2018) Engaged Living Scale Very good Results in line with (Trompetter et al, 4 Hypo (+4) and 2013) not in line with 1 (-1) Very good Results in line with 4 Hypo (+4) Trindade el al, 2016 Very good Results in line with 5 Hypo (+5) Very good Results in line with 5 Hypo (+5) Comprehensive Very good Results in line with Assessment of 5 Hypo (+5) Acceptance and Commitment Therapy Processes (Francis et al, 2016) Multidimensional Psychological Flexibility Inventory (Rolffs et al, 2018) Very good Results in line with 15 hypotheses (+15) Pain Flexibility Very good Results in line with Scale in Parents 4 Hypo (44) (Cederberg et al, 2017) Values Tracker Very good Result in line with (Pielech et al, 5 Hypo (+5) 2015) Acceptance and Very good Results in line with Action Questionnaire 6 Hypo (+6) and for Substance Abuse not in line with 1 (Luoma et al, 2011) Hypo (-1) Acceptance and Action Questionnaire for Obsessions and Compulsions (Jacoby et al, 2018) Very good Results in line with 6 Hypo (+6). (D) Adequate (C) Values, Acceptance, Very good Results in line with Mindfulness Scale 5 Hypo (+5) (Lundgrcn et al, 2018) Valued Living Scale Very good Results in line with (Jensen et al, 2015) 2 Hypo (+2) Instrument Convergent and Discriminant Validity Meth. Result Rating Quality Bulls-Eye Values n/a Survey (Lundgren et al., 2012) Values Wheel Very good Convergence Correlations (O'Connor et al., with existing values in press) measures not [greater than or equal to] 0.70, at .26 and .16 (-) Valuing Questionnaire (Smout et al 2014) Very good Convergence Correlations with existing measure not [greater than or equal to] 0.70 (-) Carvalho et al. (2018) Very good Convergence Correlations with existing measures not > 0.70 (-) Valued Living Questionnaire (Wilson et al. 2010) VanBuskirk et al. Very good Discriminant 2012) No significant correlations between importance/consistcncy scores and unrelated demographic variables (+) Valued Living Questionnaire for Alcohol Use (Miller et al. 2016) Valued Living Questionnaire adapted for Dementia Caregiving (Romero-Moreno et al., 2016) Chronic Pain Values Inventory (McCracken & Yang, 2006) Akerblom et al. -2017 Valued Time and Difficulty Questionnaire (Drake et al, 2018) Engaged Living Scale (Trompetter et al, 2013) Trindade el al, 2016 Very good Very good Convergence Correlations with existing measures not [greater than or equal to] 0.70 (-) Comprehensive Very good Discriminant Assessment of Acceptance and No significant correlations Commitment Therapy between CompACT Processes (Francis scale scores and social desirability (+) et al, 2016) Convergence Assessed with AAQ-II, not values measures. Multidimensional Psychological Flexibility Inventory (Rolffs et al, 2018) Very good Discriminant Low to moderate correlations between MPFI values subscale Pain Flexibility scores and unrelated Scale in Parents construct (+) (Cederberg et al, 2017) Values Tracker Very good Convergence Correlations (Pielech et al, with existing measures not 2015) [greater than or equal to] 0.70 (-) Acceptance and Action Questionnaire for Substance Abuse (Luoma et al, 2011) Acceptance and Action Questionnaire for Obsessions and Compulsions (Jacoby et al, 2018) Very good Discriminant (D) Adequate Correlations between [C] AAQ-OC and related measures were significantly larger in magnitude than correlations with unrelated measures (+) Convergent Correlations between AAQ-OC and related constructs all <70 (-) Values, Acceptance, Mindfulness Scale (Lundgrcn et al, 2018) Valued Living Scale (Jensen et al, 2015) Instrument Test-retest Reliability Meth. Result Rating Quality Bulls-Eye Values Survey (Lundgren et al., 2012) Adequate Pearson's r = .85, .70, .71 (Values attainment over 3 time points) (+) Pearson's r = .89, .90. .71 (Persistence barriers over 3 time points) (+) Values Wheel Adequate Spearman's (O'Connor et al., r = .57, .57 and .65 in press) (over 3 time points. 2 week interval) Valuing Questionnaire (Smout et al 2014) Carvalho et al. Adequate 117 women completed (2018) second VQ. Pearson's r = .62, .52 (Obstruction and Progress, respectively) (-) Valued Living Adequate Pearson's Questionnaire r = .90 (Wilson et al. 2010) Importance (+) and r =. 58 Consistency (-) and r = .75 Composite Score (+) VanBuskirk et al. 2012) Valued Living Adequate Pearson's Questionnaire for r = .74 Importance Alcohol Use (Miller (+) et al. 2016) Pearson's r = .66 Alcohol Consistency (-) Valued Living Questionnaire adapted for Dementia Caregiving (Romero-Moreno et al., 2016) Chronic Pain Values Inventory (McCracken & Yang, 2006) Akerblom et al. -2017 Valued Time and Difficulty Questionnaire (Drake et al, 2018) Engaged Living Scale (Trompetter et al, 2013) Trindade el al, 2016 Adequate Pearson's r = .84-.88. (+++) Adequate Pearson's r = .78, .81,-86 (Valued Living, Life Fulfilment, Total Scale, respectively) (+++) Comprehensive Assessment of Acceptance and Commitment Therapy Processes (Francis et al, 2016) Multidimensional Psychological Flexibility Inventory (Rolffs et al, 2018) Pain Flexibility Adequate Pearson's Scale in Parents r = .87 (Cederberg et al, (Total scale) 2017) Pearson's r = .80 (Valued Action subscale) (+) Values Tracker (Pielech et al, 2015) Acceptance and Action Questionnaire for Substance Abuse (Luoma et al, 2011) Acceptance and Action Questionnaire for Obsessions and Compulsions (Jacoby et al, 2018) Values, Acceptance, Mindfulness Scale (Lundgrcn et al, 2018) Valued Living Scale (Jensen et al, 2015) Note. EFA = Exploratory Factor Analysis, CFA = Confirmatory Factor Analysis, CFI = Comparative Fit Index, PCA = Principal Component Analysis, RMSEA = Root Mean Square Error of Approximation, SRMR: Standardized Root Mean Residual
|Printer friendly Cite/link Email Feedback|
|Title Annotation:||ORIGINAL ARTICLE|
|Author:||Barrett, K.; O'Connor, M.; McHugh, L.|
|Publication:||The Psychological Record|
|Date:||Dec 1, 2019|
|Previous Article:||Unfinished Business: J.E.R. Staddon's The New Behaviorism, 2nd ed.|
|Next Article:||Assessing a Derived Transformation of Functions Using the Implicit Relational Assessment Procedure Under Three Motivative Conditions.|