Randomized trial of stage-based interventions for informed medicare choices.
The 1997 Balanced Budget Balanced budget
A budget in which the income equals expenditure. See: budget.
A budget in which the expenditures incurred during a given period are matched by revenues. Act (BBA BBA
Bachelor of Business Administration ) authorized new health plan options under the Medicare Medicare, national health insurance program in the United States for persons aged 65 and over and the disabled. It was established in 1965 with passage of the Social Security Amendments and is now run by the Centers for Medicare and Medicaid Services. +Choice program, providing Medicare beneficiaries with a range of choices that could potentially offer more benefits at lower cost than traditional fee-for-service Medicare. The 2003 Medicare Prescription Drug, Improvement, and Modernization Act The Medicare Prescription Drug, Improvement, and Modernization Act (Pub.L. 108-173, 117 Stat. 2066, also called Medicare Modernization Act or MMA) is a law of the United States which was enacted in 2003. (MMA (Microcomputer Managers Association, Inc.) A membership organization with chapters throughout the U.S. that was devoted to educating personnel responsible for personal computers. It disbanded in 1996.
Mma - A fast Mathematica-like system, in Allegro CL by R. Fateman, 1991. ) added a prescription drug prescription drug Prescription medication Pharmacology An FDA-approved drug which must, by federal law or regulation, be dispensed only pursuant to a prescription–eg, finished dose form and active ingredients subject to the provisos of the Federal Food, Drug, benefit and expanded plan options. Those changes and others introduced with the MMA dramatically increase the complexity of the Medicare Program and decisionmaking process for beneficiaries and new enrollees.
To help meet Medicare beneficiaries' need for information and decision support, CMS (1) See content management system and color management system.
(2) (Conversational Monitor System) Software that provides interactive communications for IBM's VM operating system. has focused on information infrastructure, public-private partnerships, its Web site, and 1-800-Medicare toll-free help line. However, improvements in these areas cannot ensure that beneficiaries will carefully consider the range of health plan options available. A recent survey found that although 52 percent of Internet Internet
Publicly accessible computer network connecting many smaller networks from around the world. It grew out of a U.S. Defense Department program called ARPANET (Advanced Research Projects Agency Network), established in 1969 with connections between computers at the users age 65 or over had heard of the Medicare. gov Web site, only 11 percent had ever visited it, and although 55 percent of seniors had heard of the toll-free Medicare help line, only 13 percent had ever called it (Kaiser Family Foundation The Henry J. Kaiser Family Foundation (KFF), or just Kaiser Family Foundation, is a U.S.-based non-profit, private operating foundation headquartered in Menlo Park, California. , 2006).
The TTM TTM
Trailing 12 months. Often used with Earnings Per Share. offers an empirically validated val·i·date
tr.v. val·i·dat·ed, val·i·dat·ing, val·i·dates
1. To declare or make legally valid.
2. To mark with an indication of official sanction.
3. framework to help beneficiaries increase their information-seeking behavior and involvement in choice. Research on the TTM has found that behavior change Behavior change refers to any transformation or modification of human behavior. Such changes can occur intentionally, through behavior modification, without intention, or change rapidly in situations of mental illness. involves progress, over time, through a series of stages that represent ordered categories along a continuum Continuum (pl. -tinua or -tinuums) can refer to:
- Continuum (theory), anything that goes through a gradual transition from one condition, to a different condition, without any abrupt changes or "discontinuities"
Compleat Angler, The
Izaak Walton’s classic treatise on the Contemplative Man’s Recreation. [Br. Lit.: The Compleat Angler]
sculpture by Rodin, depicting contemplative man. , preparation, action, and maintenance (Prochaska and DiClemente, 1983). The model includes additional dimensions central to change:
* Decisional Balance--The pros and cons associated with a behavior's consequences (Janis and Mann, 1977).
* Self-Efficacy--Confidence to make and sustain changes in difficult situations, and temptation Temptation
Terror (See HORROR.)
as fruit of the tree of knowledge in Eden, has come to epitomize temptation. [O.T.: Genesis 3:1–7; Br. Lit. to slip back into old patterns (Bandura ban`dur´a
n. 1. A traditional Ukrainian stringed musical instrument shaped like a lute, having many strings. , 1977).
* Processes of Change--10 cognitive, affective affective /af·fec·tive/ (ah-fek´tiv) pertaining to affect.
1. Concerned with or arousing feelings or emotions; emotional.
2. , and behavioral behavioral
pertaining to behavior.
see psychomotor seizure. activities that facilitate progress through the stages of change (Prochaska and DiClemente, 1985).
Over 25 years of research on a variety of health behaviors and in a variety of populations have identified the principles and processes of change that work best in each stage to facilitate progress. This research served as a foundation on which to build stage-matched interventions to increase participation in informed health plan choice among Medicare beneficiaries.
In Phase I of a program of research applying the TIM TIM Timothy
TIM Technical Interchange Meeting
TIM Transient Intermodulation Distortion
TIM Time Is Money
TIM The Invisible Man (movie)
TIM Telecom Italia Mobile (Italian cellular provider) to informed choice in the Medicare population, CMS' Center for Beneficiary beneficiary
Person or entity (e.g., a charity or estate) that receives a benefit from something (e.g., a trust, life-insurance policy, or contract). A primary beneficiary receives proceeds from a trust or insurance policy before any other. Choices contracted with Pro-Change Behavior Systems to" (1) develop an operational definition of informed choice, (2) develop and validate To prove something to be sound or logical. Also to certify conformance to a standard. Contrast with "verify," which means to prove something to be correct.
For example, data entry validity checking determines whether the data make sense (numbers fall within a range, numeric data measures of the core constructs of the TTM, and (3) establish decision rules and cutoff scores necessary for the development of TIM stage-based interventions. Phase I measures and findings demonstrating the applicability of the TIM to informed choice are described elsewhere (Levesque et al., 2006).
In Phase II, Pro-Change developed TIM-based interventions to increase informed choice among new enrollees by accelerating progress through the stages of change for comparing plans, operationally defined as:
* Finding out what your Medicare health plan choices are.
* Gathering information on the different Medicare health plans.
* Comparing the advantages and disadvantages of your choices, such as cost, benefits covered, doctors and hospitals you can use, rules you must follow to get care, and the quality of the health care provided.
* Using this information to choose the plan that best meets your needs given your health and financial situation.
For a detailed description of the TIM-based intervention A procedure used in a lawsuit by which the court allows a third person who was not originally a party to the suit to become a party, by joining with either the plaintiff or the defendant. materials refer to Levesque and Cummins (2002).
In Phase III Noun 1. phase III - a large clinical trial of a treatment or drug that in phase I and phase II has been shown to be efficacious with tolerable side effects; after successful conclusion of these clinical trials it will receive formal approval from the FDA , a randomized ran·dom·ize
tr.v. ran·dom·ized, ran·dom·iz·ing, ran·dom·iz·es
To make random in arrangement, especially in order to control the variables in an experiment. trial was conducted to assess the efficacy of the TIM intervention materials among individuals newly eligible for Medicare benefits. Additional information on approvals from administrative agencies and organizations, the structure and content of the intervention materials, and study eligibility criteria and recruitment procedures are available in Levesque and Cummins (2004).
The multimedia computer-administered expert system intervention, entitled en·ti·tle
tr.v. en·ti·tled, en·ti·tling, en·ti·tles
1. To give a name or title to.
2. To furnish with a right or claim to something: "Making Good Choices about Medicare Health Plans", was intended for new enrollees who had just received their initial enrollment package from CMS 3 months prior to reaching age 65. The program was disseminated disseminated /dis·sem·i·nat·ed/ (-sem´i-nat?ed) scattered; distributed over a considerable area.
Spread over a large area of a body, a tissue, or an organ. over the Internet with multimedia components residing on a CD-ROM. During a 20-minute interactive session, the expert system compiled text paragraphs, audio files, and images as it administered a TIM assessment and provided immediate individualized in·di·vid·u·al·ize
tr.v. in·di·vid·u·al·ized, in·di·vid·u·al·iz·ing, in·di·vid·u·al·iz·es
1. To give individuality to.
2. To consider or treat individually; particularize.
3. feedback on the participants' stage of change, decisional balance, processes of change, self-efficacy self-efficacy (selfˈ-eˑ·fi·k , and strategies for taking small steps to progress to the next stage. The expert system also referred participants to sections of the manual. At the end of the session, enrollees could print out a report containing their feedback. The final screens in the interactive session and the printed report included links to parts of the Medicare Web site that were most stage appropriate. For example, participants in the contemplation stage were linked to "Helpful Contacts," and those in action were linked to "Medicare Compare."
The 30-page manual is based on TTM research on new Medicare enrollees. Designed to be used alone or with the expert system, the manual provides a definition of comparing plans, asks readers to assess their own readiness to compare plans, and guides them to sections containing stage-matched information and exercises based on TIM principles and processes of change. The manual integrates CMS educational information, support tools, and services in a stage-appropriate fashion, and includes a plan comparison worksheet.
Recruitment and Random Assignment
The efficacy of the manual plus expert system intervention was tested in a sample of 451 new Medicare enrollees who had access to a computer with a CD-ROM and Internet connection (Subsample sub·sam·ple
A sample drawn from a larger sample.
tr.v. sub·sam·pled, sub·sam·pling, sub·sam·ples
To take a subsample from (a larger sample). One); the efficacy of the manual alone was tested in a sample of 900 new enrollees, most of whom did not have computer access (Subsample Two). New enrollees were randomly selected from CMS' Initial Enrollment File, a repository (1) A database of information about applications software that includes author, data elements, inputs, processes, outputs and interrelationships. A repository is used in a CASE or application development system in order to identify objects and business rules for reuse. of data for persons who were going to turn age 65 in the next 3 months and become eligible for Medicare. A telematch service located telephone numbers for 48 percent of selected enrollees, whose contact information was then forwarded to a survey research center.
At baseline The horizontal line to which the bottoms of lowercase characters (without descenders) are aligned. See typeface.
baseline - released version , the Making Good Choices about Medicare Health Plans Survey was administered using a mixed-mode methodology involving an initial mail survey with a telephone followup for non-responders. At the start of the survey, a question assessed access to a computer with a CD-ROM and Internet connection. Participants with computer access were assigned as·sign
tr.v. as·signed, as·sign·ing, as·signs
1. To set apart for a particular purpose; designate: assigned a day for the inspection.
2. to Subsample One, and those without were assigned to Subsample Two. Once recruitment for the first subsample was completed, individuals with computer access were included in the second subsample until the target sample size was reached. Next, individuals in Subsample One were randomly assigned to an intervention group that received the manual plus expert system intervention, or to a control group that did not; individuals in Subsample Two were randomly assigned to the intervention group that received the manual alone, or to a control group that received no materials.
The baseline survey response rate was 70 percent. On average, respondents In the context of marketing research, a representative sample drawn from a larger population of people from whom information is collected and used to develop or confirm marketing strategy. completed the baseline survey 7.1 weeks (standard deviation [SD]=I.8) before age 65. Forty-seven percent had access to a computer with a CD-ROM and Internet connection-100 percent of Subsample One and 19 percent of Subsample Two. Eighty-three percent of the sample was White non-Hispanic, 7 percent Black non-Hispanic, 3 percent Hispanic Hispanic Multiculture A person of Mexican, Puerto Rican, Cuban, Central or South American, or other Spanish culture or origin, regardless of race Social medicine Any of 17 major Latino subcultures, concentrated in California, Texas, Chicago, Miam, NY, and elsewhere , and 7 percent other. Fifty-seven percent were female, 48 percent had annual household incomes above 530,000, and 70 percent were married. Sixteen percent had not completed high school, 38 percent had completed high school or earned a GED GED
1. general equivalency diploma
2. general educational development
GED (US) n abbr (Scol) (= general educational development) → , 28 percent had attended some college, and 18 percent had a college degree. Fifty-six percent of participants reported that they had already chosen a Medicare health plan; 6 percent of the sample reported that they will be covered by Medicaid in addition to Medicare. At baseline, 36 percent of the sample were in the precontemplation stage for comparing plans, 6 percent were in contemplation, 14 percent in preparation, and 44 percent in action. Among participants who had already chosen a Medicare health plan, 34 percent were in precontemplation, 3 percent contemplation, 6 percent preparation, and 57 percent action. Presumably pre·sum·a·ble
That can be presumed or taken for granted; reasonable as a supposition: presumable causes of the disaster. , individuals in the preaction stages who had already chosen a health plan had done so without comparing their options.
Control and treatment group characteristics for Subsamples One and Two at baseline are presented in Table 1. Measures assessed at baseline revealed no significant differences between the Subsample One control and treatment groups, or between the Subsample Two control and treatment groups for demographics The attributes of people in a particular geographic area. Used for marketing purposes, population, ethnic origins, religion, spoken language, income and age range are examples of demographic data. , health status, Medicaid eligibility, or other study variables. However, there were significant differences (p<0.01) between Subsamples One and Two on all measures (Table 1). For example, individuals in Subsample One had more education and higher incomes, and scored higher on objective and subjective measures of Medicare knowledge. These subsample differences precluded any direct comparison of the manual plus expert system intervention versus the manual alone.
After completing the baseline survey, intervention materials were mailed to treatment group participants. One month later, a brief survey was mailed to assess whether treatment group participants had used the materials, and whether they found them to be understandable, easy to use, attractive, and helpful. One hundred and thirtytwo (29 percent) of the 449 new enrollees who received the manual alone and 51 (23 percent) of the 226 new enrollees who received the manual plus expert system intervention returned the acceptability survey (1). A majority of respondents found the manual to be easy to use and understand, attractive, and informative; 87 percent still had the manual, and 85 percent reported that they would recommend it to a friend. Acceptability ratings for the expert system intervention were similar to those for the manual. A more detailed description of the acceptability findings is available in Levesque and Cummins (2004).
At 6-months' followup, the Making Good Choices about Medicare Health Plans Final Survey assessing outcomes was completed by 1,087 of 1,351 study participants (81 percent). Compared to respondents, non-respondents had significantly lower baseline knowledge scores, were less likely to have attended some college, and were more likely to be of Hispanic origin or descent descent, in anthropology, method of classifying individuals in terms of their various kinship connections. Matrilineal and patrilineal descent refer to the mother's or father's sib (or other group), respectively. , or separated. On average, participants completed the followup survey 18.6 weeks (SD=3.3) after reaching age 65.
The baseline survey assessed demographics and health status. The outcome measures were assessed at baseline and followup, unless otherwise noted.
* Medicare Knowledge--A 7-item knowledge quiz A quiz is a form of game or mind sport in which the players (as individuals or in teams) attempt to answer questions correctly. Quizzes are also brief assessments used in education and similar fields to measure growth in knowledge, abilities, and/or skills. taken from Rounds 23 and 26 of the Medicare Current Beneficiary Survey (MCBS MCBS Medicare Current Beneficiary Survey
MCBS Microcomputer Business Services ) assessed knowledge about the Medicare Program. A scale score was computed as the total number of correct responses (Bann et al., 2003). Cronbach's (1951) Alpha for the scale was 0.64 in the current study.
* Perceived Knowledge--Three questions from Rounds 24 and 26 of MCBS assessed how much new enrollees felt they knew about: (1) what medical services Medicare covers, (2) supplemental or Medigap insurance, and (3) the availability and benefits of Medicare managed care plans. Response options ranged from 1 (just about everything you need to know) to 5 (almost none of what you need to know). The response scale was reversed and a scale score was calculated by taking the mean of the three items (Bann et al., 2003). Cronbach's (1951) Alpha was 0.81 in the current study.
* Use of Traditional Medicare Information-Questions adapted from the 2003 CMS Survey of New Medicare Beneficiaries assessed information seeking Information seeking is the process or activity of attempting to obtain information in both human and technological contexts. Information seeking is related to, but yet different from, information retrieval (IR). . Respondents were asked to place a checkmark next to each information source they used during the last 12 months. For each source used, respondents indicated how useful they found the information in understanding their health plan choices. Response options ranged from 1 (not useful at all) to 4 (very useful). We examined use of and satisfaction with two formal information sources: the Medicare & You handbook and the Medicare Web site.
* Satisfaction with Medicare Information--At followup only, two questions taken from Rounds 20 and 23 of MCBS assessed satisfaction with Medicare information: (1) "In general, do you think the Medicare program is understandable?" and (2) "How satisfied are you in general with the availability of information about the Medicare program when you need it?" Response options for the first question were "yes," "no," and "I don't know;" response options for the second were "very satisfied," "satisfied," "very unsatisfied, "not applicable," and "I don't know."
* Health Plan Choice--Participants were asked whether they had chosen a Medicare health plan and, if so, to indicate the general type of plan they had chosen. Response options were "the Original Medicare plan (with or without supplemental Medigap insurance)," "a Medicare managed care plan (HMO HMO health maintenance organization.
A corporation that is financed by insurance premiums and has member physicians and professional staff who provide curative and preventive medicine within certain financial, )," "a Medicare Private Fee-for-Service Plan," and "I am not sure what I chose."
* Stage of Change for Comparing Plans--Participants were presented with the operational definition of comparing plans previously presented, and then asked: (1) "Have you compared different Medicare health plans in the last 6 months?" (2) "Do you intend to compare different health plans in the next 3 months?" and (3) "Do you intend to compare different plans in the next 30 days?" Individuals who had compared plans in the last 6 months were classified in the action stage. Among individuals who had not compared plans, those who had no intention of doing so in the next 3 months were classified in precontemplation, those intended to do so in the next 3 months were classified in contemplation, and those who intended to do so in the next 30 days were classified in preparation. Validation See validate.
validation - The stage in the software life-cycle at the end of the development process where software is evaluated to ensure that it complies with the requirements. research found that individuals in later stages were more knowledgeable about the Medicare Program, engaged in more information seeking, and were more likely to have chosen a Medicare health plan (Levesque et al., 2006).
Medicare Knowledge and Perceived Knowledge--Two 3-way repeated measures analyses of variance The discrepancy between what a party to a lawsuit alleges will be proved in pleadings and what the party actually proves at trial.
In Zoning law, an official permit to use property in a manner that departs from the way in which other property in the same locality assessed the effect of the TTM interventions on change in Medicare knowledge and perceived knowledge, the dependent variables. The between-subjects independent variables were intervention condition (treatment/ control) and subsample (One/Two), and the within-subjects independent variable was assessment timepoint (baseline/followup). or continuous measures, [[eta].sup.2] (eta square) was used as the measure of effect size. For [[eta].sup.2], 0.010 generally indicates a small effect, 0.059 a medium effect, and 0.138 a large effect (Cohen cohen
(Hebrew: “priest”) Jewish priest descended from Zadok (a descendant of Aaron), priest at the First Temple of Jerusalem. The biblical priesthood was hereditary and male. , 1988). However, for large-scale population-based intervention trials, which tend to find effect sizes in the 0.010 range, small, medium, and large effects can be defined as 0.005, 0.010, and 0.015, respectively (Rossi, 2003). On a population basis, small effects can have enormous practical importance and overall impact.
Use of and Satisfaction with Medicare Information--Among study participants who had not chosen a Medicare health plan at baseline, one-tailed 2x2 Chi Square chi square (kī),
n a nonparametric statistic used with discrete data in the form of frequency count (nominal data) or percentages or proportions that can be reduced to frequencies. tests examined the main effect of intervention condition (treatment/control) on the following 6-month measures: (1) reading the Medicare & You handbook; (2) perceived helpfulness of the handbook among users; (3) use of Medicare Web site; (4) perceived helpfulness of the Web site among users; (5) whether Medicare information was understandable; and (6) satisfaction with Medicare information. We then assessed condition x subsample interaction effects by calculating the mean of the effect sizes across the information use and satisfaction measures in Subsample One, and then in Subsample Two. A t-test of the difference between Subsamples One and Two effect size means assessed whether there was a differential impact of the manual plus expert system intervention versus the manual alone in their respective subsamples. For dichotomous di·chot·o·mous
1. Divided or dividing into two parts or classifications.
2. Characterized by dichotomy.
di·chot measures, Cohen's h was used as the measure of effect size. Generally, h=0.20 indicates a small effect, 0.50 a medium effect, and 0.80 a large effect (Cohen, 1988). However, for large-scale population-based intervention trials, small, medium and large effects can be defined as h=0.15, 0.20, and 0.25, respectively (Rossi, 2003).
Health Plan Choice--Among participants who had not chosen a Medicare health plan at baseline, one-tailed 2x2 Chi Square tests examined the main effect of intervention condition (treatment/control) on whether participants had chosen a plan at followup and could indicate the general type of plan chosen. Participants who were not sure what type of plan they had chosen were grouped with participants who had not chosen a plan, since their choice was unlikely to have been an informed one. Condition x subsample interaction effects were examined by calculating and comparing intervention effect sizes for Subsamples One and Two.
Stage Progression and Regression--Among participants who had not chosen a Medicare health plan and were in a preaction stage at baseline, stage progression was defined as movement to a more advanced stage at followup. Among study participants who had not chosen a plan and were in the contemplation, preparation, or action stage at baseline, stage regression regression, in psychology: see defense mechanism.
In statistics, a process for determining a line or curve that best represents the general trend of a data set. was defined as movement to an earlier stage at followup. Two one-tailed 2x2 Chi Square tests examined the overall or main effect of treatment condition (treatment/ control) on stage progression and regression. Condition x subsample interaction effects were examined by calculating and comparing intervention effect sizes for Subsamples One and Two.
Medicare Knowledge and Perceived Knowledge
Figure 1 shows Medicare knowledge scores at baseline and followup for Sample One and Sample Two treatment and control groups. There was a within-subjects effect for time (mean knowledge scores increased from 4.0 (SD--1.9) at baseline to 4.4 (SD--1.8) at followup, F(1,1024)=46.9, p<0.001, [[eta].sup.2]=0.044), and a between-subjects effect for subsample (knowledge scores were higher in Subsample One, F(1,1024)=24.3, p<0.001, [[eta].sup.2]=0.023). There was an effect for condition x time (F(1,1024)=12.3, p<0.001, q2[[eta].sup.2]=0.012), indicating that treatment group participants showed greater gains in Medicare knowledge over time than control group participants. No effect was found for condition x subsample x time (F(1,1024)=0.8, p=0.38, [[eta].sup.2]=0.001), indicating that the pattern of treatment-control group differences in Subsample One was not different from the pattern in Subsample Two.
[FIGURE 1 OMITTED]
Results for perceived knowledge show a similar pattern of improvement (Figure 2), with a within-subjects effect for time (in the whole sample, scores increased from 2.8 (SD--1.1) to 3.0 (SD=I.0), F(1,1070)=47.5, p<0.001, [[eta].sup.2]=0.043), a between-subjects effect for subsample (F(1,1070)=39.6, p<0.001, [[eta].sup.2]=0.036), an interaction effect for condition x time (F(1,1070)=4.3, p<0.05, [[eta].sup.2]=0.004), and no interaction effect for condition x subsample x time (F(1,1070)-0.0, p=0.912, [[eta].sup.2]=0.000).
Use of and Satisfaction with Medicare Information
First, examining the overall sample, Table 2 shows rates of use of and satisfaction with Medicare information among treatment and control group participants who had not chosen a Medicare health plan at baseline. Findings on the individual outcome measures were in the predicted direction, but did not always reach statistical significance. Across all six measures, the mean effect size was 0.183 (95 percent confidence interval confidence interval,
n a statistical device used to determine the range within which an acceptable datum would fall. Confidence intervals are usually expressed in percentages, typically 95% or 99%. (CI)=0.115 to 0.250), which was significantly different from 0.
[FIGURE 2 OMITTED]
Next, we examined the effect of the TIM interventions on the same six outcome measures in the separate subsamples. In all cases, findings were in the predicted direction: individuals in Subsamples One and Two treatment groups were more likely than their peers in the control groups to: (1) have read their Medicare & You handbook (57 versus 42 percent in Subsample One, 41 versus 37 percent in Subsample Two); (2) have found their handbook somewhat or very helpful if they had read it (85 versus 71 percent in Subsample One, 86 versus 78 percent in Subsample Two); (3) have used the Medicare Web site (31 versus 18 percent in Subsample One); (4) have found the Web site somewhat or very helpful if they had used it (92 versus 67 percent in Subsample One); (5) find Medicare information understandable (67 versus 49 percent in Subsample One, 53 versus 48 percent in Subsample Two); and (6) be satisfied or very satisfied with Medicare information (81 versus 66 percent in Subsample One, 65 versus 63 percent in Subsample Two).
Table 2 shows effect sizes for the manual plus expert system intervention in Subsample One, and for the manual alone in Subsample Two. The mean effect size in Subsample One was about 31/2 times greater than the mean effect size in Subsample Two (0.384, 95 percent CI=0.276 to 0.492 versus 0.110, CI=0.033 to 0.187). The mean effect sizes for the two subsamples were both significantly greater than 0, and were significantly different from each other (t(8)-3.6, p<0.01).
A post hoc post hoc
adv. & adj.
In or of the form of an argument in which one event is asserted to be the cause of a later event simply by virtue of having happened earlier: test was conducted to examine whether differences in the effect sizes for the two subsamples might be explained, at least in part, by education, one of the major differences between the two subsamples, rather than by any differences between the two interventions. Previous analyses were expanded by assessing condition x subsample x education interaction effects among participants with low (high school or less) and high (at least some college) education. Table 3 shows effect sizes for the Medicare information use and satisfaction measures among Subsamples One and Two participants with the two levels of education. The mean effect size for the manual plus expert system intervention among Subsample One participants with low and high education were 0.360 (CI=0.004 to 0.715) and 0.423 (CI=0.334 to 0.512), respectively; the mean effect size for the manual alone among Subsample Two participants with low and high education were 0.018 (CI=-0.074 to 0.110) and 0.270 (CI=0.183 to 0.357), respectively. With the exception of the Sample Two low education group, mean intervention effects for all groups were statistically different from 0, quite large for a population-based intervention trial, and roughly equivalent to each other. Multiple comparisons found that the mean effect size for the Sample Two low education group was significantly lower than the mean effects for the Sample One and Sample Two high education groups (t(8)=6.0, p<0.001 and t(6)=3.9, p<0.01, respectively).
Health Plan Choice
Among study participants who had not chosen a Medicare health plan at baseline, 59 percent of treatment group participants had chosen a plan at followup and could indicate the type of plan chosen, compared to 50 percent of control group participants ([chi square] (1,N=446)=3.3, p<0.05, h=0.173). Effect sizes in Subsamples One (h=0.174) and Two (h--0.170) were virtually equivalent.
Stage Progression and Regression
At baseline, 39 percent of participants who had not chosen a Medicare health plan were in the precontemplation stage for comparing plans, 10 percent were in contemplation, 23 percent preparation, and 28 percent action. Only 30 percent of these participants progressed one or more stages during the 6-month followup period, and 43 percent regressed. For the treatment and control groups, stage progression rates were identical, at 30 percent, and stage regression rates were 41 versus 45 percent, respectively. Neither effect was statistically significant.
Results of Chi Square tests and intervention effect sizes are reported in Table 4. For the separate subsamples, rates of stage progression for the treatment and control groups were 26 versus 28 percent, respectively, in Subsample One, and 32 versus 31 percent in Subsample Two. Rates of stage regression for the treatment and control groups were 41 versus 44 percent in Subsample One and 40 versus 45 percent in Subsample Two. Effect sizes for the two subsamples were not statistically different from each other.
These null A character that is all 0 bits. Also written as "NUL," it is the first character in the ASCII and EBCDIC data codes. In hex, it displays and prints as 00; in decimal, it may appear as a single zero in a chart of codes, but displays and prints as a blank space. findings regarding the effect of the TIM materials on stage progression and regression were surprising, given that the intervention materials are associated with greater increases in Medicare knowledge, a greater likelihood of using traditional Medicare materials, and a greater likelihood of making a health plan choice, all presumably requisites for comparing Medicare health plans. In addition, earlier work found these same dimensions to be related as predicted with stage of change for comparing plans (Levesque et al., 2006).
To confirm the negative findings, we repeated the analyses and included in the treatment group only those participants who were known to have at least examined the intervention materials: that is, those who had returned the acceptability survey (Table 4). Compared to control group participants, individuals who examined the intervention materials showed a non-significant trend toward higher rates of stage progression (45 versus 30 percent, p=0.07), and were significantly less likely to regress REGRESS. Returning; going back opposed to ingress. (q.v.) to an earlier stage of change for comparing health plans (27 versus 45 percent, p<0.05).
For the separate subsamples, rates of stage progression for the treatment and control groups were 33 versus 28 percent, respectively, in Subsample One, and 48 versus 31 percent in Subsample Two. Rates of stage regression were 33 versus 44 percent in Subsample One and 25 versus 45 percent in Subsample Two. Effect sizes for the two subsamples were not statistically different from each other.
While both the treatment and control groups showed significant gains in Medicare knowledge during the study period--which would be expected during a critical period when enrollees were being asked to choose a plan and were being exposed to new information from CMS, health plans, and other sources--knowledge gains were significantly greater among participants who received the TTM manual plus expert system intervention or the manual alone. Among most enrollee groups, the TIM intervention increased use of and satisfaction with traditional Medicare information, so it is possible that at least a portion of the treatment groups' differential gain in knowledge was due to increased use of--or ability to benefit from--existing CMS educational materials. However, among less-educated and less-advantaged enrollees, the TIM manual alone did not increase use of and satisfaction with traditional Medicare materials, but generated significant increases in knowledge nonetheless, perhaps by filling a critical information void. The manual contains a great deal of CMS educational information, a plan comparison chart, and recommended CMS support tools and services, but presents them in a gradual, stage-appropriate, and supportive fashion that may be more likely to engage and less likely to intimidate in·tim·i·date
tr.v. in·tim·i·dat·ed, in·tim·i·dat·ing, in·tim·i·dates
1. To make timid; fill with fear.
2. To coerce or inhibit by or as if by threats. users who may have lower literacy or self-efficacy, or who may have skill deficits.
Among participants who still needed to choose a plan at baseline, we found a discouragingly dis·cour·age
tr.v. dis·cour·aged, dis·cour·ag·ing, dis·cour·ag·es
1. To deprive of confidence, hope, or spirit.
2. To hamper by discouraging; deter.
3. low rate of stage progression (30 percent) and high rate of stage regression (43 percent) during the follow-up period. Although the TIM interventions had a significant positive impact on three important dimensions of informed choice--Medicare knowledge, use of CMS educational materials, and choosing a plan--their impact on stage of change for comparing plans was observed only among treatment group participants who had evaluated them for the acceptability survey. Compared to control group participants, treatment group participants who examined the TIM materials were about 50 percent more likely to progress to a later stage of change for comparing plans, and 60 percent less likely to regress to an earlier stage. Comparing plans, as operationally defined, requires far more than Medicare knowledge and information seeking; it also requires an integrated, sustained effort to see the plan comparison process to the end. Through the operational definition of comparing plans, the TIM expert system and manual outline the steps involved in making an informed choice. With the stage-based guidance and feedback, they provide an individualized roadmap to help users progress through the stages and prevent regression to an earlier stage when the task becomes difficult. The materials are unlikely to achieve these aims if enrollees do not use them, or use them just as a reference guide.
This study has several limitations. First, in the baseline survey, administered an average of 7 weeks before study participants reach age 65, 56 percent reported that they had already chosen a Medicare health plan. As a result, over one-half of the intervention group received their materials too late to be maximally max·i·mal
1. Of, relating to, or consisting of a maximum.
2. Being the greatest or highest possible.
An element in an ordered set that is followed by no other. useful (e.g., 43 percent of study participants who had already chosen a plan were still in a pre-action stage, and thus, had made their choice without carefully weighing their options). We took this into account in some analyses. Ideally, the intervention materials would have been delivered several months earlier, to give users enough time to use them and progress through the stages of change before needing to make a health plan choice.
It is possible that the acceptability survey served as a prompt for new enrollees to use the TTM materials or seek other Medicare information, and thus, may account for a portion of the intervention effect--especially the effect for stage progression and regression observed among individuals who returned the acceptability survey. In real-world applications, new enrollees and beneficiaries may similarly benefit from additional prompts, reminders, and encouragement to use health plan information to make an informed choice. Some disadvantaged This article or section may contain original research or unverified claims.
Please help Wikipedia by adding references. See the for details.
This article has been tagged since September 2007. segments of the
Medicare population were underrepresented un·der·rep·re·sent·ed
Insufficiently or inadequately represented: the underrepresented minority groups, ignored by the government. in the study sample, including enrollees without telephones, and enrollees who could not read or write English. In addition, enrollees who dropped out of the study at followup tended to be less knowledgeable about the Medicare Program, less educated, and more likely to be Hispanic than those who completed the followup survey, limiting the generalizeability of the findings.
With these limitations in mind, results suggest that providing individualized guidance and feedback matched to stage of change for comparing plans could be helpful in increasing new enrollees' Medicare knowledge and participation in informed choice. The intervention materials have been revised to incorporate recent changes to the Medicare Program and the increased complexity of the decisions beneficiaries will need to make under the MMA. The challenge is to identify channels for dissemination dissemination Medtalk The spread of a pernicious process–eg, CA, acute infection Oncology Metastasis, see there . One possibility is to integrate stage-based messages into traditional Medicare materials, or to post the manual on the Medicare.gov Web site. Stage-based materials could also be disseminated to older workers in the workplace well before they reach age 65.
We gratefully acknowledge the contributions and support of David Miranda and Amy Heller. We also wish to thank three anonymous reviewers for their helpful comments.
Bandura, A.: Self-Efficacy: Toward a Unifying Theory of Behavior Change. Psychological Review 84(2) :191-215, March 1997.
Bann, C.M., Terrell, S.A., McCormack, L.A., et al.: Measuring Beneficiary Knowledge of the Medicare Program: A Psychometric psy·cho·met·rics
n. (used with a sing. verb)
The branch of psychology that deals with the design, administration, and interpretation of quantitative tests for the measurement of psychological variables such as intelligence, aptitude, and Analysis. Health Care Financing Review 24(4):111-125, Summer 2003.
Cohen, J.: Statistical Power Analysis for the Behavioral Sciences behavioral sciences,
n.pl those sciences devoted to the study of human and animal behavior. . Second Edition. Erlbaum. Hillsdale, NJ. 1988.
Cronbach, L.J.: Coefficient coefficient /co·ef·fi·cient/ (ko?ah-fish´int)
1. an expression of the change or effect produced by variation in certain factors, or of the ratio between two different quantities.
2. Alpha and the Internal Structure of Tests. Psychometrika 16(3):297-334, 1951.
Janis, I.L. and Mann, L.: Decision Making: A Psychological Analysis of Conflict, Choice and Commitment. Free Press. New York New York, state, United States
New York, Middle Atlantic state of the United States. It is bordered by Vermont, Massachusetts, Connecticut, and the Atlantic Ocean (E), New Jersey and Pennsylvania (S), Lakes Erie and Ontario and the Canadian province of . 1977.
Kaiser Family Foundation: Selected Findings on the Medicare Drug Law. Health Poll Report Survey December 2-5, 2004. Internet address There are two kinds of addresses that are widely used on the Internet. One is a person's e-mail address, and the other is the address of a Web site, which is known as a URL. Following is an explanation of Internet e-mail addresses only. For more on URLs, see URL and Internet domain name. : http://www. kff.org/kaiserpolls/loader.cfm?url=/commonspot/ security/getfile.cfm&PageID=50510 (Accessed 2006.)
Levesque, D./L and Cummins, C.O.: Application of the Transtheoretical Model The transtheoretical model of change in health psychology explains or predicts a person's success or failure in achieving a proposed behavior change, such as developing different habits. It attempts to answer why the change "stuck" or alternatively why the change was not made. to Informed Choice in the Medicare Population: Development of TTMBased Interventions to Increase Informed Choice Among New Enrollees. Final Report to the Centers for Medicare & Medicaid Services. Pro-Change Behavior Systems, Inc. Kingston, RI. 2002.
Levesque, D.A. and Cummins, C.O.: Application of the Transtheoretical Model to Informed Choice in the Medicare Population: Results of a Randomized Trial of Transtheoretical Model-Based Interventions for Informed Choice in the Medicare Population. Phase III Report. Final Report to the Centers for Medicare & Medicaid Services. Pro-Change Behavior Systems, Inc. Kingston, RI. 2004.
Levesque, D.A., Cummins, C.O., Prochaska, J.M., et al.: Stage of Change for Making an Informed Decision about Medicare Health Plans. Health Services Research Health services research is the multidisciplinary field of scientific investigation that studies how social factors, financing systems, organizational structures and processes, health technologies, and personal behaviors affect access to health care, the quality and cost of health care, . 41(4pl):1372-1391, August 2006.
Prochaska, J.O. and DiClemente, C.C.: Stages and Processes of Self-Change of Smoking: Toward an Integrative Model of Change. Journal of Consulting and Clinical Psychology The Journal of Consulting and Clinical Psychology (JCCP) is a bimonthly psychology journal of the American Psychological Association. Its focus is on treatment and prevention in all areas of clinical and clinical-health psychology and especially on topics that appeal to a broad 51(3):390-395, June 1983.
Prochaska, J.O. and DiClemente, C.C.: Common Processes of Change in Smoking, Weight Control, and Psychological Distress psychological distress The end result of factors–eg, psychogenic pain, internal conflicts, and external stress that prevent a person from self-actualization and connecting with 'significant others'. See Humanistic psychology. . In Shiffman, S., and Wills, T. (eds.): Coping and Substance Use: A Conceptual Framework For the concept in aesthetics and art criticism, see .
A conceptual framework is used in research to outline possible courses of action or to present a preferred approach to a system analysis project. . Academic Press. New York, NY. 1985.
Rossi, J.S.: Comparison of the Use of Significance Testing and Effect Sizes in Theory-Based Health Promotion Research. Paper Presented at the 43rd Annual Meeting of the Society for Multivariate Experimental Psychology. Keystone key·stone
1. Architecture The central wedge-shaped stone of an arch that locks its parts together. Also called headstone.
2. The central supporting element of a whole. , CO. September 2003.
Deborah A. Levesque, Ph.D., Carol O. Cummins, M.Ed., M.L.I.S., Janice M. Prochaska, Ph.D., and James O. Prochaska, Ph.D.
Reprint reprint An individually bound copy of an article in a journal or science communication Requests: Deborah A. Levesque, Ph.D., Pro-Change Behavior Systems, Inc., P.O. Box 755, W. Kingston, RI 02892. E-mail: dlevesque @prochange.com
(1) Acceptability survey respondents had higher levels of perceived knowledge than non-respondents, and were less likely to have incomes above $50.000. No other differences on baseline measures or demographics were observed.
Deborah A. Levesque, Carol O. Cummins, and Janice M. Prochaska, are with Pro-Change Behavior Systems, Inc. James O. Prochaska is with the University of Rhode Island History
The University was first chartered as the state's agricultural school in 1888. The site of the school was originally the Oliver Watson Farm, and the original farmhouse still lies on the campus today. . The research in this article was supported by the Centers for Medicare & Medicaid Services (CMS) under Contract Number 500-01-002 (TO6). The statements expressed in this article are those of the authors and do not necessarily reflect the views or policies of Pro-Change Behavior Systems, Inc., University of Rhode Island, or CMS.
Table 1 Control and Treatment Group Characteristics for Subsamples One and Two at Baseline Subsample One Total Control Treatment Characteristic N (n=225) (n=226) Percent Race/Ethnicity White, non-Hispanic 1,116 90.2 87.6 Black, non-Hispanic 92 2.2 3.6 Hispanic 37 2.2 3.1 Other 99 5.4 5.8 Sex Male 584 52.0 44.2 Female 767 48.0 55.8 Marital Status Married 930 76.7 80.8 Widowed 173 9.0 7.6 Divorced/Separated 185 11.2 10.3 Never Married 49 3.1 1.3 Income <$10,000 116 2.5 3.0 $10,000--$19,999 265 17.5 11.2 $20,000--$29,000 214 19.0 14.7 $30,000--$49,999 279 24.5 34.0 $50,000> 277 36.5 37.1 Education Less than High School 211 4.9 4.9 High School or GED 512 30.0 28.4 Some College or 2-Year Degree 378 31.4 38.2 Four-Year College Degree 237 33.6 28.4 Health Status Excellent 200 18.4 20.0 Very good 457 37.2 38.7 Good 454 33.6 28.4 Fair 188 9.0 10.7 Poor 42 1.8 2.2 Eligible for Medicaid Yes 69 1.6 2.2 No 1,019 98.4 97.8 Had Chosen Medicare Health Plan at Baseline Yes 757 67.0 64.2 No 589 33.0 35.8 Stage of Change at Baseline Precontemplation 476 26.0 31.8 Contemplation 84 4.9 4.5 Preparation 180 10.8 13.5 Action 591 58.3 50.2 Mean (SD) Knowledge at Baseline Medicare 1,297 4.4 (1.7) 4.1 (1.8) Perceived 1,342 2.9 (1.1) 3.0 (1.1) Subsample Two Control Treatment Characteristic (n=451) (n=449) Percent Race/Ethnicity White, non-Hispanic 81.7 78.5 Black, non-Hispanic 7.6 10.1 Hispanic 3.1 2.5 Other 7.6 8.9 Sex Male 40.8 40.8 Female 59.2 59.2 Marital Status Married 64.8 65.1 Widowed 16.3 14.3 Divorced/Separated 14.5 16.3 Never Married 4.5 4.3 Income <$10,000 13.9 13.9 $10,000--$19,999 25.8 29.4 $20,000--$29,000 20.0 19.0 $30,000--$49,999 20.5 22.7 $50,000> 19.7 15.0 Education Less than High School 21.2 21.3 High School or GED 42.0 43.7 Some College or 2-Year Degree 26.3 23.5 Four-Year College Degree 10.5 11.5 Health Status Excellent 13.1 12.4 Very good 29.3 35.0 Good 37.6 33.0 Fair 15.8 16.5 Poor 4.2 3.2 Eligible for Medicaid Yes 7.7 9.5 No 92.3 90.5 Had Chosen Medicare Health Plan at Baseline Yes 53.2 49.9 No 46.8 50.1 Stage of Change at Baseline Precontemplation 38.8 39.6 Contemplation 7.7 6.6 Preparation 15.3 13.1 Action 38.1 40.7 Mean (SD) Knowledge at Baseline Medicare 3.8 (1.9) 3.7 (1.8) Perceived 2.6 (1.1) 2.6 (1.1) NOTE: Numbers in parentheses are standard deviations. SOURCE: Making Good Choices about Medicare Health Plans Survey, 2002-2003. Table 2 Rates of Use and Satisfaction with Medicare Information Among Treatment and Control Group Participants Who Did Not Choose a Health Plan Baseline Overall Sample Treatment Control Measure N Percent Percent Read Medicare & You Handbook 589 45.1 38.2 Found Handbook Somewhat or Very Useful 249 85.5 75.7 Used Medicare Web Site 589 10.2 7.4 Found Web Site Somewhat or Very Useful 53 84.4 71.4 Found Medicare Information Understandable 457 57.0 48.2 Satisfied or Very Satisfied with Medicare 439 69.3 63.9 Information Mean Effect Size -- -- -- Overall Sample [chi square] Effect Measure (df=1) Sizes * Read Medicare & You Handbook 2.8 0.140 Found Handbook Somewhat or Very Useful * 3.9 0.250 Used Medicare Web Site 1.5 0.099 Found Web Site Somewhat or Very Useful 1.3 0.316 Found Medicare Information Understandable * 3.5 0.176 Satisfied or Very Satisfied with Medicare 1.4 0.115 Information Mean Effect Size -- 0.183 Subsample One Measure Effects Sizes (1) Read Medicare & You Handbook 0.299 Found Handbook Somewhat or Very Useful 0.336 Used Medicare Web Site 0.313 Found Web Site Somewhat or Very Useful 0.657 Found Medicare Information Understandable 0.357 Satisfied or Very Satisfied with Medicare 0.341 Information Mean Effect Size 0.384 Subsample Two Measure Effect Sizes (1) Read Medicare & You Handbook 0.078 Found Handbook Somewhat or Very Useful 0.219 Used Medicare Web Site -- Found Web Site Somewhat or Very Useful -- Found Medicare Information Understandable 0.108 Satisfied or Very Satisfied with Medicare 0.035 Information Mean Effect Size 0.110 (1) Cohens h. Generally, h=0.20 indicates a small effect, 0.50 a medium effect, and 0.80 a large effect (Cohen, J.: Statistical Power Analysis for the Behavioral Sciences. Second Edition. Erlbaum. Hillsdale, NJ. 1988). However, for large-scale population-based intervention trials, small, medium and large effects can be defined as 10.15, 0.20, and 0.25, respectively (Rossi, J.S.: Comparison of the Use of Significance Testing and Effect Sizes in Theory-Based Health Promotion Research. Paper Presented at the 43rd Annual Meeting of the Society for Multivariate Experimental Psychology. Keystone, CO. September 2003). * p < 0.05. NOTE: Given the under-representation of individuals with computer access in Subsample Two, use of and perceived helpfulness of the Medicare Web site were not examined in that group. SOURCE: Making Good Choices about Medicare Health Plans Final Survey, 2003. Table 3 Rates of Use of and Satisfaction With Medicare Information Among Treatment and Control Group Participants With Two Education Levels Subsample One Measure High School or Less Effect Sizes (1) n Read Medicare & You Handbook 0.107 56 Found Handbook Somewhat or Very Useful 0.284 20 Used Medicare Web Site 0.050 56 Found Web Site Somewhat or Very Useful 1.231 6 Found Medicare Information Understandable 0.110 42 Satisfied or Very Satisfied with Medicare 0.377 40 Information Mean Effect Size 0.360 -- Subsample One Measure Some College Effect Sizes (1) n Read Medicare & You Handbook 0.360 99 Found Handbook Somewhat or Very Useful 0.390 57 Used Medicare Web Site 0.376 99 Found Web Site Somewhat or Very Useful 0.618 31 Found Medicare Information Understandable 0.483 83 Satisfied or Very Satisfied with Medicare 0.309 79 Information Mean Effect Size 0.423 -- Subsample One Measure High School or Less Effect Sizes (1) n Read Medicare & You Handbook 0.045 294 Found Handbook Somewhat or Very Useful 0.102 106 Used Medicare Web Site -- -- Found Web Site Somewhat or Very Useful -- -- Found Medicare Information Understandable 0.040 222 Satisfied or Very Satisfied with Medicare -0.117 212 Information Mean Effect Size 0.018 -- Subsample One Measure Some College Effect Sizes (1) n Read Medicare & You Handbook 0.167 135 Found Handbook Somewhat or Very Useful 0.337 64 Used Medicare Web Site -- -- Found Web Site Somewhat or Very Useful -- -- Found Medicare Information Understandable 0.224 106 Satisfied or Very Satisfied with Medicare 0.350 104 Information Mean Effect Size 0.270 -- (1) Cohen's h. Generally, h=0.20 indicates a small effect, 0.50 a medium effect, and 0.80 a large effect (Cohen, J.: Statistical Power Analysis for the Behavioral Sciences. Second Edition. Erlbaum. Hillsdale, NJ. 1988). However, for large-scale population-based intervention trials, small, medium and large effects can be defined as 1=0.15, 0.20, and 0.25, respectively (Rossi, J.S.: Comparison of the Use of Significance Testing and Effect Sizes in Theory-Based Health Promotion Research. Paper Presented at the 43rd Annual Meeting of the Society for Multivariate Experimental Psychology. Keystone, CO. September 2003). SOURCE: Making Good Choices about Medicare Health Plans Final Survey, 2003. Table 4 Rates of Progression and Regression in Stage of Change for Treatment and Control Group Participants Comparing Medicare Health Plans Overall Sample Treatment Control Measure N Percent Percent Stage Progression 310 30.3 30.3 Stage Regression 288 40.5 45.0 Treatment Group Limited to known Intervention Users Stage Progression 190 44.7 30.3 Stage Regression 173 27.3 45.0 Overall Sample Measure [chi square] (df=1) Effect Sizes (1) Stage Progression 0.0 0.002 Stage Regression 0.6 0.091 Treatment Group Limited to known Intervention Users Stage Progression 2.9 0.299 Stage Regression * 3.5 0.371 Subsample One Subsample Two Measure Effects Sizes (1) Effect Sizes (1) Stage Progression -0.034 0.015 Stage Regression 0.059 0.105 Treatment Group Limited to known Intervention Users Stage Progression 0.069 0.212 Stage Regression 0.301 0.412 (1) Cohen's h. Generally, h=0.20 indicates a small effect, 0.50 a medium effect, and 0.80 a large effect (Cohen, J.: Statistical Power Analysis for the Behavioral Sciences. Second Edition. Erlbaum. Hillsdale, NJ. 1988). However, for large-scale population-based intervention trials, small, medium and large effects can be defined as f=0.15, 0.20, and 0.25, respectively (Rossi, J.S.: Comparison of the Use of Significance Testing and Effect Sizes in Theory-Based Health Promotion Research. Paper Presented at the 43rd Annual Meeting of the Society for Multivariate Experimental Psychology. Keystone, CO. September 2003). SOURCES: Making Good Choices about Medicare Health Plans Survey, 2002-2003, and Making Good Choices about Medicare Health Plans Final Survey, 2003.
|Printer friendly Cite/link Email Feedback|
|Author:||Levesque, Deborah A.; Cummins, Carol O.; Prochaska, Janice M.; Prochaska, James O.|
|Publication:||Health Care Financing Review|
|Date:||Jun 22, 2006|
|Previous Article:||Medicare beneficiary knowledge: measurement implications from a qualitative study.|
|Next Article:||SCHIP structure and children's use of care.|