The Use of Couple Data to Determine the Reliability of Self-Reported Sexual Behavior.

Cultural norms and ethical principles make it highly unlikely that human sexual behavior will ever be studied by direct observation of a representative sample of the general population. As a result, much sex research involves the use of questionnaires and interviews to obtain retrospective participant self-reports of the relevant behaviors. Recently, Lewontin (1995a, 1995b) has criticized the reliance on such data, and has argued that because sexual behavior is not value neutral, individuals will be motivated to lie, both to themselves and to their interviewers. Indeed, it is disturbing to find significant discrepancies in men's and women's reports of the number of their sexual partners (cf. Morris, 1993; Phillis & Gromko, 1985) and in the reporting of participation in oral-genital and anal-genital sexual behavior (Johnson, & Wadsworth, 1994; Laumann, Gagnon, Michael, & Michaels, 1994; Wellings, Field, Messiah, Blin, Fiche, & ACSF group, 1995). For example, the Laumann et al. (1994) data show that for the most recent session more men than women reported anal sex (2.3% vs. 1.2%). Discrepancies like this may be due to a lack of participant honesty (Lewontin, 1995a), the limitations inherent in any retrospective reporting of behavior (Bradburn, Rips, & Shevell, 1987; Downey, Ryan, Roffman, & Kulich, 1995; Wheeler & Reis, 1991), or other systematic differences in how likely men and women are to report specific behaviors (Laumann et al., 1994). Whatever the reasons may be, the feasibility of a scientific basis for the study of sexual behavior is thrown into serious question if highly reliable data cannot be collected. Determining the reliability and validity of self-report data is, therefore, a major concern for all researchers in this area.

A variety of strategies and techniques have been developed to improve the reliability of self-reported sexual behavior (Catania, 1999; Catania, Binson, Van Der Straten, & Stone, 1995; Catania, Gibson, Chitwood, & Coates, 1990; Coates et al., 1986; Coates et al., 1988; Gribble, Miller, Rogers, & Turner, 1999; James, Bignell, & Gillies, 1991; Udry, 1980; Udry & Morris, 1967; Williams, Suen, & Baffi, 1993; Zimmerman & Langer, 1995); however, insufficient attention has been paid to assessing the sexual partner's reports. Purely practical and logistical considerations (e.g., contacting sexual partners would double the number of participants in a study) may be partly responsible for the relative paucity of analyses based on the sexual behavior reporting of both partners. Determining the appropriate methodology to follow in the analysis of the resultant data sets poses yet another challenge. It has been argued that partner based reporting does not "provide information on the validity of self-reported sexual events, ... [and they] still need to be validated against some objective index" (Catania et al., 1995, p. 84). Catania emphasizes the importance of assessing respondent influences on measurement error, test-retest indices, self-presentation bias, and so on. We agree that these factors represent important influences on report reliability, and thus have implications for validity as well. A careful analysis of couple-based self-reports would provide unique reliability data, as well as clarifying whether systematic dishonesty of the kind described by Lewontin (1995a) contributes to report discrepancies.

An early attempt at assessing the reliability of sexual behavior reports made by Kinsey (Kinsey, Pomeroy, & Martin, 1948) revealed high correlations (Pearson's r = .61 -.93) in lifetime incidence statistics for a selection of eight foreplay behaviors and seven intercourse positions reported by 231 pairs of spouses. These participants' responses to very general questions (e.g., "Male above in coitus, has ever occurred": Yes/No) represent a modest starting point in assessing the reliability of reporting sexual behavior. More complex analyses of partner report data have only recently begun to appear (Julien, Bouchard, Gagnon, & Pomerleau, 1992; Seal, 1997). In addition, the onset of the AIDS crisis has encouraged systematic efforts to contact homosexual partners and thereby obtain comprehensive couple-based data on sexual behavior (e.g. Catania et al., 1995; Seage, Mayer, Horsburgh, Cai, & Lamb, 1992). Despite these studies, most research tends to limit its focus to overly simple constructs, such as the frequency of sexual intercourse (Call, Sprecher, & Schwartz, 1995; Clark & Wallin, 1964; Edwards & Booth, 1976; Levinger, 1970; Rao & DeMaris, 1995), or investigates only a very small set of behaviors (James et al., 1991; Seal, 1997; Upchurch et al., 1991). Participants are also typically asked to retrospect over four weeks, a period of time which limits the level of detail that can be obtained about sexual behavior (Coates et al., 1986; Edwards & Booth, 1976; Rao & DeMaris, 1995; Udry, 1980). Some researchers have used one partner's positive report of sexual activity to infer that this activity occurred even when the other partner did not report it or denied it (Call et al., 1995; Rao & DeMaris, 1995).

When couple behavioral data is reported, it tends to be reported in terms of percentage agreement alone or in conjunction with significance testing of Pearson's r (Coates et al., 1988; Hyde, DeLamater, Plant, & Byrd, 1996; Messiah et al., 1995; Seal, 1997; Upchurch, et al., 1991). The oft-utilized percentage agreement index is calculated by adding up the number of consistent responses made by the couple and dividing by the total number of responses made (Appendix A). The major drawbacks of this approach are that it does not take into account the amount of agreement that would be expected to occur simply by chance, and it conflates agreement between couple members concerning the occurrence of a behavior and agreement concerning the nonoccurrence of a behavior. It has been cogently argued for some time (Cohen, 1960) that significance tests are not an appropriate measure of agreement (reliability) because a certain level of agreement between couple members is expected. An index or indices which accurately quantify the extent of agreement between partner reports is required. Correlational measures such as Pearson's r or phi (in the case of dichotomous data) represent an improvement over simple means testing. However, with the Kappa statistic (Cohen, 1960), the proportion of agreement due to chance is explicitly taken into account, and its conceptual grounding is clearly relevant to the task of measuring agreement (cf. Appendix A).

Despite Kappa's advantages over other measures of agreement, it suffers from certain limitations. For instance, when a behavior is reported at a very high frequency by both couple members, Kappa's value may be very low even if only a relatively small proportion of the reports are actually inconsistent (Appendix A). In addition, Kappa (as well as percentage agreement) are limited in that they can not inform the reader about the relative importance of the two ways couples might be consistent in their reporting of a given behavior (agreement on occurrence vs. nonoccurrence). Given that this latter shortcoming is inherent in the use of any single measure of agreement, we considered the advantage of using two conditional probability indices in addition to Kappa.

The Positive Conditional Probability index (CP+) is a measure of the averaged conditional probability that if either couple member has reported a behavior's occurrence, the other one has reported it as well. The Negative Conditional Probability index (CP-) is the averaged conditional probability that if one couple member did not report a behavior, the other did not report it either. Using the paired measures of conditional probability in addition to the Kappa allows one to readily evaluate the degree of agreement on the two possible contributing factors to concordance. Whereas the conceptual grounding of Kappa makes it the best single measure for assessing the reliability of couple-based reports in general, the addition of the conditional probability indices may enhance the interpretation of couple data, especially when evaluating the consistency of high base-rate behaviors. In calculating the CP+ and CP- indices (cf. Appendix A), no adjustment for chance agreement was made, although it would in principle be possible to do so.

We collected couple-based sexual behavior data in the context of a larger study (Ochs & Binik, 1998). Couples were exposed to either an expert computer system which engaged couples in a dialogue about their sexual relationship, bibliotherapy, a human therapist interview, or a control condition. As part of this study, each member of a sexually active, cohabiting couple completed detailed questionnaires concerning 68 different sexual behaviors. A number of steps were taken to increase the likelihood that partners would be able to produce reliable reports of their sexual behavior (cf. Bradburn et al., 1987; Catania et. al, 1995). For example, participant recall was focused on the immediately preceding week, all behaviors were specified in short phrases, and, in the case of intercourse positions, line drawings (available from authors) were used in addition to short descriptions (for related approaches see Coxon et al., 1992; Marks, Cordess, & Verde, 1988). Measures of couple adjustment and sexual comfort were obtained in order to assess factors that have been shown to be related to the consistency of reporting of sexual behavior (Catania et al., 1990).

We predicted that our data concerning couples' sexual behavior would demonstrate high levels of concordance on all four reliability indices used in the analyses. In particular, we expected to demonstrate the feasibility of obtaining highly reliable reports of sexual behaviors that had been previously shown to be significantly discrepant in the literature (oral and anal sex). We also predicted that, despite Kappa's superiority as a measure of reliability compared with simple percentage agreement, Kappa could produce overly conservative values for those behaviors occurring at very high base-rates. The addition of the conditional probability indices to the analysis of reliability was expected to overcome this limitation, as well as to make explicit the relative contribution of couple concordance concerning the occurrence versus the nonoccurrence of the behaviors examined.

METHOD

Participant Recruitment

Our participants were recruited through advertising in campus and community newspapers requesting couples interested in participating in a study to enhance their sexual relationship. It was necessary for participants to be at least 18 years of age, to have English language fluency, and to have been a member of a sexually active, cohabiting heterosexual couple for at least 9 months. A current, planned, or recent pregnancy; medication with potential sexual side effects; previous personal contact with investigators in the study; or any planned major lifestyle disruptions resulted in exclusion from the study. Of the 144 individuals who responded to the advertisements, 119 met the criteria for study participation, and 80 were successfully scheduled for an initial session. Seventy-seven couples completed the entire study protocol and were reimbursed for their participation. Seventy of these couples reported at least one sexual activity session during their participation in the study. The data collected from these 70 sexually active couples was used in the analyses.

Materials

Each participant completed the following questionnaires: Participant Background Information, Dyadic Adjustment Scale (Spanier, 1976), Sexual Comfort Scale, Sexual Session Record, and Sexual Activity Questionnaire.

Participant Background Information. This questionnaire included a series of very brief questions concerning age, sex, occupation, and therapy history.

Dyadic Adjustment Scale (DAS). The DAS gives an overall score of a couple's relationship adjustment by assessing couple satisfaction, consensus, cohesion, and expression of affection. The scale consists of 32 items and questions, which couples individually rate on a scale of 0 (very negative) to 5 (very positive).

Sexual Comfort Scale. Couples were asked to imagine themselves engaged with their partner in the same 68 activities described on the Sexual Activity Questionnaire. They indicated their comfort level with each activity on a separate 9-point Likert-type scale of -4 (very uncomfortable) to +4 (very comfortable).

Sexual Session Record. Couples received instructions to record the following data about all sexual interactions that occurred during the week prior to the first scheduled meeting: (a) its duration, (b) when it occurred, and (c) where it took place. No other information was recorded at that time. This data collection was repeated for the second week of the study. Sexual interaction was defined as "any more or less continuous duration of time spent engaging in some kind of sexual activity with your partner, regardless of the exact nature of the activities involved."

Sexual Activity Questionnaire. For each session of sexual activity indicated on the Sexual Session Record, participants independently indicated the activities they had engaged in. Sexual activity sessions were reported by each respondent, in the order of the most recent to the least recent session. The Sexual Activity Questionnaire comprises a check list of 44 foreplay/erotic activities (e.g. "My partner kissed me on the body"), and 24 line-drawings of penetrative sexual intercourse positions. The Sexual Activity Questionnaire (available from the authors on request) was specifically developed to assess changes in as large a number of differentiable sexual behaviors as feasible (Ochs & Binik, 1998).

Procedure

The first author screened volunteer couples. Couples were instructed on the phone to keep a record together of only the location, the duration, and the date on which they engaged in any sexual activity. During the first lab session the study was explained, and the couples signed a consent form. They then independently completed the Participant Background Information, Sexual Comfort Scale, Dyadic Adjustment Scale, the Sexual Activity Questionnaire, and other questionnaires (cf. Ochs & Binik, 1998). Each couple was then exposed to one of four experimental manipulations of a 45-minute duration: a sex-expert computer program, a sex educational video, an interview with a sex therapist, or a control condition. The total duration of the first session was, on average, 135 minutes. In the second session, each participant again completed the forms used in the first session (except the Participant Background Information), as well as other questionnaires to assess the impact of the interventions (cf. Ochs & Binik, 1998). Finally, couples were debriefed and paid $30 ($15 each) for their participation in the study. The second session required an average of 75 minutes of the couple's time.

RESULTS

Participant Background Information

The 70 participants ranged in age from 18 to 56 years (M = 27, SD = 6.2), and had been living together (70%) or married (30%) for between 9 and 264 months (M = 38, SD = 5.2). Thirty-five of the participants were working or seeking work, 33 were full-time students, and 2 were homemakers.

Sexual Session Record

The number of sessions identified by each couple was used to calculate the average frequency of sexual behavior. These data were subjected to a repeated-measures analysis of variance (ANOVA) with time as the repeated factor. Couples reported an average of 3.17 and 2.64 sessions of sexual activity in weeks 1 and 2 of the study respectively (average M = 2.91, SD = 1.25, Range = 1 to 7, n = 70). This represents a significant decline in the number of sessions reported over time (F(1,69) = 8.79, p [is less than] .005).

Sexual Activity Questionnaire

The 68 behaviors on the Sexual Activity Questionnaire were initially analyzed to determine if any significant changes occurred over time or between genders in the reporting of foreplay or intercourse behaviors. Thus, the 43 foreplay and 25 intercourse behaviors were subjected to separate multivariate analyses of variance (MANOVA), which included time as a repeated factor and gender as a within-case variable (i.e., the couple is treated as the case). The averaged multivariate test of significance demonstrated no significant difference in the frequency of behaviors reported by gender for either foreplay or intercourse behaviors. However, a significant time effect was obtained for the number of foreplay behaviors reported per week (F(27,43) = 3.09, p [is less than] .001). Given the significant decrease in sessions reported over time, we decided to test if the decrease in behaviors was a likely result of the decrease in sessions or a decrease in behaviors per session. Thus, the behaviors reported in the most recent session report were subjected to the same MANOVA described above, with the result that there were no significant time or gender effects. An average of 19.1 (SD = 5.68) behaviors per session were reported in the most recent session of weeks 1 and 2. In addition, the overall number of behaviors per session in weeks 1 and 2 was not significantly different.

Couple Sexual Behavior Report Reliability

We began by analyzing all the data concerning the 68 behaviors collected from each partner for their concordance as measured by 4 reliability indices (Table 1). Using percentage agreement, concordance values ranging from 63% to 99% were obtained for the behaviors reported (mean rating 87%). The percentage agreement values listed in Table 1 reveal that 53 of the 68 behaviors polled attained 80% or greater concordance (e.g. dancing, hug/cuddle, sixty-nine), whereas the 6 least concordant behaviors fell between 63% and 67% agreement (e.g. massage partner, stripping). These results appear to indicate a high level of concordance in the reporting of behavior; however, the lack of a correction for chance agreement (dependent on a behavior's base-rate) may make interpretation and comparisons among percentage agreement values difficult.

Table 1. Participant Reporting of Activities, Type of Agreement, and Reliability

                               Count              Agreement

# Activity                   M       W      Neg.    Pos.    None

Foreplay/erotic activities

 1 Dining                     28      35      365      21      21
 2 Dancing - c                20      19      383      15       9
 3 Wrestle                    51      49      337      30      40
 4 Sexy clothing - m          17      11      385       6      16
 5 Sexy clothing - w          44      53      332      22      53
 6 Erotica                    21      19      381      14      12
 7 Vibrator                   19      17      384      13      10
 8 Tying up partner - m        4       3      402       2       3
 9 Tying up partner - w        1       1      405       0       2
10 Dance for partner - m      26       9      377       5      25
11 Dance for partner - w      20      10      383       6      18
12 Strip self - m            148     124      203      68     136
13 Strip self - w            142     122      208      65     134
14 Undress partner - m       100      97      252      42     113
15 Undress partner - w        97     100      264      31     112
16 Undress each - c          122     101      251      67      89
17 Kissing on lips           368     376       17     354      36
18 Kiss body - m             372     346       17     328      62
19 Kiss body - w             332     354       26     305      76
20 Hug/cuddle - c            380     380       10     363      34
21 Hug partner - m           355     378        9     335      63
22 Hug partner - w           344     379       10     326      71
23 Caress partner - m        363     366       15     337      55
24 Caress partner - w        344     359       22     318      67
25 Massage partner - m       223     188      130     134     143
26 Massage partner - w       214     185      139     131     137
27 Bite/lick partner - m     315     278       56     242     109
28 Bite/lick partner - w     267     267       97     224      86
29 Cunnilingus               151     148      226     118      63
30 Fellatio                  190     206      174     163      70
31 Sixty nine                 45      48      347      33      27
32 Masturb. self - m          49      58      332      32      43
33 Masturb. self - w          65      69      314      41      52
34 Masturb. partner - m      223     226      131     173     103
35 Masturb. partner - w      216     234      123     166     118
36 Masturb. each other       122     128      239      82      86
37 Mast. partner II - m      213     174      140     120     147
38 Mast. partner II - w      175     188      150     106     151
39 Shower - c                 78      73      306      50      51
40 Food - m                    5       3      400       1       6
41 Food - w                    5       2      401       1       5
42 Rimming partner - m        27      14      375       9      23
43 Rimming partner - w         9      12      392       6       9

Sexual intercourse positions

44 Anal sex                   22      17      385      17       5
45 Missionary 1              245     221      140     199      68
46 Missionary 2               80      98      271      42      94
47 Missionary 3              204     209      162     168      77
48 Man on top 4               80      61      299      33      75
49 Woman on top 1            108     108      264      73      70
50 Woman on top 2             38      48      340      19      48
51 Woman on top 3             25      20      369       7      31
52 Woman on top 4            166     177      205     141      61
53 Woman on top 5            120      99      256      68      83
54 Woman on top 6             10      15      385       3      19
55 Woman on top 7             30      19      371      13      23
56 Woman on top 8             30      22      371      16      20
57 Doggy style 1             121     100      261      75      71
58 Doggy style 2              18      12      385       8      14
59 Half rear entry            18      10      384       5      18
60 Spooning                   64      73      313      43      51
61 Standing 1                 14      17      385       9      13
62 Standing 2                 16      12      387       8      12
63 Standing 3                 13      10      390       6      11
64 Standing 4                 14      20      383      10      14
65 X position 1               39      50      341      23      43
66 X position 2               18      17      383      11      13
67 Edge of bed                15      15      385       8      14
68 Sideways                   36      43      347      19      41

   Unweighted average        116     113      266      87      54
   Total reporting         7,854   7,702   18,072   5,929   3,675

                              Reliability Index

# Activity                  %    Kappa   CP+   CP-

Foreplay/erotic activities

 1 Dining                  95%    .64    .67   .97
 2 Dancing - c             98%    .76    .77   .99
 3 Wrestle                 90%    .54    .60   .94
 4 Sexy clothing - m       96%    .41    .43   .98
 5 Sexy clothing - w       87%    .38    .45   .93
 6 Erotica                 97%    .68    .70   .98
 7 Vibrator                98%    .71    .72   .99
 8 Tying up partner - m    99%    .57    .57   .99
 9 Tying up partner - w    99%    .00    .00   .99
10 Dance for partner - m   94%    .26    .29   .97
11 Dance for partner - w   96%    .38    .40   .98
12 Strip self - m          67%    .25    .50   .75
13 Strip self - w          67%    .25    .49   .76
14 Undress partner - m     72%    .24    .43   .82
15 Undress partner - w     72%    .25    .31   .86
16 Undress each - c        78%    .45    .60   .85
17 Kissing on lips         91%    .44    .95   .49
18 Kiss body - m           85%    .27    .91   .35
19 Kiss body - w           81%    .30    .89   .41
20 Hug/cuddle - c          92%    .33    .96   .37
21 Hug partner - m         85%    .14    .91   .22
22 Hug partner - w         83%    .14    .90   .22
23 Caress partner - m      86%    .28    .92   .35
24 Caress partner - w      84%    .30    .90   .40
25 Massage partner - m     65%    .30    .65   .65
26 Massage partner - w     66%    .33    .66   .67
27 Bite/lick partner - m   73%    .33    .82   .51
28 Bite/lick partner - w   79%    .53    .84   .69
29 Cunnilingus             85%    .67    .79   .88
30 Fellatio                83%    .66    .82   .83
31 Sixty nine              93%    .67    .71   .96
32 Masturb. self - m       89%    .54    .60   .94
33 Masturb. self - w       87%    .54    .61   .92
34 Masturb. partner - m    75%    .49    .77   .72
35 Masturb. partner - w    71%    .41    .74   .68
36 Masturb. each other     79%    .50    .66   .85
37 Mast. partner II - m    64%    .28    .62   .66
38 Mast. partner II - w    63%    .25    .58   .67
39 Shower - c              87%    .59    .66   .92
40 Food - m                99%    .24    .25   .99
41 Food - w                99%    .28    .29   .99
42 Rimming partner - m     94%    .41    .44   .97
43 Rimming partner - w     98%    .56    .57   .99

Sexual intercourse positions

44 Anal sex                99%    .87    .87   .99
45 Missionary 1            83%    .66    .85   .80
46 Missionary 2            77%    .33    .47   .85
47 Missionary 3            81%    .62    .81   .81
48 Man on top 4            82%    .36    .47   .89
49 Woman on top 1          83%    .56    .68   .88
50 Woman on top 2          88%    .38    .44   .93
51 Woman on top 3          92%    .27    .31   .96
52 Woman on top 4          85%    .69    .82   .87
53 Woman on top 5          80%    .48    .62   .86
54 Woman on top 6          95%    .22    .24   .98
55 Woman on top 7          94%    .50    .53   .97
56 Woman on top 8          95%    .59    .62   .97
57 Doggy style 1           83%    .56    .68   .88
58 Doggy style 2           97%    .52    .53   .98
59 Half rear entry         96%    .34    .36   .98
60 Spooning                87%    .55    .63   .92
61 Standing 1              97%    .56    .58   .98
62 Standing 2              97%    .56    .57   .98
63 Standing 3              97%    .51    .52   .99
64 Standing 4              97%    .57    .59   .98
65 X position 1            89%    .46    .52   .94
66 X position 2            97%    .61    .63   .98
67 Edge of bed             97%    .52    .53   .98
68 Sideways                90%    .43    .48   .94

   Unweighted average      87%    .45    .61   .83
   Total reporting         87%    .67    .76   .91

Note. Suffixes are used where necessary in the activity descriptions to indicate which couple member is the active agent: m = man active, w = woman active, c = both active. Agreement equals a total of 407 session reports (Negative + Positive + None) made by couples. M and W equals the counts of the reports of the behavior's occurrence by men and women respectively. Neg. equals the total number of sessions in which both partners agreed the behavior did not occur. Pos. equals the total number of sessions in which both partners agreed the behavior did occur. None equals the total number of sessions in which just one member of the couple reported the behavior. % equals Percentage Agreement. CP+ equals Positive Conditional Probability: the probability that if one couple member reports a behavior, the other member does as well. CP- equals Negative Conditional Probability: the probability that if one couple member does not report a behavior, the other member does not either.

The corresponding Kappa values for the 68 behaviors ranged from 0 to .87. The mean Kappa for this data set may be calculated by two different methods. The first mean Kappa is obtained by summing all individually calculated Kappa values and then dividing by the total number of behaviors to obtain an unweighted average. This method of calculation ignores the different frequency of each behavior, as well as the impact of behavioral frequency on its associated Kappa. The mean Kappa obtained by this method may be overly influenced by the impact of the low Kappa values associated with high frequency behaviors, in addition to the disproportionate impact of very low frequency behaviors. In order to overcome these problems the mean Kappa was also derived from the combined reporting of all 68 behaviors (i.e., all behavior reports of agreement and disagreement were totaled prior to calculating the overall Kappa). These two calculation methods result in mean Kappa values of .45 (unweighted) and .67 (based on total behaviors). The data in Table 1 also show that individual Kappa values can range widely, especially in the case of high (e.g., hug partner) and low (e.g., tying up partner) frequency behaviors.

For the 68 behaviors the values of the CP+ reliability index ranged from 0 to .96, and the values of the CP- index ranged from .22 to .99. The alternate treatment of data described above for calculating two mean Kappa values also resulted in different mean CP+ and CP- values. The results were .61 and .76 for CP+, and .83 and .91 for CP-, using unweighted and total behaviors respectively.

Given that the foreplay and intercourse behaviors were defined differently (text versus text with drawings respectively), and that this might have implications for the reliability of subsequent reporting, the data were evaluated separately. Table 2 presents couple-report data and the derived reliability indices for foreplay (1-43), intercourse (44-68), and total (1-68) behaviors for weeks 1 and 2 of the study. Two facts were readily apparent from these data: the previously noted decrease in numbers of behaviors reported, and the substantial degree of stability of the four agreement indices over time. Percentage agreement remained virtually unchanged over time. A comparison of each of the 68 individual behaviors showed that only 10 behaviors registered a change of greater than 5%: Of these, only 1 was in the direction of less agreement over time (woman masturbating herself, -5 %). The total Kappa index also remained virtually unchanged between week 1 and 2. However, of the 68 behaviors, 12 exhibited a change in Kappa of greater than .30 between time 1 and 2. Note that, with one exception (woman masturbates herself), all of these behaviors occurred at a base-rate of less than 6%. Thus, it appears that the individual Kappa values are more variable than the percentage agreement values, especially when sample size or frequency is very low. The mean CP+ and CP- indices also remained relatively unchanged from Week 1 to Week 2. Of the 68 behaviors, 16 exhibited a change of greater than .20 on the CP+ index. On the CP- index, none of the 68 behaviors exhibited a change in value of greater than. 18.

Table 2. A Summary of Behavior Data for Weeks 1 and 2

                    Count             Agreement

Category          M       W      Neg.    Pos.     Non

                                Week 1

Foreplay        3,431   3,376    5,392   2,639   1,515
Intercourse       901     891    4,363     605     582
All behaviors   4,332   4,267    9,755   3,244   2,097

                                Week 2

Foreplay        2,879   2,833    4,518   2,266   1,171
Intercourse       643     602    3,799     419     407
All behaviors   3,522   3,435    8,317   2,685   1,578

                            Total Weeks 1 & 2

All behaviors   7,854   7,702   18,072   5,929   3,675

                   Reliability Index

Category         %    Kappa   CP+   CP-

                        Week 1

Foreplay        84%    .65    .78   .88
Intercourse     90%    .61    .68   .94
All behaviors   86%    .66    .75   .90

                        Week 2

Foreplay        85%    .68    .79   .89
Intercourse     91%    .62    .67   .95
All behaviors   87%    .69    .77   .91

                   Total Weeks 1 & 2

All behaviors   87%    .67    .76   .91

Note. Columns M and W represent the total number of reports made of each behavior category by men and women respectively.

Given that the number of sessions reported by each couple varied from 1 to 7 per week (M = 2.9), a number of ways exist to analyze the resulting data set (note that Table 1 only depicts the results obtained from using the entire data set). We assessed report-reliability independently of the distorting effects of the varying number of couple session reports and the errors of reporting behaviors in the incorrect session. To achieve this, the complete set of session reports (n = 222 and 185 for weeks 1 and 2 respectively) were collapsed into incidence data (n of 70) for each week of the study. By recalculating the agreement indices based on whether or not a behavior had occurred during a given week, the reliability of reporting behavioral incidence is emphasized (as opposed to the reliability of temporal reporting). The reliability indices obtained in this fashion were highly similar to those derived from the complete data set. In addition, the reports made by couples concerning only the most recent session of sexual activity (n of 70) were assessed for concordance. Note that this analysis also eliminates the differential impact of varying numbers of session reports. Once again, the reliability indices were highly similar to those based on the full data set.

Behavior Comfort Ratings and the Reliability Measures

The 68 separate ratings of comfort were collapsed into one overall rating for each couple member. These data were analyzed with a 2 (Gender) x 2 (Week) multivariate analysis of variance (MANOVA). Only the gender effect was significant, (F(1,69) = 4.59, p [is less than] .05), with men rating themselves as experiencing significantly higher levels of comfort than women (7.7 vs. 7.4 on the 10-point scale). Follow-up univariate analyses of variance on the 68 individual behaviors revealed that women reported 3 behaviors as significantly more comfortable than men, whereas men reported 24 as significantly more comfortable than women (see Table 3). The 3 behaviors which women rated as more comfortable (cuddling together, giving partner a hug, and dancing) were among the least explicitly sexual of all the behaviors polled. In contrast, men rated the following behaviors as much more comfortable (p [is less than] .0001): fellatio, anal intercourse, woman-on-top intercourse, and vaginal rear-entry intercourse. Couple members' overall comfort ratings were significantly positively correlated with one another (Pearson's r = .36 and .43 at time 1 and 2 respectively).

Table 3. Behaviors For Which Significant Differences in the Average Level of Comfort Were Reported

                               Comfort rating

#    Activity                Men   Women     F

20   Hug/cuddle - c          8.6    8.8    .050 w
22   Hug partner - w         8.6    8.8    .025 w
52   Woman on top 4          8.7    8.3    .019
49   Woman on top 1          8.5    8.2    .020
57   Doggy style 1           8.6    8.0    .003
15   Undress partner - w     8.4    8.1    .014
53   Woman on top 5          8.5    8.0    .034
30   Fellatio                8.5    7.6    .0001
 2   Dancing - c             7.7    8.3    .005 w
63   Standing 3              8.2    7.7    .030
60   Spooning                8.1    7.7    .046
13   Strip self - w          8.2    7.5    .002
48   Man on top 4            8.2    7.5    .007
 5   Sexy clothing - w       8.0    7.3    .008
58   Doggy style 2           8.0    7.2    .004
31   Sixty nine              7.8    7.0    .003
59   Half rear entry         7.8    6.9    .0001
55   Woman on top 7          7.8    6.7    .0001
64   Standing 4              7.8    6.6    .0001
54   Woman on top 6          7.7    6.6    .0001
56   Woman on top 8          7.5    6.7    .013
11   Dance for partner - w   7.0    5.9    .001
33   Masturb. self - w       6.9    5.9    .005
 7   Vibrator                6.1    5.1    .002
44   Anal sex                4.8    3.4    .0001
42   Rimming partner - m     4.5    3.6    .019
43   Rimming partner - w     4.6    3.5    .006

Note. Sexual behaviors are listed in descending order of average couple comfort ratings. Sexual intercourse positions (#45 through #68) are defined by one square inch size line-drawings in addition to the written descriptions used (however, item #44 uses a written description only). Comfort is rated on a scale from 0 (extreme discomfort) to 9 (completely comfortable). The univariate F-tests have df(1,69). In the F column of the table, the letter "w" indicates that womens' comfort is significantly greater than mens' comfort; for all other comparisons mens' comfort is significantly greater.

In order to produce independent evidence of the validity of the data set and to make an assessment of the relative performance of the reliability indices, Pearson's correlations were calculated between sexual comfort and the reliability indices. To reduce the chance that unanticipated artifacts in our data set might distort these relationships, the correlations were based on the same 3 subsets of the data as used in the analysis of overall reliability described above. The average of the partner's comfort ratings for each behavior was used in the calculation of the correlation between couple comfort and reliability. The results of these analyses (Table 4) show that the Kappa index demonstrated a significant positive correlation with sexual comfort (average Pearson's r = .38) in each of the 3 data subsets, whereas the percentage agreement index did not. The CP+ index also demonstrated a significant correlational relationship with comfort in each of the 3 data subsets (average Pearson's r = .49), whereas its complementary index CP- did not.

Table 4. The Relationship of the Reliability Measures with Sexual Comfort and Dyadic Adjustment

                                     Reliability Index

                               %    Kappa      CP+        CP-

Sexual Comfort Ratings

All behavior reports          .08   .40(***)   .50(***)   -.04
collected (n = 407)

Weekly incidence(a) of        .11   .34(**)    .48(***)   -.05
behaviors (n = 70)

Most recent behavior report   .11   .41(***)   .49(***)   -.01
only (n = 70)

Dyadic Adjustment Scale

All behavior reports          .10   .27(*)     .31(**)     .04
collected (n = 407)

Weekly incidence(a) of        .16   .26(*)     .32(**)     .06
behaviors (n = 70)

Most recent behavior report   .18   .34(**)    .35(**)     .13
only (n = 70)

Note. Pearson's r was calculated with the averaged reliability indexes for the complete data set and two subsets of the data.

(a) If a participant reported a behavior in one or more sessions, it was only counted once.

(*) p < .05.

(**) p < .01.

(***) p < .001.

Couple Adjustment and Report Reliability

A comparison of men's and women's ratings of their own dyadic adjustment revealed no significant differences (paired t-test: t(69) = .31, p =.76); therefore, all subsequent findings relating to the Dyadic Adjustment Scale are reported using averaged couple scores. Couple Dyadic Adjustment Scale ratings ranged from 71 to 139 (M = 112, SD = 13.8). Twelve couples reported ratings below 100, which indicates a significant degree of marital distress (Crane, Allgood, Larson, & Griffin, 1990).

As another independent check on the validity of the data set, the relationship between couples' dyadic adjustment and the concordance of their sexual behavior reporting was assessed by calculating Pearson's correlations as described above. The results of these analyses (see Table 4) show a similar pattern of findings to those found with comfort; in particular, higher dyadic adjustment was correlated with greater reliability as measured by Kappa.

DISCUSSION

Data collected independently from relatively young, well-adjusted, and sexually-active members of couples, concerning 68 different sexual behaviors, demonstrated a high degree of concordance based on the 4 reliability indices we used. These results were obtained whether the data were analyzed as a whole or in parts, and whether or not participants felt comfortable about performing these behaviors or not. Although the number of sessions reported decreased significantly from Week 1 to Week 2, this did not result in changes in the reliability of the behaviors reported within sessions. However, the four reliability indices we used do have different advantages and disadvantages.

Using the simple percentage agreement index, an average of 87% concordance was obtained. Although this index is the most intuitive and easy to compute, it does not take into account agreement that results from chance, and it conflates agreement arising from behavioral occurrence and nonoccurrence. In addition, we found that dyadic adjustment and sexual comfort were positively correlated with the concordance of partner reports as measured by Kappa but not by percentage agreement. These significant relationships demonstrate that the correction for chance made by the Kappa index is essential.

For these reasons, Kappa is generally viewed as a better reliability index than percentage agreement. Landis and Koch (1977) have suggested that a Kappa value of 0 to .20 equals slight agreement, .21 to .40 equals fair agreement, .41 to .60 equals moderate agreement, .61 to .80 equals substantial agreement, and .81 to 1.00 equals almost perfect agreement. For most of the 68 behaviors studied, moderate to substantial agreement was obtained on the Kappa index. The low Kappa values obtained appear to be related to those behaviors which occurred at high or low base-rates (e.g. tying up partner and hugging partner). This probably results from two different factors: the difficulty of accurately remembering the occurrence of a frequent or regularly occurring behavior like hugging a partner (Loftus & Loftus, 1980), and the overly conservative nature of the Kappa index when dealing with behaviors that occur at high and low base-rates. For example, in reporting a woman tying up her partner there is 99% agreement but a Kappa of 0. Therefore, special care should be taken when interpreting the Kappa index of behaviors with very high or very low base-rates.

We calculated an average Kappa across all 68 behaviors in two different ways: (a) by adding individual Kappa values together and dividing by the total number to produce an unweighted average, and (b) by adding together all the behavior reports which were in agreement prior to calculating an overall Kappa. These two calculation methods resulted in substantially different Kappa values: .45 and .67 (see Table 1). We believe that the use of the unweighted calculation method significantly underestimates the overall Kappa due to the disproportionate influence of the very high and very low base-rate behaviors on the total Kappa.

The conditional probability indices may be useful when it is necessary to calculate the relative importance of agreement concerning the absence or presence of certain behaviors; neither Kappa nor percentage agreement can provide this information by themselves. In our data set, couples were somewhat more reliable when reporting the absence rather than the presence of a given behavior (.91 vs. .77 respectively). The two conditional probability indices provide insight into the large discrepancies which sometimes occur between percentage agreement and Kappa. In the example mentioned above, concerning a woman tying up her partner, the conditional probability indices make it clear that couples agree nearly perfectly about sessions in which neither reports this behavior (CP- = .99), but never agree when one partner reports the behavior (CP+ = .00).

It should be noted that the average conditional probability indices can be calculated in a weighted or unweighted fashion as was done for Kappa (see Table 1). The conditional probability indices we used were not corrected for chance. In principle such a correction could be implemented, but it seems more straightforward to use the uncorrected conditional probabilities in conjunction with Kappa.

A comparison of our sexual behavior data with that collected in the previous large-scale surveys of the ACSF Group (1993), Laumann et. al, (1994), and Wellings et al., (1994), shows roughly similar frequencies of comparably defined behaviors, and a slight difference in the overall frequency of sexual sessions. For example, in Laumann et al.'s cross-sectional sample the average frequency of sexual activity for participants aged 25 - 39 was reported as 7.6/month and for cohabiting couples was reported as 8.7/month (roughly equivalent to 1.9/week and 2.2/week respectively), compared with 2.9/wk in our study sample. These differences in overall frequency of reporting of sexual sessions may arise from the different time frames sampled, differences in methodology (e.g., interview vs. questionnaire), differences in definition of terms, and differences in the people who volunteer for sex studies.

The aforementioned differences in the reporting of sexual-session frequency are of relatively minor importance in comparison with the significant gender-discrepant reporting evidenced in the literature. For example, a number of major studies have demonstrated that men report a significantly higher incidence of oral (27% vs. 20%) and anal (2.3% vs. 1.2%) sex than do women (Laumann et al., 1994). In contrast to these significant differences in the reporting of sexual behavior, our data-gathering protocols have not generated any evidence of consistent discrepancies in the reporting of 68 different sexual behaviors. When comparable data concerning only the most recent data were analyzed, we found that behavior rates were often identical.

Study Limitations

One possible but unlikely confound in our data is that we asked couples to keep a record together of the time, duration, and place of their sexual behavior. We did this to facilitate accurate reporting. Couples did not initially know that they would be asked independently and in detail about the sexual behaviors that took place during these sexual sessions. It is possible that couples discussed the 68 different behaviors that we asked them about. However, the lack of substantial reliability differences between week 1 (when they did not know they would be asked) and week 2 (when they did know) suggests that this was not a significant factor. Nonetheless, the fact that the couples were asked to keep a record of when and where their sexual behavior took place probably increased the reliability of their reporting, and it made it impossible to address the reported gender discrepancy in the frequency of sexual behavior sessions.

Our participants volunteered for a study offering to teach them about their sex lives, and they were given the opportunity to think about their sexual behavior, in detail, in the laboratory. It is therefore unlikely that they constitute a representative sample of couples of their age. They may, in fact, have been more motivated than a representative sample to attend to the details of their sexual life and to report them accurately (cf. Wiederman, 1999). It is also likely that asking a representative group of couples to recall their sexual behavior over longer periods of time would result in lower reliability.

The fact that a minority of items obtained poor concordance (e.g., three pairs of foreplay behaviors which were defined specifically in terms of one partner doing something to the other) suggests that further experimental testing of alternate forms and wordings may be necessary to clarify ambiguous definitions. It also remains to be seen whether other adjustments to the methodology of data collection could reduce the impact that monitoring of sexual behavior appears to have on the frequency of sexual behavior.

Conclusion

Future studies should attempt to confirm our results and investigate the limits to the reliability of couple-based reporting. In particular, such studies should attempt to investigate gender discrepancies in the recollection of the frequency of sexual behavior. This could easily be done with our current methodology, by not asking couples to keep an advance record of sexual sessions. Survey research in general should avoid the exclusive use of percentage agreement as the sole measure of reliability. In general, Kappa is a more appropriate measure of agreement than percentage agreement, and it can be supplemented by the addition of the two conditional probability indices.

The methodology we have described has potentially important applications. In survey research it would seem very useful to interview a subset of the partners of individuals surveyed, to determine the reliability of the data collected and if gender discrepancies exist. In the clinical realm, our procedures could be easily adapted to collect reliable information concerning couples' sexual repertoires. For cases of sexual dysfunction, this is crucial for assessment and for evaluation of treatment outcome.

REFERENCES

ACSF Group: Spira, A., Bajos, N., Bejin, A., Beltzer, N., Bozon, M., Ducot, B., Durandeau, A., Ferrand, A., Giami, A., Giraud, M., Gilloire, A., Leridon, H., Ludwig, D., Messiah, A., Moatti, J., Mounier, L., Olomucki, H., Poplavsky, J., Riandey, B., Spencer, B., Sztalryd, J., Touzard, H., Colomby, P., Firdion, J., Le Pont, F., Pelletier, A., & Warszawski, J. (1993). Les comportements sexuels en France: Rapport au ministre de la recherche et de l'espace. [Sexual Behavior in France: Report to the Ministry of Research and Space]. Paris: La documentation Francaise.

Bradburn, N. M., Rips, L. J., & Shevell, S. K. (1987). Answering autobiographical questions: The impact of memory and inference on surveys. Science, 236, 157-161.

Brennan, P. F., & Hays, B. J. (1992). The kappa statistic for establishing interrater reliability in the secondary analysis of qualitative clinical data. Research in Nursing & Health, 15, 153-158.

Call, V., Sprecher, S., & Schwartz, P. (1995). The incidence and frequency of marital sex in a national sample. Journal of Marriage and the Family, 57, 639-650.

Catania, J. A. (1999). A framework for conceptualizing reporting bias and its antecedents in interviews assessing human sexuality. The Journal of Sex Research, 36, 25-38.

Catania, J. A., Binson, D., Van Der Straten, A., & Stone, V. (1995). Methodological research on sexual behavior in the AIDS era. Annual Review of Sex Research, 6, 77-125.

Catania, J. A., Gibson, D. R., Chitwood, D. D., & Coates, T. J. (1990). Methodological problems in AIDS behavioral research: Influences on measurement error and participation bias in studies of sexual behavior. Psychological Bulletin, 108, 339-362.

Clark, A., & Wallin, P. (1964). The accuracy of husbands' and wives' reports of the frequency of marital coitus. Population Studies, 18, 165-173.

Coates, R. A., Calzavara, V. M., Soskolne, C. L., Read, S. E., Fanning, M. M., Shepherd, F. A., Klein, M. H., & Johnson, J. K. (1988). Validity of sexual histories in a prospective study of male sexual contacts of men with AIDS or an AIDS-related condition. American Journal of Epidemiology, 128, 719-728.

Coates, R. A., Soskolne, C. L., Calzavara, L., Read, S. E., Fanning, M. M., Shepherd, F. A., Klein, M. M., & Johnson, J. K. (1986). The reliability of

sexual histories in AIDS-related research: Evaluation of an interview-administered questionnaire. Canadian Journal Of Public Health, 77, 343-348.

Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20, 37-46.

Coxon, A. P., Davies, P. M., Hunt, A. J., Weatherburn, P., McManus, T. J., & Rees, C. (1992). The structure of sexual behavior. The Journal of Sex Research, 29, 61-83.

Crane, D. R., Allgood, S. M., Larson, J. H., & Griffin, W. (1990). Assessing marital quality with distressed and nondistressed couples: A comparison and equivalency table for three frequently used measures. Journal of Marriage and the Family, 52, 87-93.

Downey, L., Ryan, R., Roffman, R., & Kulich, M. (1995). How could I forget? Inaccurate memories of sexually intimate moments. The Journal of Sex Research, 32, 177-191.

Edwards, J. N., & Booth, A. (1976). Sexual behavior in and out of marriage: An assessment of correlates. Journal of Marriage and the Family, 38, 73-81.

Gribble, J. N., Miller, H. G., Rogers, S. M., & Turner, C. F. (1999). Interview mode and measurement of sexual behaviors: Methodological issues. The Journal of Sex Research, 36, 16-24.

Hyde, J. S., DeLamater, J. D., Plant, E. A., & Byrd, J. M. (1996). Sexuality during pregnancy and the year postpartum. The Journal of Sex Research, 33, 143-151.

James, N. J., Bignell, C. J., & Gillies, P. A. (1991). The reliability of self-reported sexual behavior. AIDS, 5, 333-336.

Julian, D., Bouchard, C., Gagnon, M., & Pomerleau, A. (1992). Insider's views of marital sex: A dyadic analysis. The Journal of Sex Research, 29, 343-360.

Kinsey, A. C., Pomeroy, W. B., & Martin, C. E. (1948). Sexual behavior in the human male. Philadelphia: W.B. Saunders Company.

Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33, 159-174.

Laumann, E. O., Gagnon, J. H., Michael, R. T., & Michaels, S. (1994). The social organization of sexuality: Sexual practices in the United States. Chicago: The University of Chicago Press.

Levinger, G. (1970). Husbands' and wives' estimates of coital frequency. Medical Aspects of Human Sexuality, 4, 42-57.

Lewontin, R. C. (1995a, April 20). Sex, lies, and social science. The New York Review, pp. 24-29.

Lewontin, R. C. (1995b, May 25). Sex, lies, and social science. The New York Review, pp. 43-44.

Loftus, E. F., & Loftus, G. R. (1980). On the permanence of stored information in the human brain. American Psychologist, 35, 409-420.

Marks, I. M., Cordess, C., & Verde, F. (1988). A notation for sexual activity. The Journal of Sex Research, 25, 555-563.

Messiah, A., Blin, P., Fiche, V., & ACSF Group. (1995). Sexual repertoires of heterosexuals: Implications for HIV/sexually transmitted disease risk and prevention. AIDS, 9, 1357-1365.

Morris, M. (1993). Telling tails explain the discrepancy in sexual partner reports. Nature, 365, 437-440.

Ochs, E. P., & Binik, Y. M. (1998). A sex-expert computer system helps couples learn more about their sexual relationship. Journal of Sex Education and Therapy, 23, 145-155.

Phillis, D. E., & Gromko, M. H. (1985). Sex differences in sexual activity: Reality or illusion? The Journal of Sex Research, 24, 437-443.

Rao, K. V., & DeMaris, A. (1995). Coital frequency among married and cohabiting couples in the U.S. Journal of Biosocial Science, 27, 135-150.

Seage, G. R., Mayer, K. H., Horsburgh, C. R., Cai, B., & Lamb, G. A. (1992). Corroboration of sexual histories among male homosexual couples. American Journal of Epidemiology, 135, 79-84.

Seal, D. W. (1997). Interpartner concordance of self-reported sexual behavior among college dating couples. The Journal of Sex Research, 34, 39-55.

Spanier, G. B. (1976). Measuring dyadic adjustment: New scales for assessing the quality of marriage and similar dyads. Journal of Marriage and the Family, 38, 15-28.

Udry, J. R. (1980). Changes in the frequency of marital intercourse from panel data. Archives of Sexual Behavior, 9, 319-325.

Udry, J. R., & Morris, N. (1967). A method for validation of reported sexual data. Journal of Marriage and the Family, 442-446.

Upchurch, D. M., Weisman, C. S., Shepherd, M., Brookmeyer, R., Fox, R., Celentano, D. D., Colletta, L., & Hook, E. W. (1991). Interpartner reliability of reporting of recent sexual behaviors. American Journal of Epidemiology, 134, 1159-1165.

Wellings, K., Field, J., Johnson, A., & Wadsworth, J. (1994). Sexual behaviour in Britain: The national survey of sexual attitudes and lifestyles. London: Penguin Books.

Wheeler, L., & Reis, H. T. (1991). Self-recording of everyday life events: Origins, types, and uses. Journal of Personality, 59, 339-354.

Wiederman, M. W. (1999). VolunTeer bias in sexuality research using college student participants. The Journal of Sex Research, 36, 59-66.

Williams, B. L., Suen, H. K., & Baffi, C. R. (1993). A controlled randomized response technique. Evaluation and the Health Professions, 16, 225-245.

Zimmerman, R. S., & Langer, L. M. (1995). Improving estimates of prevalence rates of sensitive behaviors: The randomized lists technique and consideration of self-reported honesty. The Journal of Sex Research, 32, 107-117.

Manuscript accepted June 25, 1999

APPENDIX A

The Reliability Indices

Note the following abbreviations which are used in the calculations that follow. The formulas defined below are applicable with dichotomous data:

M          The number of mens' reports of the occurrence of a given
           behavior.

W          The number of womens' reports of the occurrence of a
           given behavior.

Positive   The number of reports in which both partners agree the
           behavior occurred.

Negative   The number of reports in which both partners agree the
           behavior did not occur.

None       The number of reports in which partners disagree whether
           the behavior occurred.

n          The total number session reports used in the analysis.
           Note that each session report consists of a man's and a
           woman's report; therefore, n = Positive + Negative +
           None.

Percent agreement represents the proportion of the total number of reports made in which both partners are in agreement. It is calculated by adding up all reports in which either both partners agree that the behavior occurred or that it did not occur, and dividing by the total number of reports made. This method gives equal weight to negative-negative and positive-positive reports. All reports made by only one member of the couple are by definition inconsistent. Under an assumption of completely unrelated reporting by men and women, we would expect to obtain a value of 50% agreement. Values above 50% would indicate agreement greater than chance, while lower values would indicate increasing disagreement. Thus, perfect disagreement would produce a value of 0%, and perfect agreement a value of 100%.

% Agreement = (Positive + Negative)/n

Kappa has been defined as the "proportion of agreement after chance agreement is removed from consideration" (Cohen, 1960, p. 40). The proportion of agreement has already been defined above as percentage agreement. The incidence of behavior reporting by men and women is used to calculate the levels of agreement expected to occur simply by chance. First calculate the probability that the behavior is reported just by chance by both partners:

a = (M/n) * (W/n)

Then calculate the probability that the behavior is not reported just by chance by either partner:

b = ((n-M)/n) * ((n-W)/n)

The proportion of chance agreement (Pc) may then be calculated by adding together a and b:

Pc = [(M/n)*(W/n)] + [((n-M)/n)*((n-W)/n)]

Kappa is derived by subtracting chance agreement (Pc) from percentage agreement, and dividing by the proportion of agreement that remains after chance agreement is removed from consideration:

Kappa = (% Agreement - Pc)/(1-Pc)

To illustrate the difference between percentage agreement and Kappa, take the case of a man and a woman who have reported that a behavior occurred during 90% of all reports made. The formula for percentage agreement would produce a value of 82% agreement which is considered reflective of a high level of reliability. However, the formula for Kappa applied to the same data would result in a value of zero (no agreement at all beyond chance). The discrepancy between the two indices tends to be the most extreme in the case of a high frequency of behavioral reporting (or a low rate of reporting). Brennan & Hays (1992) have noted that the Kappa index, by treating a large proportion of consistent reports as inconsequential, may result in an overly conservative estimate of report reliability.

The Positive Conditional Probability Index (CP+) is defined as the proportion of all reports made in which both couple members have agreed that a given behavior did occur. Since the value of the single variable Positive represents a report by a man and a woman, it must be multiplied by two when compared with the number of positive reports made by men and by women. Given that the total number of individual reports of a behavior's occurrence is simply M + W, the CP+ index is derived as follows:

CP+ = 2(Positive)/(M+W)

The Negative Conditional Probability Index (CP-) is similar to CP+, but it is based on couple members agreement that a given behavior did not occur. The number of reports made by men in which the behavior is reported as not having occurred is (n - M) and the corresponding number for women is (n - W). Since the value of the single variable Negative represents a report by a man and a woman, it must be multiplied by two when compared with the number of negative reports made by men and by women. Therefore, the CP- index is derived as follows:

CP- = 2(Negative)/(n-M) + (n-W)

In most situations, the two measures of conditional probability should be used together, since each index only tells half the story of couple agreement.

The following algebraic relationships and variable definitions may be of use in understanding the measures and indices presented in this appendix and in the paper:

% Agreement = [(CP+)(M + W)] + [(CP-)(2n - (M + W))]/2n

Kappa = (CP+) + (CP-) - 1 (M + W) = [(None) + 2(Positive)]

Many thanks to T. Fleischer, E. Stelzer, and D. Beauchamp for their assistance in collecting the data, and to M. Mendelson, R. Amsel, D. Moskowitz, and two reviewers for their editorial and statistical advice. E. Ochs was supported by a Natural Sciences and Engineering Research Council (NSERC) graduate fellowship; this paper is in partial fulfillment of the requirements for a Ph.D. in clinical psychology at McGill University. The research was also supported by a Health Canada and a Medical Research Council of Canada grant to Y. M. Binik.

Address correspondence to Dr. Y. M. Binik, McGill University, Department of Psychology, 1205 Dr. Penfield Ave., Montreal, Quebec, Canada, H3A 1B1; e-mail: binik @ego.psych.mcgill.ca.

Eric P. Ochs McGill University Montreal, Quebec, Canada

Yitzchak M. Binik McGill University and The Royal Victoria Hospital Montreal, Quebec, Canada