Printer Friendly

An Examination of Uncivil and Reasoned Comments and Perceived Civility in Politics.

A normative theory of political communication posits that deliberative political discussions are central to a healthy democracy (e.g., Althaus, 2012; Dahlberg, 2001; Delli Carpini, Cook, & Jacobs, 2004; Freelon, 2015). Such discussions are open, welcoming, and respectful in nature and are intended for mutual comprehension (e.g., Jamieson & Hardy, 2012; Moy & Gastil, 2006; Stromer-Galley, 2007). Whereas online media enable citizens to freely discuss political issues and have the potential to facilitate political deliberation, not all online comments foster this ideal. In fact, a rich body of research finds that the prevalence of incivility online, such as name-calling and stereotyping, may hinder deliberative discussions (e.g., Coe, Kenski, & Rains, 2014; Herbst, 2010; Rowe, 2015; Sobieraj & Berry, 2011).

Online deliberation scholars have examined the implications of uncivil online political discourse. For example, previous research has shown that exposure to online incivility fosters political polarization (Anderson, Brossard, Scheufele, Xenos, & Ladwig, 2014; Hwang, Kim, & Huh, 2014; Y. Kim & Kim, 2019), promotes use of incivility (Cicchirillo, Hmielowski, & Hutchens, 2015; Gervais, 2014; Han & Brazeal, 2015), and decreases willingness to read others' comments online (Y. Kim & Kim, 2019). Although uncivil comments that unnecessarily disrespect, label, and attack others derail the focus of a discussion and undermine citizen engagement (McClurg, 2006; Moy & Gastil, 2006), differences are inherent in politics and are not necessarily harmful to the democratic process. To the extent that citizens exchange views in a civil, reasoned manner, online political comments may promote a sense of civility and motivate political participation. Research indeed suggests that reason-based opinion exchanges online can facilitate deliberation and active political engagement (Han & Brazeal, 2015; Hwang et al., 2014; Min, 2007).

The goal of our study is to investigate exposure to two types of online political comments, uncivil and reasoned messages, in relation to perceived civility in society and off-line and online political participation. While previous research has extensively examined the political implications of online incivility (e.g., Chen, 2017; Coe et al., 2014; Gervais, 2014, 2015; Rowe, 2015; Sobieraj & Berry, 2011), the role of reason-based online comments in political participation has received relatively little attention (e.g., Valenzuela, Kim, & Gil de Zuhiga, 2012). Drawing from recent research that makes a theoretical connection between online discourse and perceived (in)civility (Kenski, Coe, & Rains, 2017), we assess how exposure to the two types of comments are related to political participation through perceived civility in society. It is useful to examine and clarify the interplay among these variables given the centrality of civility and reasoning in the political process amid the prevalence of incivility in political and news discourse.

Civility and Incivility in Online Discussion

Several normative approaches exist for the assessment of online political communication (Freelon, 2015). Althaus (2012) argues that normative approaches should be chosen for their applicability to the investigation of the theoretical or social concerns relevant to the study at hand. The online deliberation normative approach is used in this study given the increasingly contentious and reportedly uncivil nature of the current political landscape in the United States (Barabak & Duara, 2016; Heyward, 2016) alongside the Internet's role in political discussion during the election season (Duggan & Smith, 2016).

Civility is a key concept in the normative tradition of political deliberation (e.g., Dahlberg, 2001; Delli Carpini et al., 2004; Gastil, Deess, & Weiser, 2002; Min, 2007). It refers to interpersonal behaviors that show mutual respect (Andersson & Pearson, 1999; Brooks & Geer, 2007) and that conform to the norms of politeness (Jamieson, Volinsky, Weitz, & Kenski, 2017). Because the meaning of mutual respect and politeness can vary across cultures (Benson, 2011), it is difficult to offer a settled definition of civility. But Jamieson and associates (2017) note that civility "connotes a discourse that does not silence or derogate alternative views but instead evinces respect" (p. 206). Thus, civility is viewed as a central component of deliberation that presupposes respect for and affirmation of all individuals and viewpoints, even in the face of differences and contention (Stryker, Conway, & Danielson, 2016).

Based on the general definition of civility as mutual respect, a group of researchers conceptualize incivility as behaviors violating the norms of politeness (Herbst, 2010; Mutz, 2015). This widely adopted conceptual approach applies rude or impolite interpersonal interactions to define incivility. For example, Coe and colleagues' (2014) analysis of comments posted on a local newspaper's website seems to have relied on this approach to identify forms of incivility that were "something unnecessary" (p. 660, emphasis in original) to the exchange, including name-calling, aspersion, lying, vulgarity, and pejorative speech. Other scholars note that incivility is not equal to impoliteness. For example, Papacharissi (2004) holds that civil forms of online political discussions often contain heated and passionate discussions that involve nonpolite manners. For Papacharissi, incivility in political discussions "demonstrate[s] offensive behavior toward social groups that their behavior becomes undemocratic" (p. 267), such as posing a threat to a democracy, denying an opponent a right to speak, and interrupting the exchange of opinions in a discussion. Muddiman (2017) synthesized these two approaches to define incivility as personal (e.g., name-calling, insults) and public forms (e.g., lack of compromise, misinformation).

Incivility is a multidimensional notion that encompasses both interactional and deliberative components. A study by Stryker and associates (2016) adopts a multidimensional approach to determining what acts are or are not considered a form of civility. Confirmatory analysis of survey data revealed three distinct dimensions of political incivility. Utterance incivility reflects impolite behaviors, such as name-calling, insults, slurs, personal attacks, and vulgarity. Discursive incivility reflects behaviors that prevent open and reasoned debate, such as interrupting people with different opinions in a discussion, refusing to let these people participate in a discussion, and not listening to a different opinion. Deception involves making exaggerating statements, failing to use evidence to support one's opinions, and intentionally making false statements.

The news media facilitate the notion that incivility is a mainstay of the contemporary political landscape, with conflict portrayed as an intrinsic element of politics. While research indicates that incivility is common in traditional and partisan news (Cappella & Jamieson, 1997; Funk, 2001; Mutz, 2015; Sobieraj & Berry, 2011), the uncivil nature of political discourse is particularly prominent online (e.g., Chen, 2017; Coe et al., 2014; Rowe, 2015; Sobieraj & Berry, 2011). For example, Coe and colleagues (2014) found that 22% of the comments posted to newspaper articles contained elements of incivility, and approximately 50% of unique commenters posted at least one uncivil comment.

Research has investigated the effects of exposure to uncivil political messages on political outcomes. For example, Gervais (2014) found that use of uncivil news programs and talk radio increased the likelihood of using incivility when expressing political opinions online. Gervais (2015) also found that exposure to a disagreeable uncivil political post attacking a subject's in-group evoked aversion, which reduced satisfaction with, and consideration of, the uncivil post. Studies also have revealed that exposure to incivility reduces political trust and efficacy (Borah, 2013; Mutz & Reeves, 2005), open-mindedness (Borah, 2014; Hwang, Kim, & Kim, 2018), and willingness to read others' comments online (Y. Kim & Kim, 2019). Exposure to incivility has also been found to promote the expression of disagreements toward the other side (Hwang et al., 2018).

Reasoning in Online Political Discussion

According to theories of deliberative democracy, open, inclusive, and respectful exchanges of ideas cultivate citizenship competence that is fundamental to the democratic process (e.g., Dahlberg, 2001; Delli Carpini et al., 2004). One normative feature of political deliberation is the use of logic and evidence (Jamieson & Hardy, 2012). A reasoned argument is "one in which assertions are grounded in empirically verifiable evidence or in shared understanding of moral or normative behavior" (Stromer-Galley, 2007, p. 4). Reasoned political discourse facilitates the clarification of areas of agreement and disagreement, the collective understanding of issues, and the openness to views different from one's own (Jamieson & Hardy, 2012; Mutz, 2006). Moy and Gastil (2006) note that political deliberation is "characterized by an openness to conflict, non-dominance, the use of clear and logical argument, and mutual comprehension" (p. 444).

Political conversations in everyday settings do not necessarily conform to strictly defined requirements for deliberation. Political conversations might simultaneously exhibit deliberative qualities and violate deliberative norms. For example, a person may present a reasoned argument, yet criticize or disagree with an opposing point of view with rude language. Analyzing online comments posted in response to the White House's Facebook posts and tweets, Oz, Zheng, and Chen (2018) find that, in certain cases, online comments carried high levels of not only incivility (e.g., threatening others' rights) and impoliteness (e.g., insulting others) but also deliberative reasoning (e.g., posting a link to supportive materials, providing a reasoned argument, presenting statistics to support an argument), particularly when users were discussing sensitive subjects. This finding indicates that reasoning and incivility are not necessarily mutually exclusive and that they are not at opposite ends of a continuum.

Research indicates that reasoning in political discussion fosters political participation (J. Kim, Wyatt, & Katz, 1999; Moy & Gastil, 2006; Stromer-Galley & Muhlberger, 2009; Valenzuela et al., 2012). For instance, J. Kim and associates (1999) found that argument quality, or the extent to which people provided reasons when arguing with others, was positively related to campaign participation through political talk. Valenzuela and colleagues (2012) examined discussion with people who use evidence to back up arguments and with people who propose alternative ideas for problem solving to measure reason-based discussion; they found that reasoning discussion was positively associated with online political participation.

Linking Uncivil and Reasoned Comments With Perceived Civility

Rather than exploring perceptions of the relative civility of online political comments, the present study examines perceptions of civility at the societal level in relation to exposure to uncivil and reasoned online political comments. Although Kenski and associates (2017) investigated perceived civility, their emphasis was on how people perceived different types of uncivil online discourse, not the extent to which people perceive civil behaviors toward one another in society at large. Perceptions of civility have been widely studied in group and organizational settings (Andersson & Pearson, 1999). In political contexts, this concept, as noted above, is closely tied to deliberation that presupposes mutual respect (Gastil, Black, & Moscovitz, 2008; Stryker et al., 2016).

How might exposure to uncivil and reasoned online political comments relate to perceived civility in society? This link can be explained by Shrum's (2009) cognitive accessibility model. The model explains an underlying process by which media exposure shapes audiences' judgments about social reality. According to Shrum, media exposure increases the accessibility of information stored in one's memory. Accessibility refers to the ease with which one can recall information from memory. The model also assumes that individuals rely on a small subset of information available in memory, instead of searching the full range of information. In other words, individuals use availability heuristics--easily retrievable, relevant information--to make reality judgments.

A salient factor that affects the accessibility of information is how often one is exposed to a construct of the world (Busselle & Shrum, 2003; Shrum, 2009). When individuals are frequently exposed to a certain construct of reality, the construct becomes activated, stored, and reinforced, and the construct readily comes to mind and influences their judgments about the world (Shrum, 2009). If exposure to a construct increases the accessibility of information in memory, and if people rely on such information to construct reality judgments, then all else being equal, frequent exposure to uncivil and reasoned online political comments, respectively, should be related to perceived civility in society. Prior experimental research supports this proposition. Exposure to the uncivil exchange of comments online has been found to have a negative effect on one's expectations of how deliberative a public discussion is likely to be, whereas the effect of exposure to the civil exchange of comments online was found to be positive (Hwang et al., 2014). Accordingly, we formulate the following hypotheses:

H1: Exposure to uncivil online political comments will be negatively associated with perceived civility in society.

H2: Exposure to reasoned online political comments will be positively associated with perceived civility in society.

Linking Incivility and Reasoning With Political Participation

Political participation consists of acts intended to influence government action "either directly by affecting the making or implementation of public policy or indirectly by influencing the selection of people who make those policies" (Verba, Schlozman, & Brady, 1995, p. 38). It concerns direct forms of engagement in politics such as voting and volunteering for campaigns. It further encompasses such involvement as contacting a local media outlet to express opinions, attending town hall meetings, and convincing others to vote (Verba et al., 1995).

Political participation conventionally takes place in off-line, face-to-face settings. However, the growth and versatility of digital, social, and mobile media have made it possible for citizens to engage in political acts solely on the Internet. For example, one can reach out to a newspaper editor via e-mail or through the newspaper's website, or send a message to a political candidate via Facebook or Twitter. Also, one can make a contribution directly to a political campaign on a candidate's website, and even create and sign an online petition via such websites as We the People ( and without face-to-face physical interaction. As online forms of political participation have become increasingly common, they warrant unique empirical attention (Valenzuela et al., 2012).

Although relatively little research has attempted to find a direct relationship between perceived civility in society and political participation, several lines of research suggest the plausibility of such a link. First, research has found that participation in deliberative discussions promotes political learning, efficacy, and participation (Fishkin & Luskin, 1999; Gastil et al., 2002; Luskin, Fishkin, & Jowell, 2002; Min, 2007). For example, Gastil and colleagues (2002) found that citizens who served on a criminal jury and had a conclusive deliberative experience were more likely to vote in subsequent elections than those who did not reach a verdict. Min (2007) found that people who participated in a discussion of concealed handgun carry on a campus characterized by democratic qualities, such as mutual respect, were more knowledgeable, self-efficacious, and willing to take action regarding the issue. Deliberation yields positive political effects because experiencing a democratic form of discussion in one occasion may cultivate the belief that such an ideal can take place in other occasions and that one can communicatively work with other citizens for the realization of a common goal (Gastil et al., 2002).

Second, organizational researchers have examined workplace civility in terms of its potential to enhance social relationships among employees (e.g., Andersson & Pearson, 1999; Leiter, Laschinger, Day, & Oore, 2011; Pearson, Andersson, & Wegner, 2001). Positive social relationships have been found to enhance engagement in the workplace. For instance, Leiter and associates (2011) proposed that acts of civility may inspire reciprocal civility, which then results in a positive spiral that may enhance "employees' commitment to the workplace and the organization as an environment in which they may fulfill their sense of belonging" (p. 1259). The researchers found that exposure to civility in the workplace enhanced trust and organizational commitment while reducing emotional exhaustion, cynicism, and withdrawal behaviors.

Pearson and colleagues (2001), on the other hand, found that uncivil behaviors, if left unchecked, spread to members of an organization beyond an instigator and target of incivility; foster a deteriorating climate that tolerates rudeness, aggression, and expression of resentment; and promote withdrawal behaviors such as departure from an organization or low performance. The role of (un)civil climates in people's willingness to participate in prosocial engagement may extend to the realm of politics, such as in cases where incivility may impede free and inclusive political discussion (Dutton, 1996; Johnson & Johnson, 2000). In a related vein, the original conceptualization of the spiral of silence theory suggests that citizens are more likely to engage in climates that they perceive as supportive of, rather than hostile--or, arguably, uncivil--to their opinions (Noelle-Neumann, 1993).

Based on this line of research, perceived civility in society might be positively related to political participation. The perception of a civil social environment in which people treat one another with respect and value differences may foster the sense that one can work with others in the political process even in the presence of different opinions, possibly making people more open to politics and more willing to be involved in it. Conversely, if people perceive members of society as unnecessarily disrespectful and self-serving, they might choose not to be actively involved in politics, because it would not be a productive and satisfying experience. Thus, we formulate the following hypotheses:

H3: Perceived civility in society will be positively associated with (a) off-line political participation and (b) online political participation.

H4: Exposure to uncivil online political comments will have indirect negative associations with (a) offline political participation and (b) online political participation through perceived civility in society.

H5: Exposure to reasoned online political comments will have indirect positive associations with (a) online political participation and (b) online political participation through its positive association with perceived civility in society.

The above hypotheses imply a mediation linkage at work, with perceived civility in society as a mediator between exposure to uncivil and reasoned political comments and off-line and online political participation. We expect exposure to uncivil online comments to be negatively associated with political participation, because frequent exposure to such comments is likely related to perceived incivility in society. In contrast, we expect that exposure to reasoned online political comments will be positively related to political participation through perceived civility in society. A theorized model is presented in Figure 1.

Although incivility might foster political participation, this possibility seems particularly plausible when incivility is examined with an in-group and out-group distinction (Brooks & Geer, 2007; Gervais, 2017; Herbst, 2010). For example, Herbst (2010) notes that uncivil messages attacking one's in-group function to mobilize group members against their political opponent. Gervais (2017) finds that people who are exposed to uncivil messages attacking their partisan in-group feel anger, an emotion associated with political participation (Valentino, Brader, Groenendyk, Gregorowicz, & Hutchings, 2011). However, our study does not make such a distinction. Our goal is to examine how the two types of online political comments might be related to people's perceptions of civility in society at large and off-line and online forms of political participation through such perceptions.


Data for this study came from two online surveys of panel members provided by Qualtrics, a private online survey software provider. The first survey was conducted in March 2015, with a total of 516 completed surveys. The first survey was skewed heavily toward young individuals (M = 24.77, SD = 3.20) and women (74.4%), as can be expected of data from online opt-in panels (Berinsky, Huber, & Lenz, 2012). To address this issue, we conducted the second survey in October and November 2016, shortly before the 2016 U.S. presidential election, and received a total of 800 completed surveys. The age, gender, and race distribution of the sample was matched with that of the general population of U.S. adults according to the 2010 census: age (M = 47.28, SD = 16.08), gender (female = 51.4%), and race (White = 64.4%). It must be noted that samples based on online panels of opt-in participants do not represent the American public in general, as every member of the population does not have an equal chance of being included in a sample. As such, results from this study should be interpreted with caution. At the same time, however, replicating results with two separate samples reduces the likelihood of making spurious conclusions.


In Study 1, eight items were used to measure off-line political participation (e.g., Hardy & Scheufele, 2005). Respondents were asked, on a 5-point scale ranging from 1 (never) to 5 (very often), how often in the past 12 months they had engaged in each of the following activities off-line: attended a meeting related to politics; wrote a letter to a newspaper editor or called into a public affairs radio talk show; circulated a petition for a candidate or issue; worked for a political campaign; contacted a public official or political party; called other people to raise funds for a political organization; contributed money to a political organization or candidate; and tried to persuade others to engage in politics. Responses were summed and averaged on a 5-piont sale. In Study 2, respondents were asked, on a 5-point scale ranging from 1 (never) to 5 (very often), how often they engaged in each of the following seven activities off-line: attended a political meeting, rally, or speech; wore a campaign button, hat, or T-shirt; displayed a campaign bumper sticker or yard sign; worked for a political party or candidate; circulated a petition for a candidate or issue; contributed money to a campaign; and tried to persuade others to vote. Responses were averaged on a 5-point scale (see Appendix Table A1 for descriptive statistics of all variables). (1)

In Study 1, five items were used to measure online political participation (e.g., Gil de Zuniga, Veentra, Vraga, & Shah, 2010). Respondents were asked, on a 5-point scale ranging from 1 (never) to 5 (very often), how often in the past 12 months they had engaged in each of the following activities online: sent an e-mail to express opinions to an editor of a news media outlet; sent an e-mail to express opinions to a politician; signed an online petition; made a campaign contribution online; and tried to persuade others to engage in politics online. Responses to these items were combined to form an additive scale. In Study 2, six items were used to measure online political participation. Respondents were asked, on a 5-point scale ranging from 1 (never) to 5 (very often), how often they engaged in each of the following political activities online: sent an e-mail to an editor of a news organization; used e-mail to contact a politician; signed an online petition; made a campaign contribution; volunteered for a campaign; and tried to persuade others to vote. Responses were averaged on a 5-point scale.

Based on past studies (J. Kim et al., 1999; Moy & Gastil, 2006; Valenzuela et al., 2012), five items were created to measure exposure to reasoned online political comments. In Study 1, respondents were asked to indicate, on a 5-point scale ranging from 1 (never) to 5 (very often), how often they encountered each of the following political comments online: comments related to politics that use statistical evidence to back up an argument; comments related to politics that use anecdotal/personal stories to back up an argument; comments related to politics that post a link to an original source of evidence to back up an argument; comments related to politics that propose an alternative policy for problem solving; and comments related to politics that acknowledge opposing political views. Respondents were averaged on a 5-point scale. In Study 2, the question wording was modified to "election-related comments" instead of "comments related to politics" to take into account the election-focused nature of the survey.

Using previous research as a conceptual guide (e.g., Coe et al., 2014; Papacharissi, 2004), several items were developed to measure exposure to uncivil online political comments. In Study 1, respondents were asked to rate, on a 5-point scale ranging from 1 (never) to 5 (very often), how often they encountered each of the following types of political comments online: offensive or vulgar language that insults the president, politicians, or political parties; racial, sexist, ethnic, or religious slurs targeted at the president, politicians, or political parties; comments that assign stereotypes to the president, politicians, or political parties; offensive or vulgar language that insults other users or their political views; racial, sexist, ethnic, or religious slurs targeted at other users; and comments that assign stereotypes to other users or their political views. Responses were averaged on a 5-point scale. In Study 2, respondents were asked, on the same 5-point scale, how often they encountered user comments online that insult a candidate with offensive or vulgar language; insult a candidate with name-calling; insult a candidate with a racial, sexist, ethnic, or religious slur; insult other users with offensive or vulgar language; insult other users with name-calling; and insult other users with a racial, sexist, ethnic, or religious slur. Responses were averaged on a 5-point scale.

Based on civility and deliberation scales used in previous research (Gastil et al., 2008; Osatuke, Moore, Ward, Dyrenforth, & Belton, 2009), five items were adapted to measure perceived civility in society. In Study 1, respondents were asked to rate, on a 7-point Likert-type scale ranging from 1 (strongly disagree) to 7 (strongly agree), the extent to which they agreed or disagreed with each of the following statements: People treat each other with respect in this country; a spirit of cooperation and teamwork exists in this country; disputes or conflicts are resolved fairly in this country; differences among individuals are respected and valued in this country; and people of different backgrounds work well in this country. Responses were averaged on a 7-point scale. The same five items were asked in Study 2 on a 5-point Likert-type scale ranging from 1 (strongly disagree) to 5 (strongly agree). Responses were averaged on a 5-point scale.

Several variables were statistically controlled to separate out the effects of potential correlates with the dependent variables. Age was measured on a ratio scale. Gender was coded with woman as the high value. Race was coded with White as the high value. Education was measured on an 8-point scale ranging from less than high school to postgraduate or professional degree (Mdn = some college, no degree). Income was measured in Study 2 on a 10-point scale ranging from under $10,000 to $200,000 or more. Political interest was measured on a 7-point scale ranging from 1 (not interested at all) to 7 (very interested; M = 4.10, SD = 1.86). Party affiliation was based on a 7-point scale ranging from 1 (strong Republican) to 7 (strong Democrat). Political ideology was measured on a 7-point scale ranging from 1 (very conservative) to 7 (very liberal). Internal political efficacy was measured with four 7-point (Study 1) and 5-point (Study 2) Likert-type scale items from the work of Niemi, Craig, and Mattei (1991), with higher scores indicating higher internal political efficacy. We also controlled for traditional media use. In Study 1, respondents were asked, on a 7-point scale ranging from 1 (no attention) to 7 (a lot of attention), how much attention they paid to each of the following to learn about politics: network television news, print newspapers, public radio off-line, and public radio online. In Study 2, the same six items were used with a 5-point scale.

Analytic Strategy

We conducted path analysis to test the proposed model. The focal variables were residualized on the control variables first (see residualized covariance matrices in Appendix Table A2). We estimated a residual covariance of off-line and online political participation, because unexplained variance of these variables might share a common antecedent not accounted for in the present model. The analysis was conducted using the R package "lavaan" (Rosseel, 2012). The significance of indirect relationships was determined by 95% bootstrap percentile confidence intervals (Hayes, 2019). An indirect relationship was determined to be significant if confidence intervals did not contain 0.


The model did not fit the data particularly well in a global sense both in Study 1 ([chi square] = 39.092, df = 4, p = .000, comparative fit index [CFI] = .951, standardized root mean square residual [SRMR] = .097, root mean square error of approximation [RMSEA] = .140) and in Study 2 ([chi square] = 36.438, df = 4, p = .000, CFI = .960, SRMR = .092, RMSEA = .117). In both Study 1 and Study 2, modification indexes suggested substantial chi-square decrements if the direct paths from exposure to reasoned online political comments to off-line and online political participation were freed. Based on theories of deliberation and past evidence that reason-based political discourse is a salient conduit to political participation (e.g., J. Kim et al., 1999; Moy & Gastil, 2006; Stromer-Galley, 2007; Valenzuela et al., 2012), we allowed the paths to vary freely. This revised model fit the data well in both Study 1 ([chi square] = 2.991, df = 2, p = .224, CFI = .999, SRMR = .020, RMSEA = 0.033) and Study 2 ([chi square] = .028, df = 2, p = .986, CFI = 1.000, SRMR = .002, RMSEA = .000; see Figure 2).

The relationships presented in Figure 2 show that exposure to uncivil online political comments was negatively related to perceived civility in society (p = -.153, p < .01 for Study 1, and p = -.262, p < .001 for Study 2), whereas exposure to reasoned online political comments had a positive link with perceived civility in society (p = .179, p < .001 for Study 1, and p = .128, p < .01 for Study 2). Perceived civility in society was positively associated with off-line political participation (p = .312, p < .001 for Study 1, and p = .166, p < .001 for Study 2) and online political participation (p = .269, p < .001 for Study 1, and p = .097, p < .015 for Study 2). The standardized coefficient of the residual covariance of off-line and online political participation was .806 (p < .001) for Study 1 and .775 (p < .001) for Study 2.

Turning to the hypothesized indirect associations, exposure to uncivil online political comments had a negative indirect relationship with off-line and online political participation both in Study 1 (off-line political participation: point estimate = -.040, 95% confidence interval (CI) [-.069, -.012], standardized coefficient = -.048; online political participation: point estimate = -.034, 95% CI [-.058, -.009], standardized coefficient = -.041) and Study 2 (off-line political participation: point estimate = -.026, 95% CI [-.041, -.011], standardized coefficient = -.041; off-line political participation: point estimate = -.014, 95% CI [-.027, -.002], standardized coefficient = -.025). Exposure to reasoned online political comments had a positive indirect association with off-line and online political participation in Study 1 (off-line political participation: point estimate = .063, 95% CI [.024, .103], standardized coefficient = .056; online political participation: point estimate = .053, 95% CI [.019, .087], standardized coefficient = .048). In Study 2, exposure to reasoned online political comments had a positive indirect link with off-line political participation (point estimate = .017, 95% CI [.003, .032], standardized coefficient = .021), yet its indirect relationship with online political participation was not statistically significant (point estimate = .005, 95% CI [-.001, .019], standardized coefficient = .012). On the whole, the data provide support for all the hypotheses except H5b.


A few conclusions can be drawn from the results of this study. First, exposure to uncivil online political comments was negatively related to perceived civility in society. People who read political comments that were disrespectful to politicians and other citizens tended to feel that people in general demonstrated little regard for one another. In contrast, exposure to reasoned online political comments was related to the perception that people showed regard for other individuals. These results seem consistent with the cognitive accessibility model (Shrum, 2009). As applied to the present context, the model suggests that respondents who frequently encountered uncivil and reasoned online comments, respectively, were more likely to recall specific instances of those comments and use those instances when they judge the civility of the larger society.

Yet an alternative explanation is also plausible. As a consequence of frequently consuming uncivil and reasoned political comments online, people may construct general prototypical notions representing uncivil and reasoned comments instead of concrete exemplary comments they have seen before (Shrum, 2009). As this process unfolds, frequent exposure makes such notions become more accessible. The present study did not assess which mechanism might be at work, yet both scenarios are likely to be observed with increased exposure to uncivil and reasoned online political comments. Although the cognitive accessibility model is typically used to explain the effects of television exposure on audiences' perceptions of reality, particularly in the context of cultivation theory (Shrum, 2009), our study suggests that the basic proposition linking frequent exposure to messages with reality perceptions can be applied to other contexts.

Second, perceived civility mediated the link between uncivil online political comments and political participation off-line and online. That is, exposure to uncivil online political comments had a negative relationship with off-line and online political participation through the perception that people in society are not respectful of one another. It is likely that those who perceived a lack of respect in society at large feel that taking part in politics, which necessarily involves an exchange of different opinions, is more unpleasant than it might actually be and that people involved in politics attack and insult opponents and are unwilling to cooperate with one another for the common good. Such perceptions and political participation were negatively related. This role of uncivil online political comments is in line with previous studies reporting negative effects of online incivility, such as increased intentions to engage in uncivil discourse online (e.g., Cicchirillo et al., 2015) and fostering polarized understandings of an issue (e.g., Anderson et al., 2014).

In contrast, our data reveal an indirect link between reasoned political comments online and political participation through civility perceptions. The finding seems to be in line with the ideal of political deliberation. Although some scholars suggest that what constitutes incivility is context-dependent and that incivility can be socially beneficial (Chen, Muddiman, Wilner, Pariser, & Stroud, 2019), theories of political deliberation posit that a reason-based, civil exchange among citizens is key to a healthy democracy (e.g., Delli Carpini et al., 2004). When citizens discuss issues of concern with mutual respect, they can learn alternative views, work out differences, and find common ground for mutual benefits (Gastil et al., 2002; Moy & Gastil, 2006). Without mutual respect, discussions may become unnecessarily disagreeable and self-serving. Participants may impose their personal views and refuse to listen to opposing arguments and evidence, thereby failing to reach a meaningful conclusion (Gastil et al., 2002; Moy & Gastil, 2006). As an extension of research on deliberation (e.g., Hwang et al., 2014; Moy & Gastil, 2006) and perceived (in)civility (Kenski et al., 2017), we provide support for the democratic role of perceived civility in society and also find that perceived civility in society is related to reason-based political comments people post online, regardless of how frequently they encountered uncivil political comments.

We acknowledge a few limitations that hamper definitive conclusions. First, the data came from nonprobability quota samples of individuals who opted in to be members of online panels recruited and maintained by an online sample vendor. A sample obtained in this way is not representative of a population of interest. Thus, caution should be exercised in interpreting the current results. To address this issue, replication work is necessary with a probability-based sample. Second, the cross-sectional nature of the present data limits causal inferences about the variables of theoretical interest. It is possible that these variables are related in different fashions. For instance, engaging in political activities might predict perceived civility, because working with others to solve common problems or make desired changes could cultivate such perceptions. To draw a robust causal inference about the observed relationships, longitudinal data would be desired. Third, as is common with any other survey research, the retrospective, self-report data raise questions about data precision. This issue seems especially important in terms of exposure to uncivil and reasoned online political comments. Respondents might have recalled more exposure to uncivil comments because such comments are more memorable than reason-based comments.

Fourth, the measurement of the independent variables can be refined. For example, the present study used the exposure measure to investigate the influences of reasoned and uncivil comments on perceived civility and political participation. Yet this measure may not adequately capture the cognitive attention individuals assign when reading political comments online. In fact, Chaffee and Schleuder (1986) note that exposure and attention measures, like the ones we used here, tap different dimensions of information consumption, with the latter capturing cognitive expenditure not adequately addressed by the passive measure of exposure to information. Thus, future work can measure attention to reasoned and uncivil comments to increase the robustness of the results. Relatedly, these measures focused entirely on online comments. As exposure to reasoned and uncivil messages can occur in face-to-face settings and can influence perceptions of civility in society, future studies should measure not only online but also off-line exposure to these types of discourse and directly assess how they affect perceived civility both individually and in interaction.

Fifth, we acknowledge the possibility that the model tested in our study is underspecified in terms of the relationship between off-line and online political participation. There might be common causes not accounted for in our model such as political knowledge. It is also possible that the two forms of participation are causally related. Y. Kim, Russo, and Amna (2016) found that online political participation at Time 1 had a positive cross-lagged association with off-line political participation at Time 2 for adolescents, with the former possibly serving as a gateway to conventional off-line political venues. Although research has not produced consistent evidence among an adult population (Lane, Kim, Lee, Weeks, & Kwak, 2017), future work should address this possibility with longitudinal data.

Finally, our study did not account for personality traits that are likely related to our outcome variables. Research has reported that conflict-avoidant people have more negative reactions to incivility (Mutz, 2015), and agreeable people tend to perceive incivility in online comments (Kenski et al., 2017). It is possible that these people report they have encountered incivility online more and, as a result, report that society is uncivil. Future work should integrate these personality traits into the present model. It would also be worthwhile to examine perceived exposure to incivility. If conflict-avoidant and agreeable people are more affected by incivility, then they might be likely to perceive online comments as uncivil, even if those comments do not contain strong uncivil elements or are not perceived as uncivil by conflict-seeking and disagreeable people. The relevant issue might be perceptions of incivility rather than simple exposure to incivility.

The above limitations, however, should be balanced against several unique insights derived from the present analysis. Our study makes a connection between different types of online political comments and perceived societal civility. Adopting a media effects framework and using two data sets, we establish that uncivil and reasoned online political comments are differentially related to the extent to which individuals perceive civility of the nation. Such perceptions are associated with variations in off-line and online political participation. If uncivil online political comments foster perceived incivility in society and political disengagement, and reason-based political comments do the opposite, it would be important to formulate strategies to effectively moderate online political discussions in ways that encourage mutual respect, reasons, and evidence.


Althaus, S. L. (2012). What's good and bad in political communication research? Normative standards for evaluating media and citizen performance. In H. A. Semetko & M. Scammell (Eds.), SAGE handbook of political communication (pp. 97-112). Thousand Oaks, CA: SAGE Publications.

Anderson, A. A., Brossard, D., Scheufele, D. A., Xenos, M. A., & Ladwig, P. (2014). The "nasty effect": Online incivility and risk perceptions of emerging technologies. Journal of Computer-Mediated Communication, 19, 373-387. doi:10.1111/jcc4.12009

Andersson, L. M., & Pearson, C. M. (1999). Tit for tat? The spiraling effect of incivility in the workplace. Academy of Management Review, 24, 452-471. doi:10.2307/259136

Barabak, M. Z., & Duara, N. (2016, November 13). "We're called redneck, ignorant, racist. That's not true": Trump supporters explain why they voted for him. Los Angeles Times. Retrieved from

Benson, T. (2011). The rhetoric of civility: Power, authenticity, and democracy. Journal of Contemporary Rhetoric, 1(1), 22-30.

Berinsky, A. J., Huber, G. A., & Lenz, G. S. (2012). Evaluating online labor markets for experimental research:'s Mechanical Turk. Political Analysis, 20, 351-368. doi:10.1093/pan/mpr057

Borah, P. (2013). Interactions of news frames and incivility in the political blogosphere: Examining perceptual outcomes. Political Communication, 30, 456-473. doi:10.1080/10584609.2012.737426

Borah, P. (2014). Does it matter where you read the news story? Interaction of incivility and news frames in the political blogosphere. Communication Research, 41, 809-827. doi:10.1177/0093650212449353

Brooks, D. J., & Geer, J. G. (2007). Beyond negativity: The effects of incivility on the electorate. American Journal of Political Science, 51, 1-16. Retrieved from

Busselle, R. W., & Shrum, L. J. (2003). Media exposure and the accessibility of social information. Media Psychology, 5, 255-282. doi:10.1207/S1532785XMEP0503_02

Cappella, J. N., & Jamieson, K. H. (1997). Spiral of cynicism. New York, NY: Oxford University Press.

Chaffee, S. H., & Schleuder, J. (1986). Measurement and effects of attention to media news. Human Communication Research, 13, 76-107. doi:10.1111/j.1468-2958.1986.tb00096.x

Chen, G. M. (2017). Online incivility and public debate: Nasty talk. New York, NY: Palgrave Macmillan.

Chen, G. M., Muddiman, A., Wilner, T., Pariser, E., & Stroud, N. J. (2019). We should not get rid of incivility online. Social Media + Society, 5. doi:10.1177/2056305119862641

Cicchirillo, V. J., Hmielowski, J. D., & Hutchens, M. J. (2015). The "mainstreaming" of verbally aggressive online political behaviors. Cyber Psychology, Behavior and Social Networking, 18(5), 253-259. doi:10.1089/cyber.2014.0355

Coe, K., Kenski, K., & Rains, S. A. (2014). Online and uncivil? Patterns and determinants of incivility in newspaper website comments. Journal of Communication, 64, 659-679. doi:10.1111/jcom.12104

Dahlberg, L. (2001). The Internet and democratic discourse: Exploring the prospects of online deliberative forums for extending the public sphere. Information, Communication & Society, 4(4), 615-633. doi:10.1080/13691180110097030

Delli Carpini, M. X., Cook, F. L., & Jacobs, L. R. (2004). Public deliberation, discursive participation, and civic engagement: A review of the empirical literature. Annual Review of Political Science, 7, 315-344. doi:10.1146/annurev.polisci.7.121003.091630

Duggan, M., & Smith, A. (2016, October 25). The political environment on social media. Washington, DC: Pew Research Center. Retrieved from

Dutton, W. H. (1996). Network rules of order: Regulating speech in public electronic fora. Media, Culture & Society, 18, 269-290. doi:10.1177/016344396018002006

Fishkin, J. S., & Luskin, R. C. (1999). Bringing deliberation to the democratic dialogue. In M. McCombs & A. Reynolds (Eds.), The poll with a human face: The National Issues Convention experiment in political communication (pp. 3-38). Mahwah, NJ: Lawrence Erlbaum.

Freelon, D. (2015). Discourse architecture, ideology, and democratic norms in online political discussion. New Media & Society, 17, 772-791. doi:10.1177/1461444813513259

Funk, C. L. (2001). Process performance: Public reaction to legislative policy debate. In J. R. Hibbing & E. Theiss-Morse (Eds.), What is it about government that Americans dislike? (pp. 193-204). New York, NY: Cambridge University Press.

Gastil, J., Black, L., & Moscovitz, K. (2008). Ideology, attitude change, and deliberation in small face-to-face groups. Political Communication, 25, 23-46. doi:10.1080/10584600701807836

Gastil, J., Deess, E. P., & Weiser, P. J. (2002). Civic awakening in the jury room: A test of the connection between jury deliberation and political participation. Journal of Politics, 64, 585-595. Retrieved from

Gervais, B. T. (2014). Following the news? Reception of uncivil partisan media and the use of incivility in political expression. Political Communication, 31, 564-583. doi:10.1080/10584609.2013.852640

Gervais, B. T. (2015). Incivility online: Affective and behavioral reactions to uncivil political posts in a Web-based experiment. Journal of Information Technology and Politics, 12, 167-185. doi:10.1080/19331681.2014.997416

Gervais, B. T. (2017). More than mimicry? The role of anger in uncivil reactions to elite political incivility. International Journal of Public Opinion Research, 29, 384-405. doi:10.1093/ijpor/edw010

Gil de Zuhiga, H., Veenstra, A., Vraga, E., & Shah, D. (2010). Digital democracy: Reimagining pathways to political participation. Journal of Information Technology & Politics, 7, 36-51.

Han, S.-H., & Brazeal, L. M. (2015). Playing nice: Modeling civility in online political discussions. Communication Research Reports, 32, 20-28. doi:10.1080/08824096.2014.989971

Hardy, B. W., & Scheufele, D. A. (2005). Examining differential gains from Internet use: Comparing the moderating role of talk and online interactions. Journal of Communication, 55, 71-84. doi:10.1111/j.1460-2466.2005.tb02659.x

Hayes, A. F. (2019). Introduction to mediation, moderation, and conditional process analysis: A regression-based approach (2nd ed.). New York, NY: Guilford Press.

Herbst, S. (2010). Rude democracy: Civility and incivility in American politics. Philadelphia, PA: Temple University Press.

Heyward, A. (2016, April 25). Introducing "Tonar": An incivility index for election 2016. MIT Media Lab. Retrieved from

Hwang, H., Kim, Y., & Huh, C. U. (2014). Seeing is believing: Effects of uncivil online debate on political polarization and expectations of deliberation, Journal of Broadcasting and Electronic Media, 58, 621-633. doi:10.1080/08838151.2014.966365

Hwang, H., Kim, Y., & Kim, Y. (2018). Influence of discussion incivility on deliberation: An examination of the mediating role of moral indignation. Communication Research, 45, 213-240. doi:10.1177/0093650215616861

Jamieson, K. H., & Hardy, B. (2012). What is civil engaged argument and why does aspiring to it matter? In D. M. Shea & M. Fiorina (Eds.), Can we talk? The rise of rude, nasty, stubborn politics (pp. 27-40). Upper Saddle River, NJ: Pearson Press.

Jamieson, K. H., Volinsky, A., Weitz, I., & Kenski, K. (2017). The political uses and abuses of civility and incivility. In K. Kenski & K. H. Jamieson (Eds.), The Oxford handbook of political communication (pp. 205-217). Oxford, UK: Oxford University Press.

Johnson, D. W., & Johnson, R. T. (2000). Civil political discourse in a democracy: The contribution of psychology. Journal of Peace Psychology, 6, 291-317.

Kenski, K., Coe, K., & Rains, S. A. (2017). Perceptions of uncivil discourse online: An examination of types and predictors. Communication Research. Advance online publication. doi:10.1177/0093650217699933

Kim, J., Wyatt, R. O., & Katz, E. (1999). News, talk, opinion, participation: The part played by conversation in deliberative democracy. Political Communication, 16, 361-385. doi:10.1080/105846099198541

Kim, Y., & Kim, Y. (2019). Incivility on Facebook and political polarization: The mediating role of seeking further comments and negative emotion. Computers in Human Behavior, 99, 219-227. doi:10.1016/j.chb.2019.05.022

Kim, Y., Russo, S., & Amna, E. (2016). The longitudinal relation between online and offline political participation among youth at two different developmental stages. New Media & Society, 19, 899-917. doi:10.1177/1461444815624181

Lane, D. S., Kim, D. H., Lee, S. S., Weeks, B. E., & Kwak, N. (2017). From online disagreement to offline action: How diverse motivations for social media can increase political information sharing and catalyze offline political participation. Social Media + Society, 3(3). doi:10.1177/2056305117716274

Leiter, M. P., Laschinger, H. K. S., Day, A., & Oore, D. G. (2011). The impact of civility interventions on employee social behavior, distress, and attitudes. Journal of Applied Psychology, 96(6), 1258-1274. doi:10.1037/a0024442

Luskin, R. C., Fishkin, J. S., & Jowell, R. (2002). Considered opinions: Deliberative polling in Britain. British Journal of Political Science, 32, 455-488. Retrieved from

McClurg, S. (2006). Political disagreement in context: The conditional effect of neighborhood context, disagreement and political talk on electoral participation. Political Behavior, 28, 349-366. doi:10.1007/s11109-006-9015-4

Min, S.-J. (2007). Online vs. face-to-face deliberation: Effects on civic engagement. Journal of Computer-Mediated Communication, 12, 1369-1387. doi:10.1111/j.1083-6101.2007.00377.x

Moy, P., & Gastil, J. (2006). Predicting deliberative conversation: The impact of discussion networks, media use, and political cognitions. Political Communication, 23, 443-460.

Muddiman, A. (2017). Personal and public levels of incivility. International Journal of Communication, 11, 3182-3202. Retrieved from

Mutz, D. C. (2006). Hearing the other side: Deliberative versus participatory democracy. New York, NY: Cambridge University Press.

Mutz, D. C. (2015). In-your-face politics: The consequences of uncivil media. Princeton, NJ: Princeton University Press.

Mutz, D. C., & Reeves, B. (2005). The new videomalaise: Effects of televised incivility on political trust. American Political Science Review, 99, 1-15. doi:10.1017/S0003055405051452

Niemi, R. G., Craig, S. C., & Mattei, F. (1991). Measuring internal political efficacy in the 1988 National Election Study. American Political Science Review, 85, 1407-1413. Retrieved from

Noelle-Neumann, E. (1993). The spiral of silence: Public opinion--Our social skin (2nd ed.). Chicago, IL: University of Chicago Press.

Osatuke, K., Moore, S. C., Ward, C., Dyrenforth, S. R., & Belton, L. (2009). Civility, respect, engagement in the workforce (CREW): Nationwide organization development intervention at Veterans Health Administration. Journal of Applied Behavioral Science, 45(3), 384-410. doi:10.1177/0021886309335067

Oz, M., Zheng, P., & Chen, G. M. (2018). Twitter versus Facebook: Comparing incivility, impoliteness, and deliberative attributes. New Media & Society, 20, 3400-3419. doi:10.1177/1461444817749516

Papacharissi, Z. (2004). Democracy online: Civility, politeness, and the democratic potential of online political discussion groups. New Media & Society, 6, 259-283. doi:10.1177/1461444804041444

Pearson, C. M., Andersson, L. M., & Wegner, J. W. (2001). When workers flout convention: A study of workplace incivility. Human Relations, 54, 1387-1419. doi:10.1177/00187267015411001

Rosseel, Y. (2012). Lavaan: An R package for structural equation modeling. Journal of Statistical Software, 48(2), 1-36. doi:10.18637/jss.v048.i02

Rowe, I. (2015). Civility 2.0: A comparative analysis of incivility in online political discussion. Information, Communication & Society, 18, 121-138. doi:10.1080/1369118X.2014.940365

Shrum, L. J. (2009). Media consumption and perceptions of social reality: Effects and underlying processes. In J. Bryant & M. B. Oliver (Eds.), Media effects: Advances in theory and research (3rd ed., pp. 50-73). New York, NY: Psychology Press.

Sobieraj, S., & Berry, J. (2011). From incivility to outrage: Political discourse in blogs, talk radio, and cable news. Political Communication, 28, 19-41. doi:10.1080/10584609.2010.542360

Stromer-Galley, J. (2007). Measuring deliberation's content: A coding scheme. Journal of Public Deliberation, 3, 1-35.

Stromer-Galley, J., & Muhlberger, P. (2009). Agreement and disagreement in group deliberation: Effects on deliberation satisfaction, future engagement, and decision legitimacy. Political Communication, 26, 173-192. doi:10.1080/10584600902850775

Stryker, R., Conway, B. A., & Danielson, J. T. (2016). What is political incivility? Communication Monographs, 83, 535-556. doi:10.1080/03637751.2016.1201207

Valentino, N. A., Brader, T., Groenendyk, E. W., Gregorowicz, K., & Hutchings, V. L. (2011). Election night's alright for fighting: The role of emotions in political participation. Journal of Politics, 73, 156-170. Retrieved from

Valenzuela, S., Kim, Y., & Gil de Zuhiga, H. (2012). Social networks that matter: Exploring the role of political discussion for online political participation. International Journal of Public Opinion Research, 24, 163-184. doi:10.1093/ijpor/edr037

Verba, S., Schlozman, K. L., & Brady, H. E. (1995). Voice and equality: Civic voluntarism in American politics. Cambridge, MA: Harvard University Press.

Table A1: Descriptive Statistics

                     Study 1

Age                  M = 24.77, SD = 3.20
Gender (female)      74.4%
Race (White)         72.7%
Education (median)   4 = some college,
                     no degree
Income (median)      --
Political interest   M = 4.10, SD = 1.86
Party affiliation    M = 4.32, SD = 1.68
Political ideology   M = 4.05, SD = 1.67
Internal political   M = 4.35, SD = 1.49, a = .88
Traditional          M = 3.72, SD = 1.68, a = .88
media use
Uncivil political    M = 3.00, SD = 1.22, a = .95
Reasoned political   M = 2.80, SD = 1.05, a = .92
Perceived civility   M = 3.81, SD = 1.49, a = .93
in society
Off-line political   M = 1.94, SD = 1.15, a = .97
Online political     M = 2.17, SD = 1.15, a = .92

                     Study 2

Age                  M = 47.28, SD = 16.08
Gender (female)      51.4%
Race (White)         64.4%
Education (median)   5 = two-year
                     associate degree
Income (median)      6 = $50,000
                     to $74,999
Political interest   M = 5.58, SD = 1.72
Party affiliation    M = 4.27, SD = 1.92
Political ideology   M = 3.91, SD = 1.66
Internal political   M = 3.44, SD = 0.87, a = .85
Traditional          M = 2.28, SD = 0.95, a = .84
media use
Uncivil political    M = 2.80, SD = 1.19, a = .97
Reasoned political   M = 2.53, SD = 0.96, a = .92
Perceived civility   M = 2.54, SD = 0.88, a = .90
in society
Off-line political   M = 1.63, SD = 0.77, a = .90
Online political     M = 1.72, SD = 0.75, a = .84

Table A2: Residualized Covariance Matrices

Study 1
  Uncivil political    1.074
  Reasoned political   0.337    0.596
  Perceived civility   -0.105   0.116    1.716
  in society
  Off-line political   0.085    0.143    0.381   0.766
  Online political     0.135    0.190    0.331   0.614   0.713
Study 2
  Uncivil political    1.158
  Reasoned political   0.425    0.641
  Perceived civility   -0.183   -0.001   0.731
  in society
  Off-line political   0.050    0.116    0.092   0.422
  Online political     0.060    0.114    0.050   0.311   0.369

MASAHIRO YAMAMOTO University at Albany, State University of New York, USA

FRANCIS DALISAY University of Guam, USA

MATTHEW J. KUSHIN Shepherd University, USA

Masahiro Yamamoto:

Francis Dalisay:

Matthew J. Kushin:

Date submitted: 2019-04-19

(1) Because the second survey was conducted prior to the U.S. 2016 presidential election, a few items were tailored to the election context.
COPYRIGHT 2020 University of Southern California, Annenberg School for Communication & Journalism, Annenberg Press
No portion of this article can be reproduced without the express written permission from the copyright holder.
Copyright 2020 Gale, Cengage Learning. All rights reserved.

Article Details
Printer friendly Cite/link Email Feedback
Author:Yamamoto, Masahiro; Dalisay, Francis; Kushin, Matthew J.
Publication:International journal of communication (Online)
Date:Jan 1, 2020
Previous Article:Farming in the Face of Uncertainty: How Colombian Coffee Farmers Conceptualize and Communicate Their Experiences With Climate Change.
Next Article:Understanding Americans' Perceptions of Nuclear Weapons Risk and Subsequent Behavior.

Terms of use | Privacy policy | Copyright © 2022 Farlex, Inc. | Feedback | For webmasters |