Printer Friendly

Global information technology pressure and government accountability: the mediating effect of domestic context on website openness. (Articles).

To what extent does the global information technology revolution affect the openness and transparency of bureaucracies? And in what ways does the domestic context of nations determine the willingness or ability of bureaucrats to be accountable to citizens through the use of information technology? Comparison of any two countries inevitably identifies differences in the value and expression of accountability in government (Peters 1995, 290-91). However, in the absence of a good theoretical framework or a natural experiment research context, even reliable cross-national measurement of dependent and independent variables often makes such comparisons unmanageably messy at best.

An identifiable set of global institutions, technologies, and values now exists that is adopted by bureaucracies worldwide and is measurable across nations (Welch and Wong 1998). In addition, comparative administration research has resurged, giving the field a set of comparative tools that may be applied globally (Heady 1996). This study combines two streams of research--the Cyberspace Policy Research Group's (CyPRG) (1) comparative analysis of website openness and Ferrel Heady's (1966) classification of the major dimensions of civil service systems--to test hypotheses about how the domestic context, as associated with the national civil systems, mediates the effect of the global information technology (IT) pressure on government accountability.

Ensuing sections of this article include a theoretical framework for analysis of the effect of global pressures on bureaucratic change, a literature review on information technology and accountability, hypotheses predicting the effect of domestic context on website openness, data and methods, findings, and discussion.

COMPARATIVE ANALYSIS OF GLOBAL PRESSURES ON BUREAUCRATIC CHANGE

For public bureaucracies worldwide, globalization represents a set of complex and interactive stimuli, demands, and opportunities that exist in the organizational environment but are not specifically associated with a particular nation. Because organizations are open systems (Selnick 1966; Thompson 1967; Aldrich 1979; Rainey 1998; Scott 1998), global pressures such as multinational agreements or the information technology revolution affect the structure and behavior of public organizations (Welch and Wong 1998). Research in comparative public administration and globalization suggests that while global pressures place similar demands on public organizations worldwide, the patterns of adoption and organizational change are discernable and depend significantly on the domestic context surrounding the public bureaucracy (Aberbach, Putnam, and Rockman 1981; Riggs 1994; Heady 1995; Welch and Wong 1998; Farazmand 1999). Accordingly, this study utilizes a previously developed comparative framework for assessing the effect of global pressures on bureaucratic change (Welch and Wong 1998; see exhibit 1).

[ILLUSTRATION OMITTED]

The framework shows that global pressures affect public organizations directly and indirectly through domestic contexts. Despite significant variation in the structures and processes of public organizations worldwide, global pressures create common exigencies upon bureaucracies that result in predictable reactions or changes by public organizations. Moreover, the model suggests that elements of the domestic context filter the effects of global pressures in predictable ways. As a result, the domestic context either offsets or reinforces the change induced by the global pressure on public bureaucracy. (2)

The analysis of the effects of global pressures on nations is often viewed as contributing to the debate on convergence theory: institutions in different countries, including the public bureaucracy, tend to converge to a common pattern (OECD 1993 and 1995; Cheung 1997; Kettl 1997; Doremus et al. 1998; Hallenberg and Basinger 1998). Research to date shows that evidence for convergence probably depends on the policy area analyzed and the level of abstraction of the comparison (Scharpf 1997). We agree with Scharpf on the importance of gaining insight from well-defined empirical studies. As we will show, the interesting questions are less about convergence per se and more about the contextual or domestic factors that influence national-level administrative change toward or away from the global trends.

Using the framework as a guide, we seek to determine how the domestic context of public agencies affects the openness of websites, where establishment of a website represents adoption of a global information technology and change in the level of website openness represents the revealed level of change in accountability of the agency. It will be shown that domestic context intervenes in predictable ways to enhance or limit the level of agency accountability.

GLOBAL INFORMATION TECHNOLOGY PRESSURE AND PUBLIC ACCOUNTABILITY

Global information technology (IT) pressure on public organizations drives procurement, application, and accountability. Pressure for IT procurement and application is felt by all nations due to the increased availability of information globally, improved data collection techniques, and demand for processing capacity. Procurement of new hardware and software provides bureaucracies with the physical capacity to acquire, store, and use information to improve measurement, innovation, and problem solving (Kraemer, Dedrick, and King 1995). Acquisition and application of information technology is also driven by political and economic demands. Politically, governments increasingly are signatories to global conventions and agreements that require sophisticated data collection and analysis capacities. Additionally, public officials and political groups must be knowledgeable about how global trends and issues affect their nations (Public Management Service 1999). Economically, application of information technology has been linked to increases in productivity and growth (Kraemer and Dedrick 1999). Global transactions hold natural barriers to trust that information disclosure can help dissolve (Farazmand 1999; Public Management Service 1999). For all these reasons, increases in training, access, and use of information technology are thought to provide national benefits.

Others see a growing trend in citizen empowerment and accountability enhancement resulting from access to information and interaction with government (Kahlin 1997; La Porte, de Jong, and Demchak 1999). Disclosure or availability of information is also a symbol of trust, modernity, and global citizenship that may be necessary for competitive vitality of the nation, political legitimacy, or may be due to other incentives such as cooperativeness (DiMaggio and Powell 1983; Strang and Meyer 1993). Welch and Wong (1998) show that global information technology pressure can expose public organization tendencies toward accountability. For example, officials in nondemocratic countries may try to control access to information flows across global technologies such as the World Wide Web to enhance local power and autonomy.

The challenge for research on accountability in public administration is that many of the expectations for performance are multiple, changing, and contradictory, making measurement difficult (Romzek and Dubnick 1987; Johnston and Romzek 1999). Accountability involves relationships in which a public organization is held to answer for its performance (Johnston and Romzek 1999). It is further specified into accountability to and accountability for dimensions. Public organizations are ultimately accountable to the citizenry. Under the paradigm of new public management reform, the focus of accountability for has been shifted from "process" to "outcome" (Bardach and Lesser 1996). Both dimensions identify a significant relationship between openness of public organizations and government accountability.

In this era of globalization, accountability is not only important in the relationship between citizenry and government at the domestic level, it is also an internationally important issue. As states become increasingly interdependent, multinational institutions are becoming more important mechanisms for resolving problems of collective action, high transaction costs, and information deficits or asymmetries (Martin and Simmons 1998). A major problem of collaboration concerns the structure of incentives for compliance (Mitchell 1994). States defect for a variety of reasons including low national benefits, concerns of free riding, and fears of reprisals. A high level of transparency in a compliance system allows effective monitoring and provides a basis for sanctions that forces signatories and members of cooperative agreements to be internationally accountable (Jervis 1985; Mitchell 1994). Monitoring of the performance and compliance of each country is further enhanced when domestic institutions are more transparent (Cowhey 1993; Martin and Simmons 1998). While effective international organization often has its own compliance system, transparency in domestic institutions provides alternative channels for information collection and verification. Therefore, greater transparency in public organizations can lead to higher accountability at both domestic and international levels, to both citizens and member states of international organizations.

The Cyberspace Policy Research Group further develops this relationship between openness and accountability in its comparative research of websites. CyPRG defines government websites openness to be a function of two factors: transparency and interactivity. Transparency refers to the extent to which an organization reveals work and decision processes and procedures. In terms of a website, transparency "constitutes a layman's basic map of the organization as depicted in the information on the site [and] reveals the depth of access it allows, the depths of knowledge about processes it is willing to reveal, and the level of attention to citizen response it provides" (La Porte, de Jong, and Demchak 1999). A more transparent government allows citizens to monitor the performance of public organization more easily through the increase in the availability of information (Reichard 1998). Interactivity refers to the quality of communication between agency and citizen. "[It] is a measure of the level of convenience or degree of immediate feedback [provided]" (La Porte, de Jong, and Demchak 1999). A more interactive public organization enhances accountability by being more responsive to the preferences of the citizenry.

CyPRG hypothesizes that higher levels of interactivity and transparency indicate a lower level of managerial control in public organizations (CyPRG 2001b). Lower levels of control are also expected to result in greater organizational effectiveness as managers are more inclined to allow greater flexibility for problem solving. Importantly, CyPRG also hypothesizes that lower levels of control (greater openness) are associated with greater accountability (La Porte, de Jong, and Demchak 1999). Because our initial thinking on the effects of the global pressure of information technology on accountability is similar to CyPRG constructs (Welch and Wong 1998), in this article we adopt CyPRG definitions and measures of openness and utilize CyPRG data that is publicly available. We will now move to an analysis of Heady's classification of civil service systems as a means for determining how domestic context affects changes in accountability over time.

THE MEDIATING EFFECTS OF DOMESTIC CONTEXT ON OPENNESS

The global effects framework indicates that economic, political, and social systems are fundamental elements of a domestic context that affect the expression of global pressures in public bureaucracies. Nevertheless, research that has used many of the more direct measures of these three broad constructs to explain variation in the openness of agency websites has resulted in weak findings. One reason may be that many of the direct measures are highly correlated and often lead to serious multicollinearity problems (La Porte, de Jong, and Demchak 1999). Importantly, the direct-measure approach appears to provide only limited causal insight (La Porte, de Jong, and Demchak 1999). For example, to test eight theory-based hypotheses the CyPRG team used such variables as national income, central government expenditures, integration with the world economy, science, research and education, computers and Internet hosts, cultural values, democracy, and legal system. None of the variables were found to be significant in the full model (3) while more parsimonious models revealed that national income is related to openness.

Overall, only those hypotheses relating to national income or wealth have a bearing on openness in both OECD and non-OECD countries, either directly or via computer ownership.... None of the other hypotheses are supported by this data. Therefore, even though we find reasonably robust results for a link between openness and national income, we still do not have a very satisfying explanation of its sources. National income only explains about a quarter of the variance observed in openness scores, and we do not know what it is about national income or wealth that produces more open government organizations. The wide variety of more specific proposed mechanisms all fail to predict the pattern we observe in the data (La Porte, de Jong, and Demchak 1999, 16).

Because of the limitations identified in previous studies, we will adopt Heady's framework (1996) that classifies national civil service systems on five dimensions: relation to political regime, socioeconomic context, focus for personnel management, qualification requirements, and sense of mission. This framework is particularly attractive because it focuses on the interaction among the political, economic, and social dimensions of the domestic context as they pertain to the civil service, which may be more appropriate than the direct measures discussed. We expect that measures of these complex and nuanced dimensions will help explain how the domestic context can affect the bureaucratic behavior as measured by the change in website openness. The following sections address each of Heady's five dimensions in turn as they relate to website openness and accountability.

Relationship of Regime and Civil Service

The relationship between the civil service and the political regime addresses the independent power of the civil service. This dimension ranges from the lowest level of independent power under a ruler responsive regime, to progressively higher levels under single party responsive, majority party responsive, and military responsive. Ruler responsive describes a context in which political power resides in the hands of a ruler or ruling group that exercises legitimate control over the civil service system. In this system, bureaucrats are controlled and have little discretion to make public policy. In single party regimes the civil service is subservient to a dominant political party in a system in which political competition is minimal (e.g., communist regimes). In majority party regimes, where competition among political parties is the norm, the civil service is responsive to the governing party. Parliamentary systems control the civil service more directly than presidentialist systems due to the lack of competition between executive and legislative branches. Competition in presidentialist systems results in more-diffuse streams of policy direction. Military responsive regimes describe systems in which military rulers operate the country in coalition with their junior partners, the civil service. Collaboration insuring success of the regime implies dependence of rulers on the administrative and policy-making ability of civil servants.

In a global environment in which power is leaking from national institutions to global institutions (Cleveland 1993; Farazmand 1999), national political regimes may seek to maintain their power bases. This situation leads to politicians' resistance to efforts to increase information disclosure (Reichard 1998). Bureaucracies that are more dependent upon the political regime for power are more likely to control information in response to political demands. One outcome of the general trend will be to reduce the level of openness of the government websites. This reasoning can be formalized by the following hypothesis.

Hypothesis 1: The greater the independent power of the civil service, the more accountable the government will become under global information pressure.

Alternatively, it may be true that much higher levels of independence lead bureaucrats to control information disclosure and limit interaction with external groups as a means of maintaining their own power. Therefore, it is possible that instead of a linear relationship between dependence and accountability, a nonlinear relationship exists in which high levels of dependence and independence lead to reduced levels of accountability while moderate levels of independence lead to higher levels of accountability. We state the competing hypothesis below.

Hypothesis 2: Higher levels of dependence and independence of the civil service lead to reductions in accountability while a moderate level of independence leads to increases in accountability (a U-shaped curve).

Qualification Requirements

Qualification requirements refer to the degree of involvement by bureaucrats in decisions about their own qualification requirements. Civil service systems are categorized patrimonial where political rulers determine qualification requirements. Increasing levels of civil service power to determine qualifications are evident under party loyalty, party patronage, professional performance, and bureaucratic determination subcategories. Where party loyalty signifies an important qualification, bureaucrats show allegiance as a prerequisite to service, while party patronage service qualification requires simultaneous and equal allegiance to party support and public work. Professional performance describes systems in which technical competence and socialized neutrality are primary qualifications, and bureaucratic determination identifies those systems in which bureaucrats determine professional and performance standards.

In a global environment, civil service systems that allow managers little decision-making autonomy over qualification requirements also allow little general decision-making autonomy. There exists greater room for managerial discretion in systems where political standards are relatively less prevalent and professional standards are relatively more important. Widely accepted objective standards and professional values provide justification for action. However, in systems where discretion is low and political standards are prevalent, government organizations will limit or reduce their adoption of global standards and professional values of accountability or openness. This reasoning is written formally into the following hypothesis.

Hypothesis 3: Civil service systems in which bureaucrats have greater ability to determine qualification requirements will be more likely to increase government accountability as a result of global information pressure.

Alternatively, it may be true that much higher levels of bureaucratic power may also lead to behavior that limits external evaluation of decisions in order to maintain power. High concentration of power in the hands of the bureaucracy causes centralization and control of information by the bureaucracy to secure power. We express this nonlinear relationship in the following hypothesis.

Hypothesis 4: High and low levels of ability of bureaucrats in determining qualification requirements lead to reductions in accountability while moderate levels of ability increase in accountability as a result of global information pressure (a U-shaped curve).

Socioeconomic Context

Socioeconomic system refers generally to the role of state in society as determined by the approach to decision making in the polity. This dimension ranges from minimal role of the state under a traditional system to progressively higher roles under pluralist competitive, mixed, corporatist, and centrally planned decision-making structures. Under a traditional system, civil servants are bound by tradition requiring obedience to the leader. Hence bureaucrats in this setting are relatively passive. Pluralist competitive systems depend significantly on outcomes from market forces and competition among individuals and groups as a basis for socioeconomic decision making. Corporatist socioeconomic systems place the state in a central decision-making role where it is the state that mediates the roles of competing groups. Mixed systems fall in between pluralist competitive and corporatist categories. Civil servants hold the greatest decision-making role relative to the other groups in centrally planned systems, which are traditionally void of political pluralism and illegitimate interest groups.

Because control over information is one way that nations can maintain power, national polities will resist disclosure and openness to limit leakage of power under globalization (Cleveland 1993; Kraemer and Dedrick 1997; Welch and Wong 2000). Public managers in domestic systems in which the role of the civil service is passive and obedient will be less likely and less politically able to increase openness and accountability. However, managers who operate in open systems are subject to the pressures of globally accepted institutions and norms where openness is increasingly a symbol of trust, modernity, and global citizenship (DiMaggio and Powell 1983; Strang and Meyer 1993). Therefore, it is expected that managers with a greater policy-making role will, over time, increase openness and move toward greater accountability.

Hypothesis 5: The greater the role of the state in society, the more likely government organizations will increase accountability levels under global information pressure.

Alternatively, it may be true that in nations where the role of the state is very high, there may be an expectation by citizens or by tradition that the state and the bureaucracy know best. Public managers in domestic systems where the policy-making role of civil servants is greater may seek greater information control as a source of power (Reichard 1998). Therefore, an alternative hypothesis follows.

Hypothesis 6: High and low levels of state role lead to reductions in accountability while moderate levels of state role lead to increases in accountability under global information pressure (a U-shaped curve).

Focus for Personnel Management

Focus for personnel management describes the extent to which personnel decisions are centralized in the nation, and hence the extent to which the civil service system is centralized. This dimension identifies four categories--chief executive, independent agency, divided, and ministry-by-ministry--and ranges from highly centralized to decentralized decision making. The chief executive category describes a system in which the ruling individual or party directly controls selection and behavior of civil servants. Independent agency refers to systems in which personnel management is a politically isolated agency. The divided category identifies those civil service systems in which some functions, such as initial recruitment or training, are handled by one agency while other functions are handled by other agencies. The greatest form of decentralization of personnel systems is delineated by the ministry-by-ministry category in which ministry or department heads control personnel decisions. This usually occurs in political systems where the ruling elite controls "appointment, retention, and behavior of ministry or department heads" (Heady 1996, 216).

Nations in which civil service systems are highly centralized are also less able to develop flexible agency-by-agency responses to complex situations. A large number of rules, regulations, and norms of conduct associated with centralization restrict managerial ability to creatively adapt organizational systems to change. In a global context, global change toward greater openness will be reflected in more effectively and functionally decentralized and less structurally differentiated systems than in centralized systems (Demchak, Friis, and La Porte 1998; La Porte, de Jong, and Demchak 1999). The following statement formalizes this expectation.

Hypothesis 7: The greater the extent to which personnel decisions are decentralized, the more likely global information pressure will lead to increases in accountability.

Sense of Mission

The final dimension of Heady's framework, sense of mission, addresses the type of values shared by the civil service. Civil service systems that are compliance oriented value conformity without question to directives, while systems defined as cooperation, policy responsiveness, constitutional responsiveness, and guidance hold an increasingly higher value for leadership and intervention. Cooperation refers to shared values for taking actions that are acceptable to political superiors and refers to preservation of the civil service status and policy-making role. Policy responsiveness delineates systems in which bureaucrats value professionalism and obligation to respond reliably to political leadership. Constitutional responsiveness identifies systems or occasions in which civil servants are responsive to constitutions or legal systems rather than to policy or political officials. In systems where guidance is a shared value, civil servants view themselves as the "most legitimate and best-equipped group for setting and achieving goals" (Heady 1996, 220).

Under globalism, politicians may resist efforts to increase information disclosure to maintain national power. In domestic contexts where civil servants exhibit higher value for responsiveness, administrative actions will conform to demands by political leaders. Contrarily, in systems in which managers see themselves as professionals legitimately responsible for achieving public goals in a global context, there will be an increased tendency for openness due to global demands or norms of accountability. As a result, government organizations composed of a relatively more professional bureaucracy will be less likely to respond to national politicians and more likely to respond to competitive and cooperative demands of the global context.

Hypothesis 8: The greater the sense of professionalism and leadership legitimacy of the civil service, the more likely global information pressure will lead to higher levels of accountability (less control).

Alternatively, it may be true that bureaucrats who view themselves as legitimate leaders may limit the ability of external entities to review decisions or contact responsible parties. Therefore, a nonlinear relationship between professionalism and accountability may exist where high and low levels of professionalism lead to reduced levels of accountability while moderate levels of professionalism lead to higher levels of accountability.

Hypothesis 9: High and low levels of professionalism in the civil service lead to lower levels of accountability while moderate levels of independence lead to higher levels of accountability (a U-shaped curve).

DATA AND METHODS

This study takes advantage of the availability of two existing sources of data: the CyPRG data on website openness of national agencies and Heady's categorization of civil service systems. Website openness is a proxy that represents change in accountability of public organizations. The CyPRG data provide the dependent variables and the Heady measures provide most of the independent variables that are used in ordinary least squares regression analysis. Data sets, data limitations, descriptive statistics, and methods will be discussed in turn.

Since 1995, the CyPRG Group has collected this data at the national and ministerial levels worldwide. Because official information provided by public organizations has both political implications and legal obligations associated with it, the nature and amount of on-line information should reflect the policy choices made by public officials and the organizational dynamics of the agency. However, CyPRG only collects data on existing national level agency websites. Absence of information on government agencies without websites confines our research to answering questions on factors affecting the extent of openness of government websites, not on the choice of having a website or not. (4) Although we may assume that the same set of domestic factors affects these two sets of decisions, we are not able to test these assumptions here.

Openness of government agency web sties is a combined function of the amount of data available on agency websites (transparency) and the ease with which users are able to access people or data (interactivity). Transparency is measured using five constructs: ownership, contacts, issue or organizational information, citizen consequences, and timeliness of data. Ownership refers to the observed importance of the website to the organization. Contacts represents information on how to reach individuals in the organization about specific agency operations. Issue or organizational information covers availability of information about the policy area of concern to the organization. Citizen consequences generally measures the extent to which the organization provides information about agency requirements for citizens and whether the agency provides citizens with the ability and instructions to respond. Timeliness concerns information on updating of the web page. Each of these constructs is measured using multiple criteria, and most measures are scored 0 or 1 indicating the absence or existence of any one of the specific criteria (CyPRG 2001c and 2001d; see the appendix). These scores are then summed to give an overall measure of transparency.

Interactivity is measured as the combined score of multiple measures associated with a different set of four constructs including ownership, reachability, issue or organizational information, and citizen consequences (CyPRG 2001e). Ownership generally measures the existence of on-line accessibility to managers or officials responsible for the website. Reachability measures the existence of directly clickable e-mail links to individuals inside the organizations (not website related). Issue or organizational information concerns the ability of users to access issue- or organization-related information on-line, and citizen consequences represents the accessibility of forms and documents of consequence to the citizen. As with transparency, the final score for interactivity is a sum of multiple measures of the four constructs (CyPRG 2001f; see the appendix). Openness is a combined score of transparency and interactivity. This data is currently on the web and has been used or referenced in at least five manuscripts (CyPRG 2001a).

For this study, we took the dependent variables we used in OLS regression equations from changes in average values from 1997 to 1999 for transparency and interactivity for eight nations. (5) Changes in transparency and interactivity were calculated by subtracting 1997 values from 1999 values. Transparency and interactivity are, as would be expected, highly correlated (r = 0.59). These two variables are combined by the CyPRG group to represent overall openness of websites. Therefore, these two variables were also combined in this study into a third variable called openness (chronbach alpha correlation coefficient for these two variables was 0.74). In all, one hundred agency websites were observed from 1997 to 1999 in the eight countries that were studied.

Independent measures of the five dimensions of civil service systems were based on Heady's research (1996). Seven countries were common to both the CyPRG and Heady data sets. An eighth, Japan, was added based on previous knowledge and analysis (Welch and Wong 2000). The eight countries are Egypt, France, Indonesia, Japan, Korea, Saudi Arabia, United Kingdom, and United States. Coding for these variables is provided in exhibit 2. (6)

Additionally, two measures of economic context were used in the model. International trade in goods in 1997 as a percent of gross domestic product measured in purchasing power of parity terms controls for the extent to which the nation is integrated with the global economy. Gross private capital flows in 1997 as a percent of gross domestic product measured in purchasing power of parity terms controls for the total value of financial transactions (debits and credits) during a given period. Both measures were taken from World Bank statistics, which were based on International Monetary Fund and World Bank databases (World Bank 1998). As with much of the national level data, there is a high correlation between these two variables. Therefore, they were combined into a new variable called open economy (chronbach alpha correlation coefficient was 0.78). Other variables that control for economic and social context show high intercorrelation and were dropped from the model due to high levels of multicollinearity.

Finally, controls were developed for agency sector. The sector variable distinguishes between internally focused agencies (justice, labor, library, and legislature) and externally focused organizations (defense, finance, immigration, foreign, state). It is a discrete (1/0) variable in which the agency is coded (0) if it is internally focused and (1) if it is externally focused.

Descriptive statistics for all variables are shown in exhibit 3. Descriptive statistics for transparency and interactivity are also shown by sector type in exhibit 4. Findings indicate that in the eight countries, transparency and interactivity increased between 1997 and 1999, with increases in interactivity slightly greater than in transparency. These findings hold for subsamples of externally and internally oriented agencies. However, relatively speaking, externally oriented agencies show a larger increase in both transparency and interactivity. We note that at a general level, these findings show some support for convergence theory--overall movement toward greater openness.

Initial analysis also showed a high degree of correlation between two pairs of the coded independent variables from the Heady framework. One pair, qualification requirements and relation to political regime, is thought to measure essentially the same construct: political autonomy of the civil service. The chronbach alpha correlation coefficient for these two variables was 0.87. Systems in which civil servants have a higher level of independent power are strongly associated with systems in which the bureaucracy has greater input into qualification requirements. These two variables were combined into a new variable called political autonomy. Following previously developed hypotheses, it is expected that higher levels of political autonomy will be associated with higher levels of transparency and interactivity. Alternatively, it may also be argued that high political autonomy of the civil service will lead to lower openness. Bureaucrats, when they are in the position of rulers, may also want to control information to prevent power leakage. Therefore, a competing hypothesis can also be set up that there is a U-shaped relationship between political autonomy and level of openness.

The second pair of correlated variables from the Heady framework includes role of state in society and focus for personnel management. These two variables also appeared to measure a similar phenomenon: the degree of structural complexity of the civil service. A new variable, structural complexity, was created by combing the two variables (chronbach alpha correlation coefficient is 0.70). Structural differentiation is one accepted measure of organizational complexity (Rainey 1998). A higher degree of horizontal differentiation of organizations exists in nations where the focus for personnel management is located in each separate agency (Blau and Schoenherr 1971; Meyer 1979; Hall 1996). We find that higher levels of horizontal differentiation (more subunits, specializations, and coordinative functions) exist in countries where the role of the state is strong and civil servants are active in policy making. This correlation can be explained by the expectation that structural complexity equips the state and the civil service with a higher capacity to sustain an active role in society under a complex environment (Lawrence and Lorsch 1967; Daft 1995). Correlations between all variables are reported in exhibit 5.

Two sets of ordinary least squares (OLS) regression equations were run. The first set utilized continuous independent variables to estimate the effect of domestic context on change of openness. The second set of regression equations created dummy variables for low, medium, and high levels of autonomy to test for nonlinear effects of autonomy on the change in openness. For each set, we ran one equation for each of the dependent variables: transparency change, interactivity change, and openness change.

FINDINGS

Results from three initial regression runs on transparency, interactivity, and openness indicate partial support for the hypotheses (see exhibit 6). All models are significant, assumptions of normality are not violated, and tests show no problems of multicollinearity. Findings from the transparency model show that greater structural complexity is negatively associated with higher levels of transparency (this relationship is also represented in bivariate plots and correlations). The strength of this variable and its significance provide evidence for rejection of the hypothesis that complexity of national civil service systems tends to provide the institutional capacity or incentive to respond more to global trends or demands for greater openness (H5 and H7). Instead, civil servants in highly differentiated, decentralized systems are less likely to respond to global demands for greater accountability.

Additionally, sector appears to be important. Externally oriented agencies are more likely to move toward greater transparency than are internally focused agencies. This suggests that externally focused agencies respond to global pressure for information disclosure and accountability more than do internally focused agencies. In this model, autonomy, mission, and the openness of the economy are not significantly associated with change in transparency, though they all have the expected signs. This leads to a number of questions. First, these results leave open the possibility of a U-shaped relationship between autonomy and transparency change (H2 and H4) and between mission and transparency change (H9). Closer inspection of data plots and correlations reveals that while the mission-transparency relationship appears to be positive linear, the autonomy-transparency relationship appears to be U-shaped. Since the autonomy variable is a combined measure of the two variables relation to political regime and qualification requirement, the regression results and the examination of the data plots lead to the rejection of H1 and H3 that there is a linear relationship between transparency and the two related variables. We will make adjustments for this. It is not clear why transparency is not significantly related to openness of the economy. In comparison with the explanatory power of the interactivity and openness models, the loss of this variable seems to significantly reduce the explanatory ability of the transparency model. The transparency model has the smallest adjusted R-square among the three models.

The interactivity and openness models indicated some results similar to those of the transparency model, with structural complexity negatively associated and sector positively associated with change in interactivity. However, findings for these two models also indicate positive association between mission and interactivity and openness changes, and between open economy and interactivity and openness changes. These results confirm expectations that nations in which civil servants are less compliance oriented and more guidance oriented will exhibit higher level increases in website interactivity and openness (H8). Similarly, results indicate a lack of evidence for the alternative hypothesis of a U-shaped relationship between mission and accountability (H9). To add to suspicions about the existence of a U-shaped relationship between autonomy and accountability, the coefficient for political autonomy is negative in these two models while it is positive in the first model (albeit insignificant). This shows some instability of the measure in the model and led to the development a second model in which autonomy was changed from a continuous variable to three dummy variables indicating low, medium, and high political autonomy to test alternative hypotheses H2 and H4.

Three additional OLS regression runs substituted dichotomous variables for low and medium autonomy for the continuous political autonomy variable. The third variable--high autonomy--was left out, which made the intercept its de facto indicator (see exhibit 7). All models are again significant while assumptions of normality and low multicollinearity were not violated. Results for all variables not related to autonomy are generally identical to the initial model. However, in this new model, high autonomy is negatively related to transparency, interactivity, and openness (the standardized coefficient is normally zero). This indicates that higher levels of autonomy in the civil service lead to a reduction of transparency, interactivity, and openness of websites. Contrary to expectations of the effects of globalism on bureaucratic behavior, civil servants in high autonomy contexts are not responding to higher demands for openness as a basis for trust and confidence. They can be seen instead as enhancing control over information.

Moderate autonomy within a competitive environment is positively associated with changes in interactivity, but it is not significantly associated with transparency or openness change. This indicates that in nations where civil servants hold a modicum of autonomous power, where political and economic competition are evident, and where structural complexity is limited, website interactivity increases more over time. The competitive environment may lead to greater need for direct information exchange and demand greater accountability for information that is disseminated. Nations with low autonomy are positively associated with changes in interactivity, transparency, and openness but the results are not statistically significant. Finally, because we were concerned that the combination of qualification requirements and role of state in socioeconomic context did not represent structural complexity, we reran the model using each of the constructs in separate models. Results showed no important differences in the strength or significance of the findings.

In interpreting the findings, it should be noted that the eight countries for the study (although a total of one hundred agencies from the eight countries makes up the data set) were chosen primarily because they represent a wide variation in the policy variables we are interested in for the study. In addition, we found it highly complex to reliably categorize national civil service systems. We therefore had to rely heavily on Heady's existing research and our own familiarity with a small set of national civil service systems in constructing the data set of the study. Therefore, strictly speaking, the nations do not represent a statistically random sample drawn from the population of all nations. This limits the generalizability of our findings to other countries. Caution should be exercised in generalizing the findings to nonsample countries. However, we believe that the findings and theoretical insights of the study should still have a reference value for other countries and theory development in public administration.

DISCUSSION

In general, findings show that increases in transparency, interactivity, and openness are positively associated with externally oriented agencies and integration with the world economy. Therefore, at one level we can conclude that accountability is induced by necessity. It is pushed by the need for governments to integrate with the global economy and by the information technology pressure. This study shows some evidence for factors determining the convergence of civil service systems toward greater openness. However, it is important to note that global pressure itself is only one factor that explains change in government accountability as expressed by the transparency, interactivity, and openness measures. While common pressures push for similar change at the global level, domestic factors can mediate the effects of the global pressures on public bureaucracy.

Structural complexity is found to be negatively associated with openness. A more active role of the state in the socioeconomic context and a less politically controlled but more bureaucrat centered focus of personnel management will offset the openness change pushed by the global pressure. Interestingly, increases in transparency, interactivity, and openness are negatively associated with high levels of autonomy. Civil service systems that are highly politically independent and able to make their own decisions about qualifications are less likely to increase government accountability through the increase in these attributes. This may indicate that powerful civil service elites attempt to maintain control over the quantity of information disclosure and resist global pressures for greater accountability. Additionally, domestic systems where civil servants hold moderate power--as in politically and economically competitive environments--are more likely to increase the level of interactivity.

Findings on change in interactivity and openness also indicate the importance of mission. In a global context in which information technology pressure calls for greater government accountability, those systems in which civil servants are more active and more professional and in which they consider themselves to be legitimate policy makers are more likely to increase the level of interactivity over time. Mission appears to have no effect on transparency. This may indicate that more-professional bureaucrats consider interactivity to be more directly associated with accountability but that they do not consider the level of disclosure (transparency) to be so.

Domestic systems with the highest levels of economic integration, market competition, and political competition are more apt to increase accountability as measured by interactivity. Information technology, as a global pressure, leads to greater transparency in general, but the extent of interactivity seems to be a function of the domestic environment of competitiveness. It is possible that interactivity represents a fundamental change in the accountability relationship between citizenry and government, and therefore it is more sensitive to the domestic context. Only a politically competitive environment, with a professionally driven and mission oriented bureaucracy, will induce an interactive government.

In summary, findings on civil service show that when global information technology pressure drives governments to increase accountability, differences in the level of accountability change of the agency depend upon the national contextual contingencies. Importantly, findings suggest that bureaucrats may think and behave as rulers in order to control information to consolidate power. Conditions of high autonomy, high complexity, and low professionalism exacerbate these tendencies. A pure administrative state governed by bureaucrats, without the checking of elected politicians, may lead to a less accountable and open government. This means that competition between political regimes and the civil service will be an important factor in determining whether the global pressure of information technology enhances or reduces accountability. It has led to the classic debate in public administration: How is the power conflict between bureaucracy and democracy in modern states to be resolved? The issue is much more than a power struggle between politicians and bureaucrats. It is related to the fundamental differences in the two models of governing--the politics model, rule by elected politicians, and the governance model, rule by professional bureaucrats (Aberbach, Putnam, and Rockman 1981). These findings reinforce the need for and importance of further consideration of the proper role and optimal distribution of power between the civil service and politicians in a regime.

Some aspects of this study could be improved over time. The sample includes only eight countries, even though the CyPRG data set includes many more. Assembling groups of experts to classify the domestic context of each country is one difficulty that must be overcome in order to expand the number of countries. Additionally, although Heady's dimensions of civil service systems are thought to range from one extreme to another in a linear fashion, the actual empirically developed relationships are, if strictly interpreted, nonlinear. Solutions, though, are possible. The dimensions developed by Heady can be reworked and reapplied. Alternatively, a nonlinear analytical method can be applied in future studies. Third, although it is not apparent in this manuscript, the authors spent a significant amount of time attempting to develop more-direct measures that distinguish national contexts. This exercise is complex, and it often results if highly collinear measures that are problematic in OLS regression analysis. The findings of indirect measures in this study have shown the usefulness of theoretical and interactive variables in empirical research. Future work should seek to combine direct and indirect measures to create a more complete picture of the effects of global pressures on domestic systems.

Despite these limitations, this analysis has two significant outcomes. First, it identifies important domestic contributions to the adoption and expression of global pressures. In some ways convergence perspectives are upheld by our findings. Government accountability, as measured by transparency, interactivity, and openness of CyPRG data, is increasing over time under the global IT pressure. Nevertheless, important domestic factors result in divergence in the rate and degree of adoption and expression of global pressures at the national level. While the global pressure of information technology leads to more government accountability at the global level, the domestic context leads to variations in the rates of adoption and expression of the global pressure at the national level.

As Kraemer and Dedrick (1997) stated regarding study of the influence of information technology at the organizational level, information technology can support different organizational arrangements (such as centralization and decentralization) that push organizational attributes (such as accountability) in opposite directions (such as control and empowerment). More often, it is the existing tendencies that determine the use and the intended impact of information technology.

The empirical research has indicated that computing per se is neither a centralizing or decentralizing influence. The context in which computing is used is a much stronger influence on whether organizations centralize or decentralize than is the technology, which can support either type of arrangement. In general, computing tends to reinforce existing tendencies, and by itself is not likely to affect organizational structure in significant ways" (Kraemer and Dedrick 1997, 101).

We hope that the study of domestic context emphasized in this research has helped identify some of these existing tendencies and their contribution to mediating the effects of the global pressure of information technology on government accountability.

A second outcome of this study is that it provides an example of empirical application of the global framework for analysis. This is the first time strictly quantitative analysis has been used to apply the framework. While this study itself may raise as many questions as it has answered, it is hoped that future research can take further advantage of the framework.
APPENDIX
Website Attribute Evaluation System (WAE System)

Transparency Measures

Transparency 1: Ownership

T1a:   Agency involvement with the site (average of webmaster within
       the agency and tailoring, both 0 or 1).
       This tests to see if the agency helped in the making of the
       website and how different it is from the websites
       of other agencies in the same government.
T1b:   Provides different webmaster within the agency from main
       government page. This tests to see if the agency, or subagency,
       has some measure of control or direct contact in regard to its
       website. Marked by a 0 or 1.
T1c:   Provides obvious tailoring indicating the agency itself has
       ownership of the site content. This tests to see how much the
       agency is involved with the content of its website.

Transparency 2: Contacts

T2a:   Provides central agency non-email addresses. This tests to see
       if the agency can be contacted by regular, non-electronic mail.
T2b:   Provides e-mail address to webmaster within the agency. This
       tests to see whether or not the agency has a webmaster within
       the agency.
T2c:   Provides e-mail address to someone inside the agency in addition
       to webmaster within the agency. This tests to see if the agency
       lists the e-mail addresses of employees and managers within the
       agency.
T2d:   Provides some kind of addresses for employees within the agency
       beyond the top echelon (e.g., shows a phone book that includes
       employees' positions). This tests to see if the agency provides
       phone numbers or addresses for the employees within the agency,
       excluding their managers and any other top-level officials.
T2e:   Provides addresses for subelements within the agency (can you
       write them a letter to this address?). This tests to see if
       the agency provides a non-electronic address for subbasements
       (such as smaller divisions) within the agency.
T2f:   Provides e-mail address to someone responsible for both content
       of the site and technical support for the site?
T2g:   Provides e-mail address only to someone responsible for
       technical support for the site?
T2h:   Provides e-mail address only to someone responsible for content
       of the site?
T2i:   Does the person responsible for technical support for the site
       appear to be with a commercial firm?

Transparency 3. Issue/Operational Information

T3a:   Provides details on senior official's experiences or vision of
       the future for the organization. This tests to see if the agency
       provides any information about the head official of the agency.
T3b:   Provides mission statement and various activities of agency.
       This tests to see if the agency provides any data as to what
       function it serves, what its goals and values are, and how it
       accomplishes these goals.
T3c:   Provides other issue related government addresses. This tests
       to see if the agency provides the addresses (URL or otherwise)
       of any other government agencies (or within the agency itself)
       whose function is related to this agency.
T3d:   Provides non-issue-related other agency addresses. This tests
       to see if the agency provides the addresses (URL or otherwise)
       of any other government agencies (or within the agency itself)
       that are not related to the agency.
T3e:   Provides issue-related and other nongovernmental information
       source. This tests for the same as T10, but the address must
       be for a nongovernment source.
T3f:   Provides organizational structure in graphic form. This tests
       to see if the agency provides an organizational graphic (such as
       a flow chart).
T3g:   Provides reports, research, laws, and regulations in an easily
       readable format on the screen. This tests to see if the agency
       has taken the time to provide an easily readable, organized
       format for reports, research, laws, and regulations.

Transparency 4. Citizen Consequences/Response

T4a:   Provides text of regulations/laws/agency research or in-depth
       explanations of requirements that are imposed on citizens and
       result from agency activities. Tests to see if the agency
       provides any data on regulations, or laws, or on research that
       the agency carries out or is related to the agency.
T4b:   Provides instructions on how to complete these actions. An
       extension of the previous attribute, this tests to see if the
       agency provides citizens with instructions, help, tips on how
       meet the requirements/regulations/laws imposed by the agency
       (such as providing instructions on how to file a tax form).
T4c:   Provides form in graphics for screen capture or copy. This tests
       to see if the agency provides any necessary graphic forms so
       that citizens may copy them to fill them out.
T4d:   Provides appeal process for decisions or address of an
       ombudsman. This tests to see if the agency provides on-line
       instructions for citizens to appeal agency decisions.

Transparency 5. Freshness/Timeliness of Data

T5a:   Notes the latest published "last updated" date (yyyymmmdd) on
       the main page or, if none, a key subordinate page, or 0 if no
       date listed on any of these pages.
T5b:   Notes the latest last updated date of the page noted in T5a
       going into View, Doc Info and noting the last update date
       (yyyymmmdd): If no published date, uses the latest from either
       the main page or a key subordinate page.

Interactivity Measures

Interactivity 1. Ownership

I1a:   Presents clickable e-mail link to webmaster within the agency.
       This tests to see if the e-mail link to the webmaster within the
       agency is clickable (a mail-to link).
I1b:   Presents clickable e-mail link to senior agency official. This
       tests to see if the e-mail link to any senior official is
       clickable.
I1c:   Provides dialog box or online form for communication to the
       webmaster within the agency.

Interactivity 2. Reachability/Contacts

I2a:   Presents clickable e-mail link to someone inside the agency in
       addition to webmaster within the agency. This tests to see if
       any other e-mail addresses provided by the agency are clickable
       (mail-to) links.
I2b:   Presents clickable e-mail link to a number of agency employees.
       This tests to see if the agency provides clickable e-mail links
       to a large portion of its employees.
I2c:   Provides an on-line issue related forum for outsider
       participation such as chat lines, and listserves. This tests
       to see if the agency provides a chat line or listserv for
       citizens, agency employees, and other interested individuals to
       discuss topics related to the agency.

Interactivity 3. Issue/Operational Information

I3a:   Presents clickable easy download of mission statement and
       various activities of agency. This tests to see if the user is
       able to easily download a list of the goals or functions of the
       agency.
I3b:   Presents clickable link to other issue-related government
       addresses. This test to see if the addresses provided are
       clickable.
I3c:   Presents clickable link to non-issue-related other agency
       addresses. This tests to see if the addresses provided are
       clickable.
I3d:   Presents clickable link to issue related other nongovernmental
       information sources. This tests to see if the addresses provided
       are clickable.
I3e:   Provides an agency newsletter, which you can get automatically
       online via a subscription (more than a pamphlet offering a list
       of reports--this has content itself). This tests to see if the
       agency has a newsletter, either hard copy or e-mail, which the
       user can subscribe to and which provides up to date information
       about the agency and its activities.
I3f:   Provides a searchable index for archived newsletters, laws,
       regulations, and requirements. This tests to see if the agency
       allows the user to search to site for old newsletters, laws,
       regulations, or anything relating to the content the agency
       provides online.

Interactivity 4. Citizen Consequences/Response Convenience

I4a:   Presents clickable link to listed subbasements within the
       agency. This tests to see if subdivisions within the agency have
       a clickable link from the main page.
I4b:   Presents clickable link to sublevels noted in the agency's
       organizational structure graphic. This tests to see if elements
       within the agency's organizational graphic (such as the area for
       president, vice president, etc.) are clickable.
I4c:   Presents clickable link to download text of regulations/laws/
       agency research or in-depth explanations of requirements,
       imposed on citizens, resulting from agency activities. This
       tests to see if the user can easily click and download the
       regulations/laws/research of the agency.
I4d:   Presents any required submission forms onscreen for clickable
       download. This tests to see if the user can easily download any
       forms needed or required by the agency for compliance with
       certain laws or regulations. Marked by adding 0.1 for each form
       available for download.
I4e:   Presents online form completion and submission. This tests
       whether the user can complete and submit a form online to the
       agency. Marked by adding 0.1 for every form available for online
       completion and submission.
I4f:   Presents an automatic response limit for response to online
       submissions. This notes whether the agency tells users how long
       it will take until they receive a response from the agency.
I4g:   Presents clickable link to appeal process for decisions or for
       an ombudsman. This tests to see if the appeals process provided
       is clickable and easy to use.
I4h:   Provides other language access to site for visitors unable to
       speak or read the language of the host country.
I4i:   Provides iconographic access to site for visitors unable to
       speak or read the language of the host country.
I4j:   Provides audio access to site for visitors unable to see the
       site.
Exhibit 2
Coding of Configurations of Civil Service
Systems and the Domestic Context

                             Single    Majority
Relation to       Ruler      Party      Party     Military
Political        Respon-    Respon-    Respon-    Responsive
Regime             sive       sive       sive

Egypt                          x
France                                    x
Indonesia                                             x
Japan                                     x
Korea                                                 x
Saudi Arabia        x
United Kingdom                            x
United States                             x

                             Plura-
                              list
Socioeconomic     Tradi-     Compe-                Corpo-    Centrally
Context           tional     titive     Mixed      ratist     Planned

Egypt                                                x
France                                    x
Indonesia                                            x
Japan                                     x
Korea                                                x
Saudi Arabia        x
United Kingdom                 x
United States                  x

Focus for         Chief     Indepen-               Minis-
Personnel         Execu-      dent                try-by-
Management         tive      Agency    Divided    Ministry

Egypt                                                x
France                                    x
Indonesia                                            x
Japan                                     x
Korea                                     x
Saudi Arabia        x
United Kingdom                            x
United States                             x

                                                  Profes-    Bureau-
                                        Party      sional    cratic
Qualification     Patri-     Party      Patro-    Perfor-    Determi-
Requirements       mony     Loyalty      nage      mance      nation

Egypt                          x
France                                               x
Indonesia                                                       x
Japan                                                x
Korea                                                           x
Saudi Arabia        x
United Kingdom                                       x
United States                                        x

                                                  Consti-
                                        Policy    tutional
Sense of           Com-      Coope-    Respon-    Respon-
Mission          pliance     ration    siveness   siveness   Guidance

Egypt                          x
France                                    x
Indonesia                      x
Japan                                                           x
Korea                                                           x
Saudi Arabia        x
United Kingdom                            x
United States                             x
Exhibit 3
Descriptive Statistics

                                            Std
Variable                      N     Mean    Dev.   Minimum   Maximum

Political autonomy           100    7.57    1.19    2         9

Qualification requirements   100    4.03    0.67    1         5

Relation to
political regime             100    3.54    0.59    1         4

Personnel structure          100    3.06    0.62    1         4

Structural complexity        100    5.79    1.25    2         8

Transparency change          100    2.32    2.93   -6.70     10.25

Interactivity change         100    2.59    2.91   -6.00      9.00

Extent of state in
socioeconomic context        100    2.73    0.80    1         4

Sense of mission             100    3.34    0.95    1         5

Sector                       100    0.42    0.50    0         1

Capital flows                100   19.53   17.99    2.10     53.50

International trade
in goods                     100   31.72   13.87    9.30     50.30
Exhibit 4
Comparison of Transparency and Interactivity Change between Sectors

                                         Std
Variable                      N   Mean   Dev.   Minimum   Maximum

Transparency Change

Externally focused sectors   42   3.04   2.81    -2.32     10.25
Internally focused sectors   58   1.80   2.93    -6.70     10.00

Interactivity Change

Externally focused sectors   42   3.35   2.42    -2.21      8.00
Internally focused sectors   58   2.05   3.12    -6.00      9.00
Exhibit 5
Correlation among Major Variables

                                                  Trans-      Inter-
                                    Structural    parency    activity
                        Autonomy    Complexity    Change      Change

Structural complexity    0.27 ***

Transparency change      0.20 **      0.36 ***

Interactivity change     0.09         0.04        0.59 ***

Sense of mission         0.37        -0.25 **     0.08       0.20 **

International
capital flows 1997      -0.24 **     -0.47 ***   -0.16       0.39 ***

International
trade in goods 1997      0.44 ***     0.25 **     0.07       0.01

Sector                   0.12        -0.04        0.21 **    0.22 **

                                     Int'l        Int'l
                        Sense of    Capital      Trade in
                        Mission    Flows 1997   Goods 1997

Structural complexity

Transparency change

Interactivity change

Sense of mission

International
capital flows 1997       -0.09

International
trade in goods 1997      -0.07      0.63 ***

Sector                    0.17 *    0.01          -0.04

* p < 0.10;

** p < 0.05,

*** p < 0.01
Exhibit 6
Regressional Results (Initial Model, Standardized Coefficients)

                         Change in       Change in      Change in
                        Transparency   Interactivity    Openness
                        (1997-1999)     (1997-1999)    (1997-1999)

Political autonomy
(low to high)             0.02           -0.32          -0.04

Structural complexity
(low to high)            -0.49 ***       -0.49 ***      -0.55 ***

Sense of mission
(low to high)             0.07           0.25 ***        0.18 **

Sector
(internal/external)       0.19 **        0.18 **         0.21 **

Open economy
(combined capital
flow 1997 and
trade in goods
% PPP GDP)                0.15           0.67 ***        0.45 ***

Model significance         ***            ***             ***

N                          100            100             100

Adjusted R-square         0.20           0.40            0.29

*** < 0.01,

** < 0.05,

* < 0.10
Exhibit 7
Regression Results
(Autonomy Model, Standardized Coefficients)

                         Change in       Change in      Change in
                        Transparency   Interactivity     Openness
                        (1997-1999)     (1997-1999)    (1997-1999)

Low autonomy                0.01           0.09            0.06

Medium autonomy            -0.08           0.22 **         0.07

High autonomy
(intercept)                -0.00 ***
COPYRIGHT 2001 Oxford University Press
No portion of this article can be reproduced without the express written permission from the copyright holder.
Copyright 2001 Gale, Cengage Learning. All rights reserved.

Article Details
Printer friendly Cite/link Email Feedback
Author:Welch, Eric W.; Wong, Wilson
Publication:Journal of Public Administration Research and Theory
Article Type:Statistical Data Included
Geographic Code:1USA
Date:Oct 1, 2001
Words:9777
Previous Article:Regulatory compliance and air quality permitting: why do firms overcomply? (Articles).
Next Article:Barriers to security clearances for gay men and lesbians: fear of blackmail or fear of homosexuals? (Articles).
Topics:

Terms of use | Privacy policy | Copyright © 2019 Farlex, Inc. | Feedback | For webmasters