Should we consider user readiness a prerequisite for information system development?

ABSTRACT

Information system development (ISD) increasingly becomes a means of organizational change and virtually always triggers a wide variety of feelings from all stakeholders who are impacted in various ways. Motivated by the stubborn progress on the success rate of ISD, this research, in contrast to the previous research, hypothesizes that user readiness underpin all other user factors involved in ISD and conducts systematic interviews of 45 diverse ISD projects to examine the responsiveness of user readiness to project-specific efforts. Our interview findings indicate that user readiness does correlate with the success of ISD although such correspondence varies according to the complexity of ISD, that the motivational aspect of user readiness responds project-specific efforts more slowly than the cognitive aspect of user readiness, and that user readiness also interferes with other kinds of readiness, such as process readiness, data readiness, and more intimately, cultural readiness. The implications derived from our research findings call for a holistic approach to promoting user readiness across ISD projects and should help researchers and practitioners refocus their efforts in search of winning strategies for ISD.

Keywords: Information System Development, Information Systems, Organizational Change, Organizational Readiness, Cognition, Motivation, User Factor, User Readiness, User Satisfaction

1. INTRODUCTION

Nearly every information system development (ISD) triggers a wide variety of feelings from all stakeholders who have diverse interests and concerns about the proposed information system (IS), albeit their feelings usually remain unnoticed by the management. Conflicts among stakeholders intensify as an ISD progresses seemingly as scheduled, but they suddenly escalate when the acceptance phase approaches (Cohen, Birkin, Garfield, & Webb, 2004; Davis, 1993). By then, it is usually too costly to address any serious issues that underpin the conflicts unless major concessions are reached. Historically, the failure rate of ISD has been much higher than other kinds of system development. Numerous reports commented on high failure rates of ISD. Representatively, the Standish Group's CHAOS survey (1999) found that 84 percent of projects are considered unsuccessful because of significant schedule delays, severe budget overruns, and/or failure to meet the expectations. Such a chaotic situation does not seem much improved. A recent survey, conducted by the US Department of Defense, reported that only two percent of its software could be used as delivered (DCITA, 2004).

Amid the challenges, this research has investigated the interrelationships among user factors in ISD in general and the influence of user readiness on ISD in particular. In an empirical approach, our research began with a literature review through which we could clarify our research hypotheses, and interviewed a variety of stakeholders who shared their observations of 45 different ISD projects implemented by different organizations. The rest of this paper is organized as follows. In the next section, we summarize our literature review. We then present our research hypotheses in Section 3, describe our investigation schema in Section 4, analyze our interview data in Section 5, and discuss implications in Section 6. Finally, we give conclusive remarks in Section 7.

2. PREVIOUS RESEARCH ON USER FACTORS IN ISD

In the literature, incorporating the concerns from all stakeholders of a proposed IS have been a major research sphere in recent years largely because ISD increasingly becomes a vital force for a business to gain competitiveness. Strategic uses of IT invoke more functional and managerial concerns than technical ones. The intense research on the impact from functional users who are involved in ISD is witnessed in various constructs of user factor. User participation, user involvement, user attitudes, and user satisfaction are among the representative examples of these constructs. Of the constructs proposed previously, user satisfaction was considered as the primary outcome of an ISD (Gutek, Bikson, & Mankin, 1984; Doll, Deng, Raghunathan, Torkzadeh, & Xia, 2004), whereas the rest were considered as influencing factors of the ISD.

Initially, the construct of user participation was introduced both to depict users' behavioral engagement and to measure users' personal partaking in ISD (Kappelman, 1995; Barki & Hartwick, 1989). It was reported that user participation in ISD led to a higher level of acceptance and a lower level of stress among users (Korunka, Weiss, & Karetta, 1993). Another report by McKeen, Guimaraes, and Wetherbe (1994) also showed a positive correspondence between user participation and user satisfaction.

Due to increased user participation, conflicts arising in ISD, especially between IT and functional staff, were widely observed (Cohen, Birkin, Garfield, & Webb, 2004). As suggested in the literature, the tension between the user and IT groups can be attributed mainly to their different interests while others, such as poor communications and a lack of competence on either side, could further worsen the situation. The negative effect of these conflicts remained significant even after conflict management had been emphasized (Barki & Hartwick, 2001). Wagner, Leana, Locke, and Schweiger (1997) proposed a cooperation model that measures user participation with two emphases, namely, the technical cooperation and the managerial cooperation. The former focuses on the quality of decision support and functional input from the user group whereas the latter aims at the importance of the agent role of users in ISD.

As an alternative focus, the user involvement construct was introduced to draw attention from researchers to the psychological engagement of users. It is measured as users' perceived importance and personal relevance of ISD (Barki & Hartwick, 1994), based on an assumption that, with or without actual participation, users could be influenced to sense the importance and relevance of an ISD. A remedial approach to addressing users' psychological engagement was to position the user attitude construct in the midst of ISD (Igbaria, 1989; Coombs, Doherty, & Loan-Clarke, 2002; Yang & Yoo, 2004). User attitude is users' unarticulated impression or their calculated judgment of the proposed IS. Interpreted in either way, however, the user attitude construct merely covers users' psychological judgment in a broader sense than user involvement.

The infusion of the organizational change perspective into ISD has redirected our understanding of user factors in ISD. Since Ives and Olson considered the organizational change theory one of the most relevant theories for "user involvement" (1984), many studies have examined such an organizational change perspective of ISD (Kuruppuarachchi, Mandal, & Smith 2002; Meyers, Sivakumar, & Nakata, 1999; Lu 1995; Newell, Swan, & Clark 1993). Notably, the inclusion of user expectation, as another factor of ISD, subsequently emerged from the organizational change approach (Hackett, Mirvis, & Sales, 1991). Early studies showed that a significant gap exists between users' evaluation of newly implemented technology and their prior-to-implementation expectations (Argote, Goodman, & Schkade, 1983), and that the efforts on equipping users with realistic expectations of a proposed IS should lead to better satisfaction with the developed IS (Ginzberg, 1981). These studies regarded user expectation as users' understanding of business issues to be addressed in ISD, operational and organizational changes, and consequent impact on jobs (Ginzberg, 1981; Hackett, Mirvis, & Sales, 1991).

In summary, it seems that all research findings about user factors in ISD have pivoted on user attitude for which user participation is considered a driving force and user satisfaction is considered a principal outcome. In addition, user attitude, user involvement, and user expectation all rely on user participation, which may likely mislead the ISD management to believe that the more user participation the better.

3. RESEARCH HYPOTHESES

Departing from the previous research, this research examines if user readiness to the resultant organizational change plays a vital role in ISD and if the user readiness construct is so profound that user participation could only make a limited contribution to it. Our assumption is that when users are ready for an ISD in both cognitive and motivational dimensions, they will positively affect its success. Therefore, the first hypothesis of this research is identified as follows:

H1: Among all user factors, user readiness is most fundamentally correlated with the success of ISD.

Inspired by the reports that show the correction to negative effect perceived by users is exponentially costlier as an ISD carries on (Boehm, 1981; Andersson & Bergstrand, 1997), we are also interested in discovering the effectiveness of managing user readiness if it has to be dealt within ISD. As discovered by some researchers (Mirvis, Sales, & Hackett, 1991; Newell, Swan, & Clark 1993), users' attitudes toward a proposed IS are formed soon after such an ISD is announced, and changing their attitudes takes a long time if it is going to happen. As ISD increasingly demands drastic changes to business operations, the caveat from Alter and Ginzberg (1978, p. 24) still sounds instrumental to us: Acceptance of the organizational change model implies comprehensive people-intensive planning at the beginning of the project. Nevertheless, when ISD is viewed as a means of organizational change, we think user readiness should be examined across organizations rather than across individual users within an organization. Therefore, the second hypothesis of our research has emerged as follows:

H2: User readiness is an organizational issue and lacks responsiveness to project-specific efforts, and thus it should be considered a prerequisite for ISD.

Our intent is to situate the user readiness construct in the midst of all user factors by investigating the implications of user readiness in a broad scope and duration. In what follows, we describe our investigation scheme, with which we have examined 45 ISD projects to attest our hypotheses.

4. INVESTIGATION SCHEMA

Before we determine what aspects of ISD to be included in our investigation, we have laid out a framework of all major factors that possess significant impact on ISD. Figure 1 shows our view of the position of user readiness and its relationships with other kinds of readiness. As indicated in the figure, we perceive that user readiness is merely a part of organizational readiness and heavily interacts with, if at all isolatable, two adjacent components of organizational readiness, namely process readiness and cultural readiness. The process readiness encompasses all explicit and implicit business operations that are enforced methodically and are executed voluntarily such that they are upheld by the organization rather than by the management. While we think process readiness crucially underlines user readiness, we consider cultural readiness, an ultimate format of user readiness, representing the mentality of user readiness, even though user readiness could be achieved without a high degree of cultural readiness. Our research, however, has focused only on user readiness without further examining the two adjacent components. By and large, infrastructure readiness and data readiness are also part of organizational readiness for ISD and often strongly impact the success of ISD, too. The tangibility of the two kinds of readiness usually draws more attention from corporate management than other intangible kinds. However, they usually are not the sources of problems, but rather are the consequences of inadequate readiness at higher layers; for example, a lack of cross-functional integration (Weiss & Anderson, 2004) often results into data disintegration.

[FIGURE 1 OMITTED]

In addition to exhibiting the focus and context of this research, Figure 1 also depicts multiple paths to the success of ISD, through which we intend to show that only organizational readiness renders a much higher success rate of ISD without extensive project-specific efforts. The framework also alerts that among other individual components of organizational readiness, user readiness is an utmost challenging task to be accomplished and therefore is extremely difficult to reach through project-specific efforts. The reason being for such an alarming notion is that without organizational readiness, individuals' readiness has to rely on perceived resultant benefits that appear increasingly less direct and less monetarily explicit. Because of the context in which we examine user readiness, the construct of user readiness in our research encompasses substantially more aspects than previously assumed.

5. ANALYSIS

Using the same set of questions and applying same interview patterns described above, our research team interviewed various stakeholders involved in 45 ISD projects. The sample space was determined neither too small to lose the diversity of sample nor too big to lose intensity of each interview. The 45 ISD projects ranged from simple transactional systems to complex informational systems, from small and medium-sized firms to some fortune 500 firms, and from non-profit to for-profit organizations. Since the questions were thorough and demanded deep thoughts, each interview resulted in a concrete report.

As summarized in Table 1, 45 ISD projects in our sample came from 22 large organizations (including 9 fortune 500 firms), 15 medium-sized organizations, and 8 small organizations. Divided in another way, 22 of them were enterprise projects and 23 were departmental projects. Note that although the number of enterprise project happened to be equal to the number of large organizations in our sample, they do not mean the same thing. To be considered as an enterprise-scope project, an ISD must have involved extensive data integration issues and provide multiple user groups with multiple business functions. Even though the rest are considered departmental, many of them were still required heavy data exchanges with the adjacent systems.

To determine which ISD projects successful or not, our research team first analyzes interview reports and synthesizes the interview data to come up with mainly two summary matrices. The first matrix highlights the characteristics of each sample ISD, including its background, size, type, scope, complexity, and purpose. The second matrix summarizes main attributes that together could portray user readiness for an ISD. When we codified the answers to our interview questions to furnish the summary matrices, we also considered a large number of examples collected in our interview reports to ensure the fairness. Afterwards, we draw a line based on the composite score of all three user satisfaction attributes. Using the same calculation, we draw a line to divide the ISD projects into two groups, namely users are ready and not ready (indicated as UR and NR in the table, respectively). As we vary the dividing score, we find the correspondence between user readiness and the success of ISD remains clear unless we set the determinant score unreasonably high or low. It should be noted that the success rate of our sample ISD projects appear quite high in part because the success criteria that are preferred to all our interviewees seemingly overlook financial overruns for reasons we will explain later.

In our sample ISD projects, nearly half of enterprise ISD projects, that is, 10 out of 22, were implemented for decision support, but only one-third of departmental ISD projects were for decision support. Nevertheless, many transactional systems were also reported to build additional capacity in order to provide a central informational system with processed data. Also notably, large organizations spent proportionately more on data integration and information collection than small and medium-sized ones. Despite of some vagueness of the criticality of user readiness to the success of ISD, our interview data reveals a pattern that user readiness does significantly correlate with the success chance of an ISD. Specifically, among 28 ISDs for which user readiness is considered adequate (UR), 21 ISD projects were considered satisfactory by users versus 7 unsatisfactory. In contrast, among those ISD projects for which user readiness were considered inadequate (NR), 7 projects were perceived satisfactory by users versus 10 were considered unsatisfactory. Undeniably, the correlation between user readiness and the success of ISD appears less apparent in enterprise projects than in departmental projects (6:5 versus 18:2), for which we have observed that enterprise projects did run into additional challenges that might diffuse the influence of user readiness. One such tough challenge could be data integration, as exemplified at one sample organization. The organization implemented a data warehouse, and 3 years after it went in production the user group still could not see much of its value while the annual cost on operating it exceeded $10 million. The system caused neither any workforce deduction nor any observable power shifting, but integration issues prevented it from operating efficiently. Such unforeseeable difficulties were actually caused by user dominance without a full consultation from both internal and external experts. In our view, this is in fact an issue of process integration.

In addition, we have noticed that our interviewees do have bias. For example, the interviewees who are in a managerial position more often speak highly of their projects even if the evidence appears vague and somewhat subjective. Besides, the interviewees who have a technical background more likely praise their ISD projects. Moreover, opinions on user participation and user involvement also differ according to the role of an interviewee. If the interviewee is a project manager, the participation or involvement from users is usually supported with the evidence that users are trained and their concerns about job security are addressed. If the interviewee is a functional user, it is referred to as close communications and corporate policies on personnel empowerment. Obviously, ordinary users lag behind on the motivational side of user readiness while managers keep pushing on the cognitive side of it.

6. IMPLICATION

With so much information collected from our extensive interviews, what lessons can we learn? Can these interviews offer any credible validation to our hypotheses? As discussed above, our interview data do not seem as pessimistic as what the past statistical reports indicated. Our explanation to such a contradiction is that our data differ from others because the criteria used by our interviewees differ from others, too. Evidently, first, the success rate of ISD can no longer be assessed in a standalone manner, especially as IS's increasingly intermingle with each other and, second, the success of ISD becomes much more difficult to quantify or even qualify as strategic uses of IT have engendered a variety of benefits to corporations.

Because individual information systems are increasingly tied to enterprise infrastructure and their direct benefits to specific groups increasingly blur, ISD progressively more demands organizational support and strategic planning. In an organization where organizational readiness is insufficient, ISD projects will inevitably consume more resources to accomplish and, as a result, such organizations will complete disadvantageously. The agility of the two aspects of user readiness perceived in our interviews is the reason in support of this assertion. The motivational readiness does not seem improvable in certain organizations where overall organizational policies, such as tenure structure and continuing education, were inconsistent with project-specific efforts. Therefore, we think that project management should carefully isolate the resources dedicated to improving organizational readiness in general and user readiness in particular from the regular project budget since the result of such effort should not be considered project-specific. We believe the more such efforts targeted to specific ISD's, the lower organizational readiness, and the less likelihood of a successful ISD. Furthermore, we think that corporate management should coordinate such efforts on improving organizational readiness; otherwise, project management will likely inefficiently spend its limited resource bringing organizational readiness up to the level suitable for taking on a challenging ISD.

Finally, our message is this: Since the success criteria of an ISD obscure, knowing the prerequisites for taking on the ISD should be vital in preventing the management from committing costly mistakes in the first place. The question now is whether we can conclude from our interview data that user readiness is a prerequisite for ISD. We think there isn't a simple yes or no answer to this question because each ISD is exceptional, but we are confident that assurance on user readiness would help appreciably increase the success rate of ISD.

7. CONCLUSIVE REMARKS

Amid the challenges to conducting increasingly complex ISD, this research has examined if user readiness can be an essential prerequisite for ISD. Departing from the previous research that has emphasized the importance of user participation and involvement in ISD, this research has positioned user readiness in the midst of all user factors involved in ISD. To establish the user readiness construct, our approach is to identify the profoundness of user readiness by incorporating other user factors into user readiness and then analyze the responsiveness of user readiness to project-specific efforts through systematic interviews of 45 diverse ISD projects. Our interview data indicate that the correspondence of user readiness to the success of ISD does exist, but varies according to the complexity of ISD, that the motivational aspect of user readiness responds project-specific efforts more slowly than the cognitive aspect of user readiness, and that user readiness also interferes with other kinds of readiness, such as process readiness, data readiness, and more intimately, cultural readiness. Based on these findings, we suggest that the ISD project management assess the status quo of user readiness as part of a project initiative and should never underestimate the amount of resource and time required to raise user readiness to the desired level.

This research points out a fresh direction for corporate management to strategically focus their efforts on improving the success rate of ISD. As ISD becomes increasingly difficult to be individually measured, the predictability of the success chance of an ISD before it embarks would help corporations avoid costly mistakes that often lead to chaotic situations. Therefore, the findings resulting from the research should serve a landmark for researchers and practitioners to redirect their efforts in search of winning strategies for ISD.

REFERENCES:

Alter, S. and Ginzberg, M., "Managing Uncertainty in MIS Implementation," Sloan Management Review, 20 (1), 1978, 23-31.

Andersson, M. and Bergstrand, J., Formalizing Use Cases with Message Sequence Charts, Unpublished master thesis, Department of Communication Systems, Lund Institute of Technology, 1997.

Argote, L., Goodman, P. S., and Schkade, D., "The Human Side of Robotics: How Workers React to a Robot," Sloan Management Review, 24 (3), 1983, 31-41.

Barki, H. and Hartwick, J., "Interpersonal Conflict and Its Management in Information System Development," MIS Quarterly 25 (2), 2001, 195-225.

Barki, H. and Hartwick, J., "Measuring User Participation, User Involvement, and User Attitude," MIS Quartely, 18 (1), 1994, 59-82.

Barki, H. and Hartwick, J., "Rethinking the Concept of User Involvement," MIS Quarterly, 13 (1), 1989, 53-64.

Boehm, B. W., Software Engineering Economics, Upper Saddle River, NJ: Prentice-Hall, 1981.

Cohen, C. F., Birkin, S. J., Garfield, M. J., and Webb, H. W., "Managing Conflict in Software Testing," Communications of the ACM, 47 (1), 2004, 76-81.

Coombs, C. R., Doherty, N. F., and Loan-Clarke, J., "The Role of User Ownership and Positive User Attitudes in the Successful Adoption of Information Systems within NHS Community Trusts," Advanced Topics in End User Computing (pp. 188-209). Hershey, PA: Idea Group Publishing, 2002.

Davis, F. D., "User Acceptance of Information Technology: System Characteristics, User Perceptions and Behavioral Impacts," International Journal of Man-Machine Studies, 38 (3), 1993, 475-486.

DCITA, "IT Governance," Data Magazine (Issue 5), 2004, http://www.dcita.qov.au/ict/publications/data_magazine/issue_5.

Doll, W. J., Deng, X., Raghunathan, T. S., Torkzadeh, G., and Xia, W., "The Meaning and Measurement of User Satisfaction: A Multi-Group Invariance Analysis of the End-User Computing Satisfaction Instrument," Journal of Management Information Systems, 21 (1), 2004, 227-262.

Ginzberg, M. J., "Early Diagnosis of MIS Implementation Failure: Promising Results and Unanswered Questions," Management Science, 27 (4), 1981, 459-478.

Gutek, B. A., Bikson, T. K., and Mankin, D., "Individual and Organizational Consequences of Computer-Based Office Information Technology," Allied Social Pschology Annual Vol. 5, 1984, 231-254.

Hackett, E. J., Mirvis, P. H., and Sales, A. L., "Women's and Men's Expectations about the Effects of New Technology at Work," Group & Organization Studies, 16 (1), 1991, 60-85.

Hitt, L. M. and Snir, E. M., "The Role of Information Technology in Modern Production: Complement or Substitute to Other Inputs," submitted for publication in Management Science, 1999.

Igbaria, M., "The Impact of User Attitudes toward Microcomputer Usage on System Usage and User Satisfaction," ACM SIGCPR Computer Personnel, 12 (2), 1989, 15-21.

Ives, B. and Olson, M. H., "User Involvement and MIS Success: A Review of Research," Management Science, 30 (5), 1984, 586-603.

Kappelman, L. A., "Measuring Involvement: A Diffusion of Innovation Perspective," Data Base Advances, 26 (2-3), 1995, 65-86.

Korunka, C., Weiss, A., and Karetta, B., "Effects of New Technologies with Special Regard for the Implementation Process per se. Journal of Organizational Behavior, 14 (4), 1993, 331-348.

Kuruppuarachchi, P. R., Mandal, P., and Smith, R., "IT Project Implementation Strategies for Effective Changes: A Critical Review," Logistics Information Management, 15 (2), 2002, 126-137.

Lu, H. P., "Managerial Behaviors over MIS Growth Stages," Management Decision, 33 (7), 1995, 40-46.

McKeen, J. D., Guimaraes, T., and Wetherbe, J. C., "The Relationship between User Participation and User Satisfaction: An Investigation of Four Contingency Factors," MIS Quarterly, 18 (4), 1994, 427-451.

Meyers, P. W., Sivakumar, K., and Nakata, C., "Implementation of Industrial Process Innovations: Factors, Effects and Marketing Implications," Journal of Production Innovation Management, 16, 1999, 295-311.

Mirvis, P. H., Sales, A. L., and Hackett, E. J., "The Implementation and Adoption of New Technology in Organizations: The Impact on Work, People, and Culture," Human Resource Management, 30 (1), 1991, 113-139.

Newell, S., Swan, J., and Clark, P., "The Importance of User Design in the Adoption of New Informational Technologies: The Example of Production and Inventory Control Systems (PCIS)," International Journal of Operations and Production Management, 13 (2), 1993, 4-22.

Standish Group, The Chaos Report, Dennis, MA: Standish Group International, Inc., 1999.

Wagner, J. A. III, Leana, C. R., Locke, E. A., and Schweiger, D. M., "Cognitive and Motivational Frameworks in U.S. Research on Participation: A Meta-Analysis of Primary Effects," Journal of Organizational Behavior, 18 (7), 1997, 49-64.

Weiss, J. W. and Anderson, D., "Aligning Technology and Business Strategy: Issues and Frameworks--A Field Study of 15 Companies," Proceedings of the 37th Hawaii International Conference on System Sciences, 2004.

Yang, H. and Yoo, Y., "It's All about Attitude: Revisiting the Technology Acceptance Model," Decision Support Systems, 38 (1), 2004, 19-31.

Myron Sheu, California State University at Dominguez Hills, California, USA

Haejin Kim, California State University at Dominguez Hills, California, USA

Dr. Myron Sheu earned his Ph.D. at Old Dominion University, Norfolk, VA in 1992. Currently he is an assistant professor of information systems at California State University, Dominguez Hills. Previously he was a systems and applications architect at the Boeing Company.

Dr. Haejin Kim earned her Ph.D. at the Rutgers University, New Brunswick, NJ in 2002. Currently she is an assistant professor of management at California State University, Dominguez Hills.

TABLE 1. SUCCESS VERSUS FAILURE ANALYSIS BASED ON THE INTERVIEW DATA.

                   Departmental Project: 23   Enterprise Project: 22
Success ~
Failure             Large   Medium   Small    Large   Medium   Small
Analysis

Information   UR     0~1     3~0      1~0      2~2     0~1
:18           NR     1~1     0~1               1~2     1~0      0~1
Transactio    UR     4~0     4~0      3~1      2~1     1~1      1~0
n:27          NR     0~0     1~1      0~1      2~3     1~0

UR: user ready; NR: user not ready