Printer Friendly

How do Indian software projects perform? A cross sectional study.

1. INTRODUCTION

IT projects have a notorious history when it comes to project success. There had been frequent reports of mismanaged software development projects in the 90s. (Pinto, 1999, Charette, 1996, Jones 1995). The surveys conducted in the recent past show an improved but still unsatisfactory scenario. The Standish Group research (2011) shows that 66% of projects are either "challenged" or downright failures, leaving 34% of projects to be considered successful. Charette (2005) suggested that 5%-15% of projects will be abandoned before or shortly after delivery. Linberg in 1999 found that 20% of software projects failed, and that 46% experienced cost and schedule overruns or significantly reduced functionality. Jorgensen and Molokken-Ostvold (2006), suggested that failure rates for software development projects could be up to 85%.

The effects of software project failure are not limited to monetary aspects alone. If the project is intended to provide a company with a strategic advantage over its competitors, its failure could have devastating results on the company's market position, as well as its ability to survive. As software companies continue to invest time and resources into the development of software, a primary area of concern revolves around how software development problems and failures can be minimized.

Growth and development of Indian software industry has gained worldwide attention and India has established her position among the market leaders in global software development. The Indian IT industry has achieved an iconic status in the Indian economy and is considered a highly significant economic growth engine in India's success. India is widely recognized as the hub of the software development of the world.

As India is projecting itself as an ideal offshore destination from the cost and execution point of view, it is important that we develop insights on the project outcome of software development projects executed in India. Based on the data collected from 305 projects, this study looks at

* How do software development projects in India fare on the outcome measures namely time, cost and quality.

* What is the interrelationship among time, cost and quality dimensions of the project performance?

* Is there a difference in project success rate depending on the characteristics of the project and organization

2. BACKGROUND

2.1. Project Outcome Measures

Project Management Institute (PMI) defines a project as "One shot, time limited goal directed major undertaking requiring the commitment of various skills and resources". A project, by definition, is a temporary activity with a starting date, an end date, defined goals and responsibilities, a budget, a plan and involvement of multiple parties. Software development projects can be looked at as a category of projects executed with the objective of developing and delivering software products (Pressman, 1997). Software development projects may include new development, modification, re-use, re-engineering, maintenance, or any other activities that result in software products.

A project is usually deemed as successful if it meets the desired requirements, is completed on time and is delivered within budget (Powell and Klein, 1996). A number of success criteria have been developed and empirically tested for IS projects. They include IS usage, user satisfaction, quality of decision making, cost/benefit analysis, team effectiveness, and project effectiveness. The triple criteria of project success--meeting cost, schedule and performance targets - have been widely used by researchers to analyze project success. The identification of these distinct dimensions of system performance illustrates that a project can be both successful and unsuccessful at the same time depending on the metric selected. One of the most popular approaches is to categorize these measures under process performance measures and product performance measures (Barki et. al 2001; Nidumolu 1995; Deephouse, 2005; Wallace, 2000; Ravichandran, 1996). Product outcome refers to measures of the "successfulness" of the system that was developed. It looks at how the software developed scores on important parameters of software quality: reliability, maintainability, easiness to use, response time, meeting the requirements, user satisfaction etc. Process outcome measures refer to the "successfulness" of the development process of the project. The focus is on completing the project within budget and within schedule. Both aspects are important as the software delivered by the project may be of high quality but the project itself may have exceeded the time and cost projections. On the other hand, well managed projects which come in below cost and time budgets may deliver poor products. The Standish group studies also define project success in terms of completion within the planned time and budget meeting the original specification. Due to the difficulty in quantifying costs and benefits, measures based on perceptions have become particularly prominent in IS literature (Nidumolu, 1996; Wallace et al., 2004).

The level of project success may vary based on the specific nature of the project as well as based on the characteristics of the software development organization undertaking the project. The Standish group reports show considerable relationship between the project success and project size, duration and team size. Brooks (1995) and McFarlan (1982) also showed how duration and team size could be influencing the project. Others factors which can influence the project success rate are "type of software developed" (Ropponen and Lyttinen, 1997) and "type and quality certification status of the organization" (Ropponen and Lyytinen, 1997, Asundi, 2001, Malhotra et.al, 2010).

2.2. Indian Software Industry

Software development in India is a $100 billion strong industry contributing around 7.5 % of the GDP and employing over 2.5 million people. The industry has registered a CAGR of around 40 % over the last 6 years. (Source: NASSCOM). Service and software exports continue to be the mainstay of the sector contributing as much as USD 70 billion. Major components of IT Services exports include Custom Application Development and Maintenance, System Integration and IT Consulting, Application Management and IS Outsourcing/Infrastructure Management Services.

Within the global sourcing industry, India was able to increase its market share from 51 per cent in 2009, to 58 per cent in 2011, highlighting India's continued competitiveness and the effectiveness of India-based providers delivering transformational benefits. While the global macroeconomic scenario remained uncertain, the industry exhibited resilience and adaptability in continually reinventing itself to retain its appeal to clients. An increasing number of IT-BPO companies continue to adopt global standards such as ISO 9001 (for Quality Management) and ISO 27000 (for Information Security). India based centres account for the largest number of quality certifications achieved by any single country. Over the years, Indian IT service offerings have evolved from application development and maintenance, to emerge as full service players providing testing services, infrastructure services, consulting and system integration. IT services exports is the fastest growing segment, growing by 19 per cent in FY2012, to account for exports of USD 40 billion

3. RESEARCH METHODOLOGY

The current research was designed as a descriptive study. The population for the study was defined as completed software development projects undertaken by software development organizations based in India. The data had to be provided by a project representative who had been part of the project from the beginning to the completion.

Triple criteria approach of measuring the project outcome in terms of time overrun, cost overrun and quality was adopted in this study.

a) Time overrun

Time overrun is an indicator of poor project performance. It indicates how much the project has exceeded the planned time. It is reported as a percentage. A negative value for time overrun indicates time underrun which is favourable.

Time overrun is calculated as

Time overrun= ( Actual Time--Planned Time) 100/Planned Time

b) Cost overrun

Cost overrun is an indicator of poor project performance. It indicates how much the project has exceeded the budgeted cost. It is reported as a percentage. A negative value for cost overrun is termed cost underrun.

Cost overrun is calculated as:

Cost overrun (Actual Cost--Planned Cost) / 100 Planned Cost

c) Quality

Quality was defined as how the software developed is rated on the following characteristics of professionally developed software: meeting the original specifications, reliability, easy to use, portability, maintainability, documentation and flexibility. The respondents were asked to rate the software developed on each of these seven dimensions on a five point Likert scale. The average score on these dimensions was taken as a measure of the overall quality of the software developed.

4. DATA COLLECTION

The survey was conducted in Chennai, Bangalore (tier I cities) Cochin and Trivandrum (tier II cities) in India. National Association of Software Companies (NASSCOM) is the most respected and recognized body of Indian software industry. NASSCOM listed companies account for over 90% of the revenue of the software industry in India. NASSCOM list of software companies in the selected cities was accepted as the sample frame for data collection. Letters were sent via email to the centre heads/HR managers of all the companies requesting them to allow their IT professionals to participate in the study. Reminder letters were sent after three weeks. 105 companies agreed to participate in the study. Data was collected from different types of projects and members in different roles but keeping the condition that only one response should be solicited from one project. The researchers distributed 1350 questionnaires to the 105 companies who agreed to participate. After two rounds of reminders, 527 filled questionnaires were collected back from 95 companies. Detailed examination of the data based on grossly missing or inappropriate values resulted in the deletion of 222 records. Thus the final data set had 305 usable records representing 305 projects from 95 companies.

5. ANALYSIS DESIGN

Project outcome measures are metric in nature whereas project and organization characteristics are categories. Pearson correlation and chi square tests were used to explore the possible relationship between pairs of project outcome measures. Based on the literature, a series of hypotheses are offered and tested regarding variation in project outcome across categories of select project and organization characteristics.

[H.sub.1]: There is a significant difference in project outcome based on the type of software developed

The following categories were included for analysis - business application software, engineering application software, web application, system software, others.

[H.sub.2]: There is a significant difference in project outcome based on the duration of the project

Three categories were considered: short (less than 6 months), medium (up to 2 years) and long (2 years and more).

[H.sub.3]: There is a significant difference in project outcome based on the number of members in the project team

Team size categories are defined as small (5 members or lesser, medium (5 to 20 members) and large (more than 20 members)

[H.sub.4]: There is a significant difference in project outcome based on the nature of the organization

The nature of the organization was defined as one of the following: Indian subsidiary of a foreign company, Indian company with international operations, Indian company with only domestic operations

[H.sub.5]: There is a significant difference in project outcome depending on whether the organization is ISO certified

Each of the stated hypotheses is individually tested with a series of one-way ANOVA tests using SPSS17.0. Wherever significant project outcome variation was seen across categories, detailed analysis was performed using pair wise t-tests to confirm the rank order of the categories based on project outcome scores.

6. DATA ANALYSIS AND RESULTS

6.1. Performance of the projects on the time and cost performance

Table 1 shows the cross tabulation of the projects on the time and cost parameters.
TABLE 1 - CROSS TABULATION OF PROJECTS BASED ON COST AND TIME
PERFORMANCE (IN NUMBERS)

                  Time Performance

Cost performance                    Underrun  No run  Overrun  Total

                  Underrun                20      14        3     37

                  No run                   9      82       41    132

                  Overrun                  8      75       53    136

                  Total                   37     171       97    305


As seen, 102 out of the 305 projects were completed within time and cost (both no run and under run included). This works out to 33.44 % which is comparable to the Standish survey figures. 17.4% of the projects suffered both time and cost overrun and the remaining 49.2% had at least one overrun. More projects had cost overrun than time overrun. Feedback from experts in the industry suggests that companies find it easier to manage time overruns by putting more staff into the projects and making them work overtime.

6.2. Exploring the relationship among project outcome measures

Pearson correlation is a bivariate measure of association (strength) of the relationship between two metric variables. Table 2 shows the pair wise correlation coefficients for the three outcome measures namely time overrun, cost overrun and quality. All the values, though not high, are significant at 1% level.
TABLE 2 - CORRELATION COEFFICIENTS AMONG PROJECT OUTCOME MEASURES

               Time Overrun    Cost Overrun

Quality              -0.243          -0.286

Time Overrun                          0.358


The values throw up interesting findings. The time overruns are likely to be accompanied by cost overruns also and vice versa. Quality has negative correlation with both overruns. Hence the argument that many projects are delayed or overspent to improve the final quality hasn't found acceptance here. The finding on quality-overrun association was further probed with subgroup analysis. The projects were categorized into three groups namely no time and cost overruns projects, both time and cost overruns projects and only one (either time or cost) overrun projects. The average quality scores of each category were analyzed with ANOVA and are reported in table 3.
TABLE 3 - ANOVA TEST RESULTS FOR VARIATION IN QUALITY ACROSS
OVERRUN CATEGORIES

           No             Only One            Both             F-
         overrun          overrun           overrun          value
           Mean    SD      Mean      SD      Mean       SD

Quality   4.2923  0.5322    4.1254  0.5337   4.0169  0.5100   5.87

           P-
         value

Quality  0.003


As seen the quality score is highest for projects with no overrun and comes down with increasing overruns.

6.3. Variation in project outcome across categories

Results of the hypotheses testing are presented sequentially.

[H.sub.1]: There is a significant difference in project outcome based on the type of software developed

Table 4 shows the ANOVA results for the variation in project outcome across categories of software developed.
TABLE 4 - ANOVA RESULTS FOR THE VARIATION IN PROJECT OUTCOME
ACROSS TYPE OF SOFTWARE

           Business           Engineering         System
         Application                             Software

               Mean    SD        Mean     SD      Mean    SD

Time            6.32  16.6         1.49  10.6     10.17  16.8
Overrun

Cost            7.58  15.9         5.63  12.0     10.34  16.1
Overrun

Quality         4.16  0.52         4.40  0.40      4.17  0.51

             Web            F-value  P-value
         Application
             Mean       SD

Time            2.87  16.9      2.2     0.059
Overrun

Cost            2.39  12.4      1.6     0.175
Overrun

Quality         4.00  0.64      4.6  0.001 **


The time and cost dimensions did not show significant differences in the mean score across various categories of software developed. But the quality score showed significance difference. Posthoc analysis revealed that Engineering applications have higher quality score than other categories. Thus it can be concluded that H1 is partly supported. The improved performance of engineering software projects may be attributed to more stable and defined requirements (Ropponen and Lyttinen,1997).

[H.sub.2]: There is a significant difference in project outcome based on the duration of the project

Standish study reports showed that longer projects are likely to suffer from increased overruns. But my findings reported in table 5 fails to identify any statistically significant variation across projects based on duration of the projects. Hypothesis 2 is rejected.
TABLE 5 - ANOVA RESULTS FOR THE VARIATION IN PROJECT OUTCOME
ACROSS PROJECT DURATION

         Short        Medium         Long        F-value  P-value
         Mean    SD    Mean    SD    Mean    SD             value

Time      6.59  16.2    3.37  17.02  4.93  12.2      1.2    0.293
Overrun

Cost      4.76  13.6    9.57  16.8   4.93  12.2      2.1     0.06
Overrun

Quality   4.16   .55    4.22   .52   4.13   .50    0.662    0.517


[H.sub.3]: There is a significant difference in project outcome based on the number of members in the project team

Will more members in the team make the performance better or the added communication and coordination costs will result in poorer performance? The findings from the study reported in table 6 shows that except for Time overrun, there is no significant variation seen. Time overrun shows significant variation (at 5% level) across team size and further analysis revealed that the large teams have lesser chance of incurring time overrun.
TABLE 6 - ANOVA RESULTS FOR THE VARIATION IN PROJECT OUTCOME
ACROSS PROJECT TEAM SIZE

         Small        Medium        Large        F-value  P-value
         Mean    SD    Mean    SD   Mean    SD

Time       6.9  17.6     5.2  14.8    0.4  13.4      3.2   0.04 *
Overrun

Cost       7.1  16.4     6.8  14.8    4.9  12.4     0.44    0.646
Overrun

Quality   4.16  0.55    4.17  0.52   4.23  0.56     0.36    0.696


[H.sub.4]: There is a significant difference in project outcome based on the nature of the organization

The results indicate that the project outcome varies across type of organizations. Big Indian firms, with international business do much better than other companies. The small local firms focusing on domestic projects are seen to be the worst in the lot. The ANOVA results are presented in table 7.
TABLE 7 - ANOVA RESULTS FOR THE VARIATION IN PROJECT OUTCOME
ACROSS ORGANIZATION TYPE

Indian                     Indian               Foreign
         Domestic        International        subsidiary
           Mean     SD       Mean        SD      Mean      SD

Time           12  15.5           0.27  10.5         8.7  14.3
Overrun
Cost         14.8  17.2            4.1  13.5         7.9  14.8
Overrun
Quality       3.8   0.7            4.4  0.47        4.01  0.50

         F-value    P-
                  value

Time        10.8  0.000
Overrun              **

Cost         4.7  0.003
Overrun              **

Quality     16.6  0.000
                     **


[H.sub.5]: There is a significant difference in project outcome depending on whether the organization is ISO certified

ISO 9001 is the creation of the International Organization for Standardization (ISO), a Swiss-based federation of national standards bodies. ISO 9001 originally targeted the manufacturing process, but now includes software development processes as well. The output of the t-test comparing means across the two categories are given in Table 8. The results show that the ISO certified companies do a much better job of completing the project on time and cost. No significant results are seen on the quality front. The focus of ISO certification is standardization and streamlining of processes which will help in improving the process outcome measures namely time and cost performance. The quality, which is a product outcome measure, is related to many factors which are not necessarily governed by ISO standards.
TABLE 8: T-TEST RESULTS FOR THE VARIATION IN PROJECT OUTCOME BASED ON
ISO CERTIFICATION

            ISO              Non           t-value  P-value
         Certified        Certified

           Mean      SD     Mean      SD

Time           4.0  15.0        8.9  17.9      2.1  0.003 **
Overrun                                              0.039 *

Cost           5.3  14.0       12.0  16.9      3.1  0.002 **
Overrun

Quality        4.2  0.55        4.1  0.50     -1.6     0.119


7. DISCUSSION AND CONCLUSIONS

The research was undertaken with respect to software development projects in India with the objective of exploring the dimensions of project success namely time performance, cost performance and quality of the final deliverable. We have seen that the performance of software projects in India, though comparable to international figures, are far from satisfactory. The success rate of 33% is nothing great for a country projecting itself as the software development centre of the IT world. The study also showed that time and cost overruns have positive correlation among them and both are inversely related to the quality. This means that time, cost and quality performance can be achieved together. One is not necessarily achieved at the expense of others. Considerable variation was observed in the project performance across different type of software companies operating in India. The big Indian companies (Tier1) have a substantially high success rate compared to others. This is good news for India as these companies account for over 75% of the software business. But number wise they are less than 10% of the total software companies in India. Thousands of small software companies operating in small towns have to improve a lot to match their big brothers. The study also showed that following ISO standards help them in managing time and cost budgets well. The engineering software had the best quality score compared to other domains. Time overrun was less for projects with more members. Project duration was not seen to be influencing any of the project outcome scores. Thus it can be concluded that there is a relationship among rime, cost and quality performance of software development projects. There are some organizational and project characteristics which will have an impact on these relationships. These findings lay the foundation for the further work in identifying the reasons underlying these relationships and variations.

REFERENCES

Asundi, J.M., Arora, A., Arunachalam, V.S. and Fernandes, R. (2001). The Indian Software Services Industry. Research Policy (30) 8,, 1267-1287

Barki, H., Rivard, S., Talbot, J., (1993). Toward an Assessment of Software Development Risk. Journal of Management Information Systems 10 (2), 203-225.

Brooks, F. P., (1995). The Mythical Man-Month: Essays on Software Engineering. Addison--Wesley Publishing Co. Massachussets

Charette, R. N., (1996). Software Engineering Risk Analysis and Management. McGraw-Hill Software Engineering Series.

Charettte, R., (2005), Why software fails. IEEE Spectrum, 42-49.

Deephouse, C., Mukhopadhyay, T., Goldenson, D. R., Kellner, Marc I. (2005). Software Processes and Project Performance. Journal of Management Information Systems (Winter 1995-96) 12 (3), 185-203.

Jorgensen, M and Molokken-Ostvold, K. How large are software cost overruns? A review of the 1994 CHAOS report, Information and Software Technology 48, (2006), 297-301.

Jones, C., (1995). Risks of Software System Failure or Disaster. American Programmer 8 (3), 2-9.

Linberg, K. R. Software developer perceptions about software project failure: A case study. Journal of Systems and Software 49, 1999.

Malhotra, R., & Temponi, C. (2010). Critical decisions for ERP integration: Small business issues. International Journal of Information Management, 30 (1), 23-37.

McFarlan, F. W., (1982). Portfolio Approach to Information Systems. Journal of Systems Management (January), 12-19.

Nidumolu, S., (1995). The Effect of Coordination and Uncertainty on Software Project Performance: Residual Performance Risk as an Intervening Variable. Information Systems Research, 191-219.

Powell, P. L., Klein, J. H., (1996). Risk Management for Information Systems Development. Journal of Information Technology 11, 309-311.

Pinto, J K and Millet I (1999), "Successful Information Systems Implementation: The human side, 2nd ed., Newton Square, PA:PMI

Pressman, R., (1997), Software Engineering: A practitioners' approach, 4th edition, McGraw-Hill Higher Education.

Ravichandran, T., (2003). Special issue on component-based software development. ACM SIGMIS Database 34 (4), 45-46.

Ropponen, J., Lyytinen, K., (1997). Can software risk management improve system development: an exploratory study? European Journal of Information Systems 6 (1), 41-50.

Standish Group International, (2011). Chaos: A Recipe for Success. The Standish Group International.

Wallace, L., Keil, M., (2000). Software project risks and their effect on outcomes. Communications of the ACM 47 (4), 68-73.

Wallace, L., Keil, M., Rai, A., (2004). Understanding Software Project Risk: A Cluster Analysis. Information and Management 42 (1), 115-125.

Sam THOMAS

School of Management Studies, Cochin University of Science and Technology, Cochin, Kerala, India--682022 sam8570@gmail.com

M. BHASI

School of Management Studies, Cochin University of Science and Technology, Cochin, Kerala, India--682022 drbhasi@yahoo.com
COPYRIGHT 2013 Academia de Studii Economice Bucuresti
No portion of this article can be reproduced without the express written permission from the copyright holder.
Copyright 2013 Gale, Cengage Learning. All rights reserved.

 
Article Details
Printer friendly Cite/link Email Feedback
Author:Thomas, Sam; Bhasi, M.
Publication:Management Research and Practice
Geographic Code:9INDI
Date:Jun 1, 2013
Words:3879
Previous Article:Integrated sewage treatment and coastal management in Mumbai Metropolitan Region.
Next Article:Key performance criteria for vendor selection--a literature review.
Topics:

Terms of use | Privacy policy | Copyright © 2018 Farlex, Inc. | Feedback | For webmasters