Making the most of the software development process.

Organisations are under increased pressure to look at development initiatives from a return on investment perspective, and failure in software applications can potentially have dire consequences for their overall business strategy. Delivering quality IT solutions and successful projects is about consistency and requires more than just general expertise. Specific knowledge of how tasks are performed in software development environments is crucial to the success of projects-not just for current operations, but also to support future strategy.

A software development methodology should be based on industry "best practices" but should also be tailored for the project, organization and specific team. Implementing an internationally recognized standard such as Rational Unified Process (RUP), is a complex task and there is too much in standard "out of the box" solutions to be effective in an organization.

However, in a recent survey it was found that of those companies who claim to use a software development process, 48 per cent followed an in-house process as opposed to a branded process based on best practices such as RUP or Dynamic System Development Method (DSDM). But how can a company measure whether its in-house process is appropriate and if its software development methods and techniques are up to scratch?

There are several ways to assess, the sophistication of an organisation's software development methods and techniques Of these, perhaps the most widely used is the Capability Maturity Model or CMM. Table 1. provides a definition of the various levels of sophistication and allows the measurement of one organisation against another:

As a rough guide, most software development organisations would like to be at, or would be satisfied with level 3; fewer reach level 5. However, based on observations, most organisations that do have a process flip-flop between levels 2 and 1. Those with no process at all would undoubtedly remain at level 1, with the emphasis on individual effort and heroics for any degree of project success. Another way to assess an organisation is to measure against standards such as those from ISO or against the key quality drivers in published processes such as RUP or XP (Extreme Programming). The comparison of an in-house process against another process/standard enables measurement of how well that process addresses issues commonly regarded as best practice. RUP claims to represent 'best practices in software engineering'. XP, on the other hand, claims to represent a faster, more efficient way to write software with fewer rules and less documentation. Since these two methodologies probably represent two ends of the current process spectrum, it is appropriate to compare in-house processes with them. Yet regardless of which software development process is used, its purpose is to increase the chances of success and to enhance the likelihood of subsequent projects also being successful as a result of lessons learned. However, this is not how many people perceive development processes and the result can sometimes appear to be merely recipes for document template completion or a time-table for an endless series of meetings.

To ascertain whether a development process addresses real software issues or not, it is important to first look at why a project fails. Then, how the process ultimately impacts on this can be examined. A quick search on the interact for "Why do software projects fail?" leads to numerous pages of well-intentioned advice.

The Standish Group www.standishgogp.co is one organisation that specialises in rooting out the source of software project problems.

The following sections are an extract from the Chaos Report which provides the top-ten reasons for project success and failure. It also provides definitions as follows:

* Project success: The project is completed on-time and on-budget, with all features and functions as initially specified

* Project challenged: The project is completed and operational but over-budget, over the time estimate, and offers fewer features and functions than originally specified

* Project impaired: The project is cancelled at some point during the development cycle

Given these definitions, it is interesting to look at what the research showed:

"The most important aspect of the research is discovering why projects fail. To do this, The Standish Group surveyed IT executive managers for their opinions about why projects succeed. The three major reasons that a project will succeed are user involvement, executive management support, and a clear statement of requirements. There are other success criteria, but with these three elements in place, the chances of success are much greater. Without them, chance of project failure increases dramatically.

"The survey participants were also asked about the factors that cause projects to be challenged"

But why include this research from the 1994 Chaos Report here? There are two reasons: firstly, it is clear that software projects still fail, and secondly that they are still failing for the same reasons despite the fact that technology has changed and the industry is another decade older. It stands to reason, therefore, that either current software development processes are not working or they are not being used properly.

But how can an organization assess whether its process is up to scratch? Having looked at the reasons why projects fail, the next step is to examine how if the current process helps to address these issues. To get the ball rolling, two examples of 'branded' processes, RUP and XP, should be examined to see how they address the issues identified by the Standish Group. In-house processes can then be compared to examine whether the current process measures up, or if they simply generate paper.

RUP is sometimes perceived as being a bit on the heavy side, especially when compared with processes like XP. However, Rational claims that it is based on Best Practices and as such it should readily attach all reasons why projects fail. The following is a list of RUP best practices and the success factors they address.

Rational Unified Process-Best Practices in software engineering?

Of the major success factors, the only ones not easy to ascribe to RUP are executive management support, competent staff, ownership and hard-working and focussed staff. -However, any project team with an existing process will benefit if these attributes are present in the project, and there is certainly no reason why a RUP project team should not have, and benefit from, such support.

The Rules and Practices of Extreme Programming (XP)*

The following tables provide a summary of the rules and practices of XP.

Naturally, the full version of the process provides significantly more detail than space allows for here. There are quite a few gaps in the right-hand columns where it may not be obvious which success factor is being realized in either process. Sometimes this is because the technique or practice is unique to the methodology, or because it describes a certain implementation technique such as RUP's Component-based development or XP's pair programming. Regardless, it is pretty clear that most, if not all, the factors that influence success are enshrined in both these methodologies. In summary, comparing like-with-like is clearly not easy when is comes to software development methodologies.

But in order to assess an in-house process the following areas should he addressed: does it address all the success factors and have an answer to all the project killers? Does it have more in common with RUP or XP? If it contains both it probably resembles RUP more closely. On the other hand, if it is less prescriptive, there may be more similarities with XP since it is clearly less formal and is inclined to be aimed at programmers rather than project managers.

One thing that is beginning to emerge from studies of this sort is that the details of the development process are probably less important than following 'a process'--any process, which case, an organisation's in-house process should serve it well if it is applied as intended. As long as it contains plenty of end-user involvement, has the full support of executive management and demands a clear, stable and complete definition of requirements, then the process stands every chance of resulting in a successful software project In summary, comparing like-with-like is clearly not easy when is comes to software development methodologies. But in order to assess an in-house process the following areas should be addressed: does it address all the success factors and have an answer to all the project killers? Does it have more in common with RUP or XP? If it contains both it probably resembles RUP more closely. On the other hand, if it is less prescriptive, there may be more similarities with XP since it is clearly less formal and is inclined to be aimed at programmers rather than project managers. One thing that is beginning to emerge from studies of this sort is that the details of the development process are probably less important than following 'a process'--any process. In which case, an organization's in-house process should serve it well if it is applied as intended. As long as it contains plenty of end-user involvement, has the full support of executive management and demands a clear, stable and complete definition of requirements, then the process stands every chance of resulting in a successful software project.

Dunstan Thomas Consulting: Dunstan Thomas Consulting Services provides software architecture consulting and related services such as software development process consulting and OOAD mentoring. Since 1986, we have partnered with companies of all sizes and across a wide range of industries to deliver technical and business architectural analysis, creative architectural and software design, project management, development process consulting, and technical and professional expertise.

*Copyright Don Wells 1999 (http:// www.extremeprogramming.org / rules.htn-d)

"Learning Perl Objects, References, and Modules" "Learning Perl," referred to as the "Llama book," is the classic introduction to programming in Perl. Its pages have instructed many a novice in the use of Perl for short and medium programs--which, traditionally, is much of the programming done in Perl. Yet, once having mastered the material in "Learning Perl," there have been a few bold programmers who have stopped to ask, "Where do we go from here?" Perhaps they want to write programs longer than one hundred lines. Or perhaps they're interested in working with larger and more complex data structures. Sadly, there hasn't been a clear answer to their question, until now. "Learning Perl Objects, References, and Modules," by Randall Schwartz with Tom Phoenix--the authors of "Learning Perl"--picks up where the first book leaves off. "Learning Perl Objects, References, and Modules" offers a gentle introduction to the world of references, object-oriented programming, and the use of Perl modules that form the backbone of any effective Perl program. Each chapter in the book is designed to be small enough to be read in an hour or two. Each chapter ends with a series of exercises to help readers practice what they've learned with answers in an appendix for easy reference. Assuming a basic understanding of Perl, this book explains how to effectively use both standard and object-oriented Perl modules, how to use namespaces and packages properly, and how to use references to build powerful data structures. The topics include:

- Packages and namespaces

- References and scoping

- Manipulating complex data structures

Perl is a different language to different people. It is a quick, scripting tool for some and a fully featured object-oriented language for others. It is used for everything from performing quick global replacements on text files to crunching huge, complex sets of scientific data that take weeks to process. Perl is what you make ofit, but regardless of what you use Perl for, "Learning Perl Objects, References, and Modules" will help you do it more effectively, efficiently, and elegantly. Chapter 3, "Introduction to References," is available free online at:

http://www.oreilly.com/catalog/ lrnperlorm/chapter/index.html

Level           Description

1. Initial      Software process is ad hoc and occasionally chaotic.
                Few processes are defined and success depends
                upon individual effort and heroics.

2. Repeatable   Basic project management processes are established
                to track cost, scheduled and functionality. The
                necessary process discipline is in place to repeat
                earlier successes on projects with similar
                applications.

3. Defined      The software process for management and
                development activities is documented, standardised
                and integrated into the organization. All projects use
                an approval, tailored version of the process.

4. Managed      Detailed metrics of the software process and product
                quality are collected. Both the software process and
                products are quantitatively understood and controlled.

5. Optimised    Continuous process improvement is enabled by the
                quantative feedback from the process and from
                piloting innovative ideas and technologies.

Table 1.

    Project Succes Factors                      % of Response

1. User Involvement                         15.9%
2. Exclusive Management Support             13.9%
3. Clear Statements of Requirements         13.0%
4. Proper Planning                          9.6%
5. Realistic Expectations                   8.2%
6. Smaller Project Milestones               7.7%
7. Competent Staff                          7.2%
8. Ownership                                5.3%
9. Clear Vision and Objective               2.9%
10. Hard-Working, Focused Staff             2.4%
Other                                       13.9%

Project Challenged Factors                  % of Response

1. Lack of user Input                       12.8%
2. Incomplete Requirement &
  Specification                             12.3%
3. Changing Requirements &
  Specification                             11.8%
4. Lack of Executive Support                 7.5%
5. Technology Incompetence                   7.0%
6. Lack of Resources                         6.4%
7. Unrealistic Expectations                  5.9%
8. Unclear Objectives                        5.3%
9. Unrealistic Time Frames                   3.7%
10.New Technology                           23.0%

RUP Best Practices           Success factors addressed

Develop iteratively          User involvement, proper
                             planning and smaller project
                             milestones

Manage requirements          Clear statement of requirements,
                             realistic expectations, clear vision
                             and objectives

Use component
architectures

Model visually (UML)

Continuously verify
quality                      Small project milestones

Manage change                Clear statement of requirements,
                             clear vision and objectives,
                             realistic expectations, executive
                             management support

Planning                               Success factors addressed

User stories are written.              User involvement
Release planning creates the           Proper planning, smaller project
schedule.                              milestones
Make frequent small releases.          Smaller project milestones
The Project Velocity is measured.      Proper planning
The Project is divided into
iteration.                             Smaller project milestones
Iteration planning starts each
iteration.                             Proper planning
A stand-up meeting starts each
day.                                   Proper planning ownership

Designing                              Success factors addressed

Simplicity?                            Clear statement of requirements,
                                       clear vision and objectives

Choose a system metaphor
Use CRC cards for design
sessions.
Create spike solutions to reduce
risk.
No functionality is added early
Refactor whenever and wherever
possible.

Coding                                 Success factors addressed

The customer is always available.      User involvement
Code must be written to agreed         Competent staff
standards.
Code the unit test first.
All production code is pair
programmed.
Only one pair integrates code at
a time.
Integrate often.
Use collective code ownership.         Ownership
Leave optimization till last.
No overtime.                           Hard-working, focused staff

Testing                                Success factors addressed

All code must have unit tests.
All code must pass all unit tests      Ownership
before it can be released
When a bug is found tests are
created.
Acceptance tests are run often and
the score is published.