Printer Friendly

Blending agile and lean thinking for more efficient it development.

Governments' information technology projects don't always deliver the promised results. Combine performance problems with a reputation for being late and over budget, and it's easy to see why officials are often reluctant to take on major IT initiatives. The problem isn't inherent in government projects, however; it's caused by the traditional "waterfall" approach to project planning. And it can be minimized by using a different kind of project planning, a Lean approach known as "Agile" development. Agile development and Lean management can lead to more cost-effective, timely production of information technology that better meets users' needs.


Waterfall development has been the dominant approach among IT professionals for years. This approach is about trying to understand the customer's new needs and then working on the development for months, after which the final output is released--a top-to-bottom process. Customer feedback is given when the overall project is completed.

One study indicates that 45 percent of the features in a typical system are never used, and only 7 percent are always used (see Exhibit 1). And overall, success rates for IT development, based on a sampling of multiple publications and studies, are not good:

* Approximately 31 percent of projects get cancelled.

* Approximately 66 percent of projects don't meet customer needs and are therefore considered failures.

* More than 50 percent of projects exceed their budgets by 200 percent.

* Deliverables are not well planned or well managed.

* Project managers could use more training.

* Approximately 10 percent of the developed code was actually used.

* Approximately 82 percent of projects cite waterfall practices as the primary reason for failure.

Many articles have been written about chaos theory and how it relates to waterfall software development. This is because so many uncertainties and variables can come into play when software is developed. Also, some of the assumptions waterfall development makes are unrealistic, such as the idea that customer needs being clearly defined upon going into the project, and the needs of the client department are thoroughly vetted and agreed upon, and that the requirements can be accurately determined at the beginning of the project. Waterfall moves along a straight path, based on initial inputs and assumptions, so errors or miscalculations made at the outset of the project will be included in the final product. Waterfall also assumes that timeframes and budgets are easy to predict in the beginning; many waterfall developments fall in quarterly timeframes for progress and outputs, which is not nearly the frequency needed for reviewing progress and alignment with customer needs, and soliciting customer feedback (which is the final step in a waterfall process, after the project is completed).


In 2001, a group of 17 IT professional gathered to develop an alternative to the waterfall approach, which they called "Agile." (1) The group agreed to an "Agile Manifesto" (see Exhibit 2), which included several values aimed at making IT development more effective:

1. Individuals and interactions should take precedence over processes and tools.

2. Working software is the desired output, as opposed to comprehensive documentation.

3. Customer collaboration is superior to negotiating contracts with clients.

4. Responding to change is more important than blindly following a plan.

Exhibit 3 illustrates an example of the difference between a waterfall and an Agile approach.

The group developed the following key aspects of the system and terminology concepts (see Exhibit 4). These can be summarized as follows:

1. The overall software client needs would be broken down to segments (known as stories) that are prioritized by their importance for development; this is known as backlog grooming. The team would do the development based on the customer's prioritized needs.

2. Stories (a segment or element the backlog development needs) would then be developed. The suggested story structure is: "As a <user type> I want to <do some action> so that <desired result>" This helps the development team identify the user, action, and required result in a request, and it is a simple way of writing requests that anyone can understand (e.g., "As a user I want a tool bar on the screen so that I can easily drop in changes").

3. The prioritized sections would be broken into sections, or stories, lasting 14 weeks; these are called Sprints, and each Sprint ends with a working functional product for customer review. Each Sprint team, normally of six to ten people, would be dedicated full time. (There could be more team members, based on the complexity of the software development, but smaller teams work better, in general.) Ideally, all team members should be at the same site for ease of communication and coordination.

4. A Sprint planning meeting is then held with the assigned Sprint team to determine what details are required to perform the work for the current Sprint. This includes developing a detailed plan with elements and timeframes. Various techniques can be used to determine a consensus for the team time requirements, such as planning poker. (2)

5. The Sprint is then broken down to daily assignments for each team member. Each Sprint should include a daily review of progress, called a Scrum, for approximately 15 minutes at the beginning of each day to review what was completed the day before, what the team expects to complete on the current day, and any impediments that team members have experienced.

6. At the end of each Sprint, the customer would review the functional output and review the prioritized product backlog. The backlog priorities could change based on the output of each sprint.

7. The team leader, or Scrum master, monitors project progress, provides support to the team, and helps remove obstacles that team members may be experiencing.
Exhibit 2: The Principles of the Agile Manifesto

1. Our highest priority is to satisfy the customer through early
and continuous delivery of valuable software.

2. Welcome changing requirements, even late in development. Agile
processes harness change for the customer's competitive advantage.

3. Deliver working software frequently, from a couple of weeks to a
couple of months, with a preference to the shorter timescale.

4. Business people and developers must work together daily
throughout the project.

5. Build projects around motivated individuals. Give them the
environment and support they need, and trust them to get the job

6. The most efficient and effective method of conveying information
to and within a development team is face-to-face conversation.

7. Working software is the primary measure of progress.

8. Agile processes promote sustainable development. The sponsors,
developers, and users should be able to maintain a constant pace

9. Continuous attention to technical excellence and good design
enhances agility.

10. Simplicity--the art of maximizing the amount of work not
done--is essential.

11. The best architectures, requirements, and designs emerge from
self-organizing teams.

12. At regular intervals, the team reflects on how to become more
effective, then tunes and adjusts its behavior accordingly.




Agile is clearly aligned with the principles of Lean, which are:

1. Add value for the customer.

2. "Learn to See" using value mapping streams.

3. Create flow--the goal is "one-stop shopping.

4. Keep pace with the customer demand rate--"pull" from the customer.

5. Aim for perfection--get it right the first time, eliminating errors and rework.

Adding Value for the Customer.

This is all about clearly understanding what the customer wants to change, fix, improve, or enhance. Any IT development project should ensure that the customer has a clear, comprehensive understanding of what they want to do and what the desired outcomes are. It's not enough for a customer to describe the desired outcomes; the customer's team or department also needs to be completely on-board with the desired outcomes. Many times, this is not the case. More time should be spent on the front end of any IT software development to clearly specify what outcomes are required and to make sure that everyone involved has the same clear understanding of all the appropriate operational definitions.

Agile offers an advantage for customer deliverables that are not well defined, in that these issues will be discovered much earlier in the software development process. Because Agile breaks down the project development into key "stories" that are then prioritized, customer communications are significantly better than in a traditional waterfall process. Agile will produce the first demonstration deliverable within two to four weeks for the customer to evaluate, along with the opportunity to look at the remaining stories and priorities in the backlog that still need to be developed.


A "project charter" document can be adapted to support the Agile process (see Exhibit 5). The customer fills out the project charter and delivers it to the Agile team, providing a good process for clarifying and verifying any changes that need to be made.

Mapping Key Value Systems, or Learning to See. Too many projects are initiated without a clear understanding of how the current process operates, including all of the process wastes and inefficiencies. Nothing should be automated until the current process has gone through a Lean review to remove wastes; automating a junk process provides automated junk. Understanding the process and its outcomes happens in the actual work area, where "the action is," and not with a few high level managers having a discussion in a conference room. When managers take the time to go to the work area and understand in great detail what actually happens, they "learn to see"--which includes being amazed at how things are really done, including the amount of variation.

Creating Flow. A process should be designed with as few handoffs as possible before the software is created. Employees often say "we have to do it this way," but don't just accept that assertion; look for the facts that support it, and make sure than any existing legislation/mandates are truly valid. Tribal knowledge, along with how effectively or ineffectively training is transmitted over time, creates beliefs about why "we have to do it this way" that usually aren't true.

The first step in getting past these beliefs is to clearly understand the current state of how things are actually done, in as much detail as possible. Then, examine the current state for wastes such as errors and rework, and ask workers about their areas of frustration and any ideas they may have to improve the process. This information allows the organization to design a streamlined and effective "future state." This future state design should be tested using an iterative process called the "Plan-Do-Check-Act/Adjust" cycle. Once these steps have been taken, an Agile development process can begin.

Keeping Pace with the Customer Demand Rate, or Pulling from the Customer. Agile is aligned with customer needs by creating useable output that can be demonstrated and tested in much shorter cycle times rather than waiting many months for a waterfall software output that might have missed the customer's mark.

Getting It Right the First Time. Each step of the development is tested to ensure that what is being designed is what the customer needs. Looking at the current state error and rework rates allows Agile developers to take corrective actions and mistake-proof the new software.


Why, then, isn't Agile used more often? People, and organizations, are resistant to change. In a magazine article, Steve Denning cited a number of perceived objections to Agile. (3) Following are some on the more frequently encountered objections, followed by counterarguments:

* "Agile is only for stars." Employees are able to use and embrace Agile when they see how it works and what it accomplishes.

* "Agile doesn't fit our organizational culture." These objections can be countered by support from management; culture is driven by leadership.

* "Agile only works for small projects, and ours are big." Large projects can be divided in smaller pieces to which Agile can be applied.

* "Agile requires co-location, and our staff is geographically dispersed." Although it is ideal for Agile teams to be at the same site, modern technology such as video conferencing makes communication easy.

* "Agile lacks project management processes." Agile can be integrated with Lean, PERT, GANNT, and other process methodologies.

* "Our individual accountability systems don't fit Agile." Accountability systems are driven by management and can be changed.

* "Agile is just a fad." Agile is an effective approach to software development that combines greater discipline with creativity and execution.

* "There are better ideas than Agile." All organizations should understand and build on best practices and "steal shamelessly, but legally."

* "Nothing new here." The Agile approach is rooted in project management approaches, combined with Lean. This combination is what makes Agile effective.

* "Not a fair comparison." Agile is a mindset, not a method. Changing mindsets regarding software development open up more highly effective approaches (such as Agile) and provide opportunities to deliver better software, on or before due dates, at or less than budgeted costs.

Agile is a Leaner approach to developing software than the traditional waterfall approach, creating more feedback and thus better alignment with the customer's needs. For the same reasons, Agile decreases risk and creates less confusion and greater employee satisfaction. With Agile, organizations use much less resource time discovering the need for corrections or adjustments. Agile helps organizations deliver better and more successful projects, faster, and at lower costs. There are challenges, of course. Leaders and management need to have a clearer understanding of what Agile is, what it does, and its associated benefits than most currently do. Agile is also more successful with a greater level of communication from leaders and management. It also creates a greater need for customer interface and feedback loops. There is really no reason not to explore and embrace Agile for software development. With proper guidance and management support, the risks are extremely low, and the benefits are great.


(1.) See the "Manifesto for Agile Software Development at

(2.) Planning Poker is available at no charge from sources such as

(3.) Steve Denning, "The Case Against Agile: Ten Perennial Management Objections," Forbes, April 17, 2012.

HARRY KENWORTHY is principal and manager of the Quality and Productivity Improvement Center (QPIC, LLC), a consulting organization he founded in 1984. He was one of the first practitioners to apply Lean in the government sector, and his consulting work has included numerous government processes that have been improved by removing waste, reducing costs, or increasing revenues in a variety of operational steps while reducing overall process cycle times and improving customer service. QPIC's business is with cities, and state and federal agencies implementing Lean in government. Kenworthy worked with W. Edwards Deming in 1983-85 on a series of seminars throughout the United States, sponsored by MIT, and he has spoken at more than 90 conferences on quality, productivity, Lean, and Six Sigma, as well as writing for several magazines.
Exhibit I: The Features and Functions that are Actually Used
in a Typical System

Never       45%

Always       7%

Often       13%

Sometimes   16%

Rarely      19%

Source: Standish Group.

Note: Table made from pie chart.
COPYRIGHT 2014 Government Finance Officers Association
No portion of this article can be reproduced without the express written permission from the copyright holder.
Copyright 2014 Gale, Cengage Learning. All rights reserved.

Article Details
Printer friendly Cite/link Email Feedback
Title Annotation:Solutions
Author:Kenworthy, Harry
Publication:Government Finance Review
Geographic Code:1USA
Date:Apr 1, 2014
Previous Article:Working with FEMA: an extension of what we do.
Next Article:The IRS and you: IRS regulations, programs, and resources for state and local governments.

Terms of use | Copyright © 2017 Farlex, Inc. | Feedback | For webmasters