Applying Agile Database Techniques.

In this feature I describe how your organization can adopt the agile database techniques described in my book 'Agile Database Techniques". My expectation is that it will be difficult for many organizations to adopt agile database techniques not because there is any inherently complex about them, the problem is due in the most part to cultural inertia within an organization. To successfully adopt these philosophies and techniques you must:

Change the way you look at software development--Understand the challenges you face--Actually try it--Block non-agile co-workers--Be realistic--Let us help you become agile

1.0 Change the Way You Look at Software Development

The philosophies and techniques described in this book have several significant implications for your organization. My experience is that you must accept that:

* Everyone needs to work closely together. Software development is a communication game, and as Cockburn (2002) argues documentation is the worst form of communication available to you whereas face-to-face communication standing around a whiteboard is the best. Simple things such as co-locating the team in a single workspace, using simple tools such as whiteboards and paper, and having project stakeholders as active members of your team will improve your productivity by at least an order of magnitude.

* The models and documents are fmished when the software ships. It's an iterative and incremental world now, not a serial one. The requirements are fully defined for the release of a system, if at all, only until the software is ready to go to final acceptance testing. Until then they might change. The same thing can be said of your logical data model, if you even create one, and your physical data. These artifacts evolve as work progresses on the system and may even change just hours before the production baseline of your system is finalized.

* Power within your IT department will shift. Changes in process always entail shifts within the power structure of an organization, and Agile Data is no exception. Agile database techniques shift the way that enterprise concerns, particularly those pertaining to data, are considered. Architectural model reviews are a thing of the past because your models evolve over time and because reviews are a "process smell' in the agile development indicating that you've made an organizational mistake earlier in your project. One way to look at it is if someone is qualified to review an artifact why didn't you involve them in its initial development? Existing organizational structures, particularly those based on specialized skills or a "command and control" structure, need to be reworked in favour of a more organic structure.

* Everyone needs to become actively involved. The day of the specialist who attends reviews, or has artefacts submitted to them for review and feedback, is over. The day of the bureaucrat who merely pushes paper and has no direct influence on the creation, maintenance, operation, or support of a system should never have come about in the first place. Bureaucrats can't hide behind their onerous and prescriptive processes anymore, instead they must roll up their sleeves, work closely with project teams, and actually add value to your IT efforts. People who fight this concept are likely good candidates for re-education and in extreme cases may need to be made aware of opportunities in other organizations.

* Everyone needs to rethink their approach and beliefs. There are many thought provoking ideas presented in this book. It is possible to take an evolutionary approach to database design. There is more to modeling than the UML, or data for that matter. There are many different ways that you can approach development, one size does not fit all. Many technical issues often thought to be the domain of databases, including both referential integrity and transaction control, are also pertinent within your objects. You have many technical options available to you, and they all have their strengths and weaknesses. The point is that these ideas are likely to go against what you currently believe, or how you currently prefer to work. Get over it.

2.0 Understand the Challenges you Face

You also need to understand the challenges that you are likely to face when introducing agile data techniques into your organization. My experience is that the most difficult challenges that you will need to overcome are not technical in nature but instead are people oriented. At the risk of promoting stereotypes, I have found that experienced IT professionals, novice developers, and managers all seem to have their own unique challenges that need to be overcome. Let's look at each group one at a time.

Experienced IT professionals, often people with twenty or more years in the industry, likely:

* Haven't invested the time to understand agile software development.

* Haven't taken, or been allowed, the time to try the techniques and philosophies described in this book.

* Aren't comfortable with change, particularly change that dramatically shifts the political power structure within your IT department.

* Are convinced that their existing approaches work (which they do in some situations) so don't see the need to change their ways.

* Have had bad experiences in the past with "code and fix" (CAF) approaches. Because they don't understand agile software development they often equate it with CAF and therefore assume that agile techniques are a bad idea.

* Have some very good points that aren't well addressed by the agile community, such as data-oriented and enterprise issues, and therefore they feel that agile techniques aren't sufficient for their needs.

* Believe that their situation is unique, perhaps they work in a Fortune 50 company (although 49 other organizations are also in this situation) or a government agency, and therefore agile techniques won't work for them.

* Focus on symptoms, and not the root causes of problems within their IT organization, and therefore they haven't questioned their preferred approach to development.

* Haven't coded in years and don't realize the implications of the new techniques and tools currently being used by developers.

* Have listened to other experienced IT professionals whom they respect, unfortunately people who are also struggling with agile concepts and whom are likely to tell them what they want to hear, and as a result feel that they don't need to continue looking into agility.

* Are narrowly focused specialists, often with years if not decades of experience, and therefore have difficulty understanding the bigger development picture.

* Are likely scared that they don't fit into agile development (and very likely don't given their current skillset).

Novice IT professionals are also struggling with learning agile techniques, although they face different challenges. They likely:

* Perceive agility as meaning they don't have to model and they don't have to write documentation.

* Have very narrow experience, if any, as a programmer.'

* Don't have the experience to appreciate the bigger picture, or at least to appreciate its nuances.

* Are focused on a single technology or programming language.

Managers within your organization have their own unique issues regarding agility. They often:

* Don't want to, or can't, provide adequate resources for your team.

* Have had bad experiences in the past with new techniques and is unwilling to try yet another one.

* Believe that agile software development is another fad and will go away after a few years.

* Don't realize that an agile development team requires other parts of the organization, including both IT groups such as enterprise architects and data management/ administration, to work in an agile manner as well when they interact with the team.

Everyone needs to approach agility with an open mind, ideally without any preconceptions. They need to look at the big picture, recognize that they have serious problems that aren't going to go away unless they act. Ideally everyone needs to work with, and be mentored by, experienced agile developers in order to learn these new approaches. Although it is possible to bootstrap your project team, and even your entire organization, into agile software development you are much better advised to seek the help of people who have gone before you. Education and training in agile software development is also important although not as critical as good mentoring.

Experienced developers and managers need the opportunity to try these new ways, and more importantly be given the time it takes to break themselves of their "bad" non-agile habits. Novice developers need to focus on education/ mentoring as well as simply gaining experience. Novice developers will learn agile techniques easier than experienced developers will as they have less baggage to discard along the way.

3.0 Actually Try It

There is a sigificant difference between theory and practice. You can read about something all you want, but until you try something you truly won't understand it and its implications. At some point you're going to have to get your feet wet and try this stuff out on a real project.

4.0 Block Non-Agile Co-workers

A common strategy is to start with a pilot project that tries the new techniques in practice so you can gain some insight into how they'll work within your organization. Although this enables the individual team to be agile the challenge is that the rest of your organization is still following your existing, non-agile approach. The implication is that they may expect certain models or deliverables to be delivered at specific times in specific formats, or they may require status reports or other management artifacts, or they may require you to follow their non-agile procedures. Ideally you should negotiate away the need to venture into this bureaucratic morass, realistically you often can't and therefore you put your pilot project at risk of failure (exactly what many of the bureaucrats are hoping for).

One way to overcome this problem is to assign one or two people on your team to be a blocker. In North American football the primary goal of a blocker is to prevent the other team from sacking your quarterback or tackling your-receivers, either of which would cause your play to fail. In software development a blocker attends the meetings of the bureaucrats and produces the artifacts that they require, freeing up the rest of the team to focus on the activities that actually lead to building your system. The blockers effectively implement a "process facade" around your team that makes it appear to the rest of the organization that your team is following their existing procedures. This satisfies the bureaucrats, yet prevents them from meddling with the people that are doing the real work. Although it sounds like a wasted overhead, and it is because it would be far more effective to divert both the blockers and bureaucrats to efforts that produce something of value, the advantage is that it enables the rest of the team to get the job done. The role of blocker is often taken on by your team's project manager or coach, although in the past I have let this be a revolving role on the project so as to spread out the pain of dealing with the paper pushers.

5. Be Realistic

The following list describes several important factors that you need to consider when bringing agile database techniques into your organization.

* Be patient, it will take a generation. I believe that the adoption curve for agile software development, including agile database techniques, will be similar to that of object technology. Object technology was first being considered within the business community in the late 1980s and at the time of this writing many organizations, often referred to as the Late Majority or Laggards on the technology adoption curve (Moore 2002), are still struggling with adopting object orientation (00). That's fifteen years by my watch, and I expect no different for agile software development (the implication is that it may be 2020 before your organization finally catches up).

* Don't be too fervent. The surest way to turn someone off of agile techniques is to try to convince them that agile is the only way. Remember the sixth philosophy of Agile Data--recognize that you should strive to find the appropriate sweet spot between extremes. The implication is that you might not become as agile as you'd like right away, but you very like could become more agile than you currently are.

* Don't go in blind. Reading this book is a good first step towards becoming more agile in your data-oriented activities, but it's only the first step. Do some more reading before you try to adopt the techniques and philosophies described in this book, www.agilealliance.org and www.agilealliance.com are great resources that you should take advantage of. Talk with people who are following agile techniques on their own projects to discover what they've experienced. Get involved with the agile data community, www.agilledata.org which lists good online resources, and share your own experiences.

* Don't underestimate the polities. Process is politics, and when you change the process as radically as I describe in this book many people will obviously take issue with the necessary changes.

* Be prepared to find work elsewhere. Your organization may not be ready for agility, and worse yet may not be for some time. The implication is that you have a very difficult decision to make - do you continue to wait and hope that an opportunity arises within your organization, do you try to create such an opportunity, or do you update your progress.