My struggle at the frontline of Japanese enterprise IT, Part I. How Japan's enterprise IT has failed to learn from its most competitive industry--manufacturing.

Whether you are an expatriate who needs to interact extensively with local IT departments, or a foreign consultant who works intimately on IT projects for Japanese firms, you may well find a few surprises during your engagement. To name but a few: favoring custom-made development over the adoption of package software, devoting meticulous effort to the cosmetic detail of system screens instead of business process innovation, unclear user requirements and vague project management, and, not to mention, there is a low capability within IT organizations to align themselves with business needs. In fact, Japan's Enterprise IT industry, which constitutes dozens of "Slers" (System Integrators), software vendors and corporate IT departments, employs more than half a million workers and generates an annual revenue of almost US$140 billion. Second in size only to that of the United States, it has long been suffering from decreasing profit margins, low moral among engineers, an inability to attract younger talent and has limited innovation that can contribute to the advancement of global IT.

The spirit of making things

After I completed my graduate engineering studies at Stanford in the early 90s, I was initially hired by a Japanese manufacturer to design machines and conduct research work in the field of metal-forming. From here, I was assigned the role of quality inspector, which saw me travel extensively throughout Japan and South-Eastern Asia, conducting qualifications for factory operations. During this time, I was as amazed to discover "Monozukuri" (the spirit of making things). A good example of Monozukuri can be found in Toyota, who for decades has has been leading the manufacturing world with their 'Toyota Production System', better known in the West as 'lean manufacturing'. Electronics and hi-tech component manufacturers have not only been releasing innovative products to drive the global manufacturing industries, but also have long been practicing high quality standards that have inspired methodologies, such as the 'Lean Six-Sigma', which is a continuous improvement methodology for manufacturing and other processes. Today, these types of methodologies have been widely adopted by global companies well beyond the manufacturing shop-floor.

I was amazed again, in quite a contrasted manner, when I started to focus my career around the enterprise IT industry from 1997. I have, since then, felt like I've been engaging in battles against puzzling inefficiency (see figure 1).

In this series of articles, I intend to discuss the culture of enterprise IT projects, practices between IT vendors and user organizations, and demonstrate typical patterns of how IT departments interact with business units in Japanese organizations.

I cannot help asking: Why have Japan's strengths and its competitiveness in manufacturing not been extended to the Enterprise IT industry?

Failure to admit uncertainty

One may argue that comparing "Monozukuri" to IT projects that are aimed at the construction of business application systems is like comparing an apple to an orange. Figure 2 indicates the major difference is that IT projects tend to entail a higher degree of uncertainty. However, this does not mean that the principles of "Monozukuri" cannot apply.

To illustrate, Mr. Ohno, the father of the Toyota Production system mentions the following in his book: "In this world, matters do not quite go according to plan; hence it is inevitable that plans need to be revised frequently based on feedback of actual progress." Based on this seemingly simple insight, some of the world-leading manufacturing systems were developed.

For example, replenishment 'Kanban', is a system to dynamically trigger goods delivery from suppliers based on actual progress of material consumption at the production line. The total actual volume of delivery does not need to perfectly match the blanket order (Naiji), which is based merely on a plan. This system is designed to explicitly allow for uncertainty, and hence to tackle it by execution in an agile and flexible manner. Toyota understands the limitation of planning and recognizes that it is inadequate to just keep focus on revising inaccurate plans. Toyota has worked with its vendors to develop a systematic way to deal with uncertainty and has established a world-leading methodology in the process.

Execution adaptability over planning accuracy

When we turn to enterprise IT, there are also scientific project management techniques developed to manage uncertainty.

For example, EVM (Earned Value Management), is a method to evaluate the actual progress of a project and convert it to monetary terms, thus, allowing vendor and users to conduct better risk management. EVM is a standard requirement for all Defense contracted projects in the US. The Japanese Ministry of Economy, Trade and Industry  (METI) have been promoting the use of EVM for years with limited success. One reason for EVM's lack of popularity in Japan lies in the denominator of equation. In fact, according to the 'White Paper on Information Technology Services Industry 2006', 80% of the IT projects are planned and estimated based purely on the 'hunch' of an experienced individual, without the application of any scientific management method. Less than 20% are estimated with Work Breakdown Structure (WBS) or Functional Point (FP) methods, which are commonly used in the western world.

One may be tempted to ask: how accurate is planning based on a 'hunch'? I was involved in an experiment at a top SIer in Japan. The most seasoned project management professionals of the company were gathered to estimate the cost of the same project scope based on their experiences. It turned out that the variability of the estimation spread a spectrum of 1 to 7 times of the actual cost!

Another related concept is Time and Material (T & M) contracts for IT services. This means that vendors charge the customer based on the actual hours of services and materials spent on the project. Interestingly, almost all the foreign IT project management professionals, whom I have met, have struggled to negotiate contracts on a T & M basis with Japanese firms.

Instead, most projects in Japan are contracted on a fixed fee basis. Often, the scopes and objectives are vaguely specified at the planning stage. It is not uncommon that a fixed fee contract gives the customer organization the wrong impression, implying that they are no longer responsible for the progress of the project and its ongoing budgetary requirements, and they are legitimate to leave all the decisions to the vendor. This phenomenon is known as "Marunage" or "being thrown at". As a result, a lot of inefficiency occurs because of renegotiations and re-planning efforts during implementation. In other words, this signifies the lack of an explicit structure between vendor and customer to mutually tackle uncertainty in execution.

Typical answers when I am asked why T & M is not adopted in Japan are:

* The Japanese budgeting process has a fixed cycle and so once the budget is approved it cannot be changed easily.

* T & M means that the vendor is not taking any risk.

None of these should be unique to Japan! It is not easy to change an approved budget in any country and no customer should accept a vendor that pushes all the risk back onto them.

In fact the essence of T & M is a structure that explicitly demands IT vendors, IT departments and business users to work as a team, set milestones, and commit to the execution of project tasks.

The lack of popularity and acceptance of methods, such as EVM and T & M, indicates that there is a need for close partnerships between vendors and customers in order to tackle any uncertainty. The complexity of today's IT projects proves that it is not yet well understood. This is quite a contrast to the manufacturing world, which has systems like Toyota's 'replenishment Kanban' that enables vendors and manufacturers to work together to control variability and to flexibly adapt to uncertainty.

IT projects have learnt from Toyota

In the US there are advances in software engineering and project management methodology, all inspired by the Toyota manufacturing process. Examples are 'Agile development' and 'Lean Project Management'.

Glancing at the Agile Manifesto, one would easily identify with Japanese business practice. "Individual and interactions over processes and tools", "Working on software over comprehensive documentation", "Customer collaboration over contract negotiation" and "The most efficient and effective method of conveying information is face-to-face conversation". Last but not least, "Responding to change-over following a plan" is very much in contrast with the "Waterfall" style of project management typically carried out in Japan (See figure 3).

[FIGURE 3 OMITTED]

'Lean Project Management' focuses on the elimination of MUDA (or waste). It's an analogy for Just-In-Time' manufacturing that advocates the reduction of 'work-in-progress' and the inventory of a production line. 'Lean Project Management' promotes the Just-In-Time creation of project documents such as test-cases, specifications and test scripts. Not too early and not too late. Documents and tasks that are created too early would lead to MUDA, because requirements may change over the course of the project. In essence, just like a Kanban system that triggers the upstream process from the actual progress of the downstream, 'Lean Project Management' calls for the 'pulling' of project tasks at the time it is needed.

"Cost estimate please. Oh, there is no project scope yet."

Despite the fact that these methodologies originated in Japan, surprisingly, there are tremendous difficulties in executing them within the Japanese enterprise IT industry. Take the example of a Business-to-Business (B2B) purchasing project that I know of. At the time this was a new business model and hence, business requirements were likely to change as the nuts were sorted from the bolts during the implementation process. 'Agile Development', in this case, was considered appropriate, particularly for this kind of project.

Alas, while the project manager tried his best to recommend 'Agile', it was adamantly rejected by his superior. The reason given was that they, the customer, had forced the IT vendor to release the cost estimate despite there being hardly any scope definition. Under the circumstances, the vendor was prompted to use the 'Waterfall' method of project planning. Just based on a few assumptions, 'Waterfall' allows for an easy estimation of a project's effort and cost. It is also easier for the customer to understand. Despite potential inaccuracies, simplicity has ruled, and thus 'Waterfall' is widely used.

However, agreement based on 'Waterfall' in such cases is likely to lead to many renegotiations and replanning at the downstream process, resulting in tons of MUDA or wasted efforts. There are well-established and proven methods to effectively cope with changes, but they are not well applied in practice.

In a culture of excess communications and consensus, activity cost for an industry that, by nature, necessitates tackling high uncertainty, is especially acute. I believe that such an inefficient practice is, in fact, hurting the national competitiveness of Japan. Moreover, when applying the Toyota manufacturing principles to classify enterprise IT activities, one could easily conclude that the majority fall into the category of 'Non-value-added' (See figure 4).

For expatriates and foreign consultants who have been used to a more efficient way of working with IT departments and vendors, the good news is that you don't have to enforce a top-down preachy style of management, which could likely result in resistance. You only need to ask Japanese IT professionals to take a closer look at what this country has been doing the best-manufacturing at the shop floor.

[FIGURE 4 OMITTED]

James Mok, MSc, MBA, BEng, a Hong Kong-born Chinese, arrived Japan as a researcher at Institute of Industrial Science of Tokyo University in 1992, after completing graduate studies at Stanford, CA and Bristol, UK. He spent 5 years working as an engineer at the shop-floor of a Japanese manufacturer before switching to a enterprise IT career, hoping there is more money to make. He ended up as a rolling stone, switching identities as e-commerce consultant of a Japanese SI company, ERP consultant at a US consulting company, IT manager of a global manufacturer and then started the Japanese subsidiary of a US software vendor making its way into the "scared frontline"--the Japanese manufacturing shop floor.

[ILLUSTRATION OMITTED]

Technical Director, Apriso Japan

[e-mail]

Monozukuri                        Enterprise IT

TPM, TQC, 5S etc major process    Almost no process management
  management innovation           innovation.
  originated from Japan.
Active participation of managers  Project managers and IT vendor
  in frontline operations.        management frequently have low
                                  expertise and 'shy' away from any
                                  hands-on involvement.
Designers and production workers  Engineers and users make a huge effort
  communicate almost by "thought  in communication, but the results are
  transference".                  still full of gaps.
Many 'Master' status experts.     'Master' status engineers and experts
                                  are almost non-existent.
Special sentiment to the making   Tendency to regard intangibles, such
  oftangible 'things'.            as services, to be free.

Figure 1: Thoughts during my 2 careers as an engineer and IT manager in
Japan.

Building a bridge                      Enterprise IT

Building a bridge tools needed for     Development tools, DB, OS and
  construction are almost fixed.       peripheral systems change during
                                       the building process.
The majority of materials and their    Components of a system such as
  usage can be decided in advance.     hardware, 3rd party software and
                                       programs are likely to be
                                       modified, customized or upgraded
                                       during the building process and
                                       thus are difficult to plan in
                                       advance.
It is clear that the objective and     The usage and objective of a
  usage of a bridge is to connect two  business' system are very complex
  points. Thus, specifications and     and continue to evolve during the
  user expectation can largely be set  building process.
  in advance.

There is higher uncertainty involved in building a system than building
a bridge!

Figure 2: Differences of bridge-building and system-building