Printer Friendly

Questions and Answers From IRI's Members.

What are IRI members talking about? These surveys were recently posed on IRI's Community Forum

Q: What metrics do you use for your innovation process?

Assessing the value of innovation is a necessary and critical aspect of portfolio management. This survey is intended to uncover best practices to characterizing benefits proposed and realized--and the impediments to doing so--in a consistent manner.

Survey Responses

We received responses from 20 IRI member organizations. The majority of respondents (11) were in chemicals, gasses, and advanced materials; the rest were distributed among a variety of industries. Respondent companies also varied in size, reported in terms of sales; nearly half (9) reported sales of $1,001 million-$5,000 million. Just four reported smaller sales, in the range of $101 million-$1,000 million, and five reported sales of $5,001 million-$10,000 million. The remainder reported sales greater than $10,000 million.

Asked what metrics were used to measure past performance of innovation activities, respondents indicated a number of metrics; the most popular, used by nearly all respondents (19), was sales or earnings, followed by cost savings, patents, time to market, and counts of successful launches and products released (Figure 1). One respondent also indicated that their company uses a vitality index, calculated as the percent of income from new projects. For metrics to predict future performance of current innovation activities, sales projections and net present value (NPV) calculations were most common, followed by measures of pipeline robustness, risk-adjusted cost and launch date, and other metrics, including risk-adjusted contribution margin (Figure 2).

Next, we asked about the accuracy of forecasts of future profits or sales from current innovation activities. Respondents' assessment of the accuracy of forecasts ranged widely, clustering around 50 percent: seven respondents indicated actual returns were 25-50 percent of forecast, and another seven indicated 51-75 percent. Four indicated that actual yields were less than 25 percent of forecasts, and the final two said actual yields were 75 percent of forecast or better. When respondents were asked to indicate the two most significant causes of differences between forecasts and actual yields, a large number (16) indicated gaps in the initial market analysis; other reasons included timing, marketing and sales effectiveness, and problems with supply chain and marketing capabilities (Figure 3). Write-in answers included the intrinsic uncertainty in the performance of a new product, globalization, and difficulties understanding customer need.

Q: What is your vitality index?

Vitality index (new product revenues as a percent of total revenues) is a common metric to measure the innovation effectiveness of an R&D organization. Two IRI organizations ask that you share how your organization defines the vitality index, what is a typical target, and how the metric is used.

Survey Responses

We received responses from 25 IRI member organizations from a wide variety of industries, ranging in size (in terms of sales revenues) from $ 101 million to greater than $100,000 million. The largest proportion (10) reported spending 3-4 percent of sales on R&D; another 5 reported spending 1-2 percent, and 4 reported spending even less. All reported using other metrics in conjunction with the vitality index, including top-line sales revenue, new/ organic sales, operating profit, and gross profit.

The important element in calculating the vitality index is new product revenues. To find out how organizations calculate this number, we asked how long the organization considers a product new and what criteria it uses to define new. A solid majority of respondents (16) indicated that products remain new for four to five years; fewer than five each indicated ranges of one to two years, three to four years, and longer than five years. Criteria used to define new included "new to market," "new to world," "next generation," "substantive change," and "model/SKU creation date" (Figure 4). Other answers included a product requiring more than one month's FTE, offering a unique benefit to the customer, or improving the core business (beyond a simple refresh or repackaging).

Based on those criteria, just over a third of respondents indicated that 11-20 percent of their companies' annual sales came from new products; the next most common response was less than 10 percent (Figure 5). Fewer respondents indicated higher proportions of sales from new products. Target five-year vitality index numbers varied widely, from less than 10 percent to greater than 40 percent (Figure 6).

Although respondent organizations reported using the vitality index in a variety of ways, including as a business report metric and to guide R&D strategy, portfolio management, and R&D investment decisions, they also reported a number of issues with the index:

* We do not discontinue products. As our base grows, it is hard to keep the vitality index. Isn't the growth in revenues from new products a better metric in that case?

* Data collection and agreement on qualified product sales.

* The way the metric was defined originally has served us well to transition to current state; however, there are voices advocating for a new definition to support the next chapter on our innovation maturity journey.

* For some businesses, it is difficult to get exact revenues on a product basis.

* Exact definition of "substantive change" varies. For BU's with products consisting of a single chemistry where R&D work is application development, how to capture "same product-new market" sales under vitality [index].

* It can cause bad behavior (incremental innovation vs. strategic or breakthrough innovations) to hit the metric. It should be weighed against other key strategic metrics and strategic needs in the organization.

* Not sure what to do with the number. What is considered a "good" vitality index? Is higher always better?

* Consistent criteria to test whether something is considered new. Also when to start the clock for timing. For some applications, the first sale may start a long qualification period for our customer, so could be potentially a year or longer until the next sale will occur.

* How to define "new" so the PVI value is useful as well as easy to maintain/update?

* At what level (business, sub-business, product family) is the PVI data actionable?

* It could drive some poor behavior-- specifically (1) not rewarding products for selling 50 years, (2) switching customers to new products when they do not want to.

* Information is not easy to gather and becomes very time-consuming for the organization.

* Product launches occur at various times in different regions; [we] typically use first launch.

* Definition of when to start the clock; length of time makes this a long-term, lagging indicator that is impossible to impact with short-term adjustments.

* Only factors new-to-world formulations. Does not capture new applications for existing formulations. Does not capture needed support for existing products.

* Technology primarily drives vs. the business. Very manual/subjective process. Continual debate around what should/should not be counted. Huge numbers reported by competitors create pressure to count things that aren't really new.

* Be careful what you ask for. You have to be clear on what constitutes a new product or not. We've defined it as improving the core, new to our company, and new to the world. If you drive a vitality index metric without a strategic business context, you'll be tempted to count every thing, which renders the metric misleading and virtually useless.

* When big turnover generators are renewed the index is very high and vice versa.

In this space, we publish the results of mini-surveys administered through IRI's e-mail-based community forum in response to member queries. Questions and responses are selected for completeness, relevance, and broad interest; questions and freeform responses may be edited for format, grammar, and to fit the available space.

The community forum is available to IRI members via the IRI website; from the home page, select Collaboration Center, then Community Forum. For more information about IRI membership, visit our website,

DOI: 10.1080/08956308.2018.1495968

Caption: FIGURE 1. What metrics do you use to measure the past performance of your innovation processes and activities? (Select all that apply.)

Caption: FIGURE 2. What metrics do you use to predict the future performance of your current innovation processes and activities? (Select all that apply.)

Caption: FIGURE 3. What are the two largest causes for differences between predictions of sales or profits from commercialized new product offerings, and the actual results? (Select two.)

Caption: FIGURE 4. Definition and duration of "new" for new products

Caption: FIGURE 5. What % of your annual sales come from products considered new?

Caption: FIGURE 6. What is your organization's target five-year vitality index goal?
COPYRIGHT 2018 Industrial Research Institute Inc.
No portion of this article can be reproduced without the express written permission from the copyright holder.
Copyright 2018 Gale, Cengage Learning. All rights reserved.

Article Details
Printer friendly Cite/link Email Feedback
Title Annotation:FORUM
Publication:Research-Technology Management
Geographic Code:1USA
Date:Nov 1, 2018
Previous Article:[Phrase omitted]: On the Greek Origins of Innovation.
Next Article:The Importance of Management Innovation.

Terms of use | Privacy policy | Copyright © 2021 Farlex, Inc. | Feedback | For webmasters |