Printer Friendly

Designing a best-in-class innovation scorecard.

During the first half of 2009, the Research & Technology Executive Council of the Corporate Executive Board interviewed and surveyed R&D executives (1) from over 100 global companies (2) about the biggest challenges facing them today, particularly related to managing R&D portfolios during a period of heightened uncertainty. One thing became clear from these conversations: R&D project valuation continues to vex even the most progressive organizations. To address this ongoing challenge, leading organizations are revisiting the way in which they measure and select R&D projects--namely, they are tweaking, if not redesigning, their innovation scorecards.

The "Top Ten List" below summarizes what we learned from these conversations and provides some quick tips for R&D and strategy executives who wish to improve the quality of R&D project selection and portfolio balancing in their organizations.

1. Get agreement across your organization on what constitutes "breakthrough" and "incremental" projects.

When balancing your R&D portfolio, you probably look at the percentage of "incremental" versus "breakthrough" projects, among several other parameters. But if everyone in your organization is defining breakthrough differently, how do you know how much you are truly innovating?

Before you begin selecting criteria for your scorecard, clearly define your project classifications by a few critical business outcomes. We spoke with one progressive chemicals company that defines its classifications according to three critical metrics: IP protection, Peak Year Sales, and Marginal Income. Every company needs to decide which metrics are most important for its organization, and once you come to a consensus on what constitutes a breakthrough project, it is much easier to design an innovation scorecard that will help spot these projects early and facilitate more accurate portfolio balancing.

2. Ground your scorecard in the business perspective.

Central R&D groups often become so focused on creating standards that they end up with scorecards or processes that ultimately have a generic fit across multiple diverse businesses. But the goal of creating a standard innovation scorecard isn't to find the lowest common denominator but to help companies make better R&D prioritization and funding decisions. This always needs to be fundamental to scorecard design. Companies should work closely with their business partners to ensure that they can meaningfully assess project value and help drive better decisions.

3. Don't ignore project interdependencies.

Project scorecards usually assess the attractiveness of individual projects as independent workstreams. When designing scorecards and decision-making processes using scorecards, companies must account for project interrelationships and interdependencies. For example, while certain projects may not appear profitable at the surface level, they may be crucial in forming the foundational technology for other large-scale projects. We spoke with one company whose innovation questionnaire explicitly asks each project manager to rate the interdependency risk and opportunity associated with each project.

4. Keep the list of criteria short.

There's only so much information that people can process to help inform meaningful decisions. We've seen some scorecards that examine over 20 different criteria and some with as few as three. Typically, companies include 5-7 criteria, and the most progressive companies try to limit the number to no more than 12. Companies should keep it short and look for opportunities to continually refine and cut down on the amount of information required to make decisions.

5. Secure early support from Finance, Marketing and Operations.

It is absolutely critical that R&D involve other functions at the earliest stages of technology and product development. When developing standard project evaluation criteria, R&D should solicit input from other stakeholder functions to ensure that everyone can agree on how to prioritize innovation projects. We spoke with several organizations that are really pulling forward these interactions; these organizations are requiring early Finance involvement in validating project valuations, soliciting long-term Marketing product roadmap plans, and scheduling multiyear launch date plans with Operations & Sales.

6. Don't let risk aversion creep into scoring guidelines.

From a process standpoint, it's important to diagnose project barriers and execution risks early. But don't let these feasibility issues overweight your scorecard so that riskier projects always score lower and become de-prioritized. Incorporate criteria that look at factors like disruptive potential, competitive advantage, and distinctiveness to add a premium to these higher-risk, higher-return projects.

7. Consider offering flexible weighting for criteria.

For diverse business groups, certain aspects of R&D projects may be significantly more important than others. One product line may be strongly focused on time-to-market while another may prioritize IP protection above everything else. These variations can make it difficult to develop standard project criteria.

One way companies can overcome this challenge is by permitting diverse businesses to create their own unique weighting schemes for those standard criteria. But this needs to be done with some restrictions. We spoke with one industrial manufacturing company that is doing just that--all business units must score all new R&D projects along seven enterprise criteria, but they can vary the weighting they apply to those criteria. This company was careful to not allow too much flexibility--each business unit must weight each criterion on a 1-5 scale, and the total weighting across the seven criteria cannot exceed 21. This helps to normalize the relative project scores and forces parity across the diverse businesses.

8. Be as explicit as possible with scoring guidelines.

Put the same project in front of two different project review panels and it can very easily receive two quite different scores. To squeeze out this subjectivity in project valuation, R&D needs to provide clear interpretation guidelines for its scorecards. If your scoring scale is 1-5, provide examples of which past projects would be considered a 1, 2, 3, 4, or 5 score. If your businesses vary in size, provide guidelines that offer flexibility by looking at absolute numbers as well as percentages. For example, specify that a score of "5" in cumulative sales means that the project will generate [is greater than or equal to] $50 million, or [is greater than or equal to] 10 percent of total sales for that business unit.

9. Use visual diagrams to highlight project and portfolio gaps.

Innovation scorecards don't have to be text-heavy spreadsheets with numbers. While a scorecard won't make a decision for you, it can foster more productive discussions around project selection and prioritization. Depicting your project scores visually can help to facilitate these discussions. For example, one company we recently profiled depicts all projects on a spider chart to show how every project is scoring across six key dimensions. This visual tool quickly highlights gaps, and when you aggregate all of these projects together, you can see if there are certain areas where your portfolio is leaving you exposed.

10. Actively promote user adoption across businesses.

Whenever leaders implement any global process change, they inevitably run into issues of user adoption. R&D teams should initially pilot the new process in 2-3 (preferably diverse) business groups, refine any shortcomings, and then implement globally. During the implementation period, R&D executives need to be aware of any specific cultural and/or geographic resistances and address them appropriately. Once usage becomes more widespread, R&D should build in accountability measures and audit performance to make sure that project scores and value projections are increasing in accuracy.

Getting Started

Several potential project criteria were listed on page 12--start with this list, then work with business partners to prioritize the few key criteria to include in your scorecard.


(1.) We interviewed R&D executives (chief technology officers, chief innovation officers, senior VPs of R&D, as well as senior R&D portfolio managers) from over 100 global companies spanning the U.S., Canada, Europe, Asia, South Africa, and Australia.

(2.) The companies were selected from the Council's membership, based on their interest as well as their competence, as in portfolio management. We assessed their competence in portfolio management through our experience in working with these member companies over the years (through research, discussions and onsites) as well as through quantitative and survey-based methods. We were careful to assemble a representative set of companies in terms of size, industry and even innovation strategy (lead innovators vs. fast followers). These companies represent a variety of industries and revenue bands as follows:

Breakdown by size: less than $5 billion in revenues, 27 percent; $5 billion -$10 billion in revenues, 33 percent; more than $10 billion in revenues, 40%.

Breakdown by industry: chemicals and materials, 26 percent; health care, 17 percent; information technology, 9 percent; industrials, 20 percent; energy, 6 percent; consumer products, 22 percent.

Amanda Eager is a research director at the Research & Technology Executive Council (RTEC), a subsidiary of the Corporate Executive Board (, in Washington, D. C. During her tenure at the Corporate Executive Board, she has led and published research studies in a variety of subject areas, including R&D portfolio management, open innovation, technology road-mapping, and supply chain & logistics. She holds a bachelors degree in finance and government from the University of Notre Dame.
Examples of Project Criteria

Technical Criteria

* Technical Risk/Feasibility
* Reuse Potential
* Protectable Intellectual Property
* Disruptive Potential/Technology Sophistication
* Project Dependencies
* Time-to-Market
* Extent of External Innovation

Resourcing Criteria

* Capital Investment
* R&D Costs for Completion
* Specialized Skills Required
* Ease of Manufacturing
* Ease of Distribution
* Ease of Procurement
* Collaboration Opportunity
* Most Likely Cause of Project Derailment

Market/Consumer Criteria

* Durability of Competitive Advantage
* Ease of Commercialization
* Growth Opportunity
* Barriers to Entry
* Regulatory Environment
* Consumer Dissatisfaction with Current Offering
* Proximity to Core Markets

Financial Criteria

* Peak Year Sales
* Expected Marginal Income
* NPV Estimate
* Operating Income
* Durability of Financial Impact
* Time to Reach Peak Sales
* Gross Margin

Strategic Criteria

* Corporate Strategy Fit
* Business Strategy Fit
* Brand Alignment
* Strategic Importance
* Critical Assumptions
COPYRIGHT 2010 Industrial Research Institute Inc.
No portion of this article can be reproduced without the express written permission from the copyright holder.
Copyright 2010 Gale, Cengage Learning. All rights reserved.

Article Details
Printer friendly Cite/link Email Feedback
Title Annotation:MANAGERS AT WORK
Comment:Designing a best-in-class innovation scorecard.(MANAGERS AT WORK)
Author:Eager, Amanda
Publication:Research-Technology Management
Geographic Code:4EUUK
Date:Jan 1, 2010
Previous Article:Managing in turbulent times.
Next Article:Industrial Research Institute's R&D trends forecast for 2010: R&D expenditures are projected to drop in 2010, but investment in new business projects...

Terms of use | Privacy policy | Copyright © 2021 Farlex, Inc. | Feedback | For webmasters |