Multicriteria model for prioritization of research and development projects.

ABSTRACT

A multicriteria model is developed for prioritization of research and development projects of a large electricity utility corporation. In this model the enterprise results of strategic planning are considered as criteria and the ELECTRE method is applied. The ELECTRE method was chosen because of aspects such as: the context of the problem and the actors in the decision process.

1. INTRODUCTION

Prioritization of research and development projects (R&D) of an enterprise should emphasize the overall planning aspect. In addition to the normative criteria usually considered, the enterprise results should also play a part in the planning methodology. Figure 1 shows this approach.

[FIGURE 1 OMITTED]

In most cases, the approach used to deal with R&D investment decision problems is related to multicriteria decision. Numerous criteria considered in investment problems can be approached and these criteria represent the objectives, in this case, multiple objectives in the decision process.

The development of each R&D project demands a scientific approach. In the same way, the management of the entirety R&D plan, the prioritization process and the R&D projects selection can be dealt with using a decision model based on a scientific approach.

In this study we are concerned with the formalization of the multicriteria decision aid model for selection of R&D projects.

The establishment of the multicriteria decision model for prioritization of R&D projects involves the use of multicriteria methods and other essential aspects of the management process. These other aspects must be associated with the establishment of objectives for R&D in the company. Thus, to implement this process in an organization, identification of the criteria, or the relevant business's objectives utilized in the planning process, is necessary.

2. THE R&D PROBLEM

R&D project planning has a macro view and has a large influence on technology investment. As we can see in the literature, many authors emphasize investment in technology and try to form up-to-date picture of the decision process and systems investment in organizations (ESCOBAR-PEREZ, 1998; GROVER ET AL, 1998, IVES & LEARMONTH, 1984, SPRAGUE & WATSON, 1989). Furthermore, ESCOBARPEREZ (1998) presents a systems evaluation and selection problem, but his objective is not the definition of models or methods of planning and prioritization.

Multicriteria decision making is the most commonly method to deal with problems of investment in R&D programs (SCHNIEDERJANS & SANTHANAM, 1993; ESCOBAR-PEREZ, 1998; HAN ET AL, 1998; MARTISONS ET AL, 1999). STEWART T. J. (1991) has developed a decision support system for the selection of a portfolio of R&D projects in an electricity corporation utilizing a multi-criteria decision approach associated with a heuristic algorithm. MEADE AND PRESLEY (2002) and GREINER ET AL (2003) present a valuable method to support the selection of projects in a research and development (R&D) environment utilizing analytic hierarchy process (AHP). In the study of GREINER ET AL (2003) specifically, the AHP is integrated with a 0-1 integer portfolio optimization model called as hybrid methodology. In other study, VANDYK and SMITH (1990) developed a multicriteria decision aid to select R&D portfolio where the characteristics of the decision alternative cannot be readily quantified. In these studies numerous investment criteria can be considered. Figure 2 below shows multicriteria decision making in R&D problems.

In the systems planning context specifically, there is a link between strategic planning and a specific planning program of the company (ESCOBAR-PEREZ, 1998; MARTISONS ET AL, 1999; PANT & HSU, 1999).

3. MULTICRITERIA DECISION MAKING METHODS

The decision model for prioritization of R&D projects involves the use of multicriteria decision aid methods (MCDA). The MCDA area has a large set of tools whose purpose is to aid the decision maker during the development of a decision process.

According to VINCKE (1992), multicriteria decision aid aims to give the decision-maker some tools in order to enable him to advance in solving a decision problem where several--often contradictory--points of view must be taken into account. The first fact which should be noted when dealing with this type of problem is that there does not exist, in general, any decision which is the best simultaneously from all points of view. Therefore, the word optimization does not make any sense in such a context; in contrast to the classical techniques of operations research, multicriteria methods do not yield 'objectively best' solutions.

In general, multicriteria decision-aid methods are divided into three large families:

(1) Unique synthesis criterion: consists of aggregating the different points of view into a unique function which must subsequently be optimised. The main method is the Multiple Attribute Utility Theory (MAUT) (KEENEY & RAIFFA ,1976).

(2) Outranking synthesis approach: these methods aim first to build a relation, called outranking relation, which represents the decision-maker's strongly established preferences, given the information at hand. The second step consists of exploiting the outranking relation in order to help the decision-maker solve his problem. As an example, the ELECTRE method is widely explored in the literature.

(3) Interactive local judgment approach: proposes methods which alternate calculation steps and dialogue steps (interaction with the decision-maker). They are most developed in the frame of multiple objective mathematical programming.

The process of choosing a multicriteria method should involve an analysis of the context of the problem, the actors and their preference relation structure. Figure 3 shows this approach.

[FIGURE 3 OMITTED]

The context of this problem is a selection problem. Given the set of alternatives A, the choice problematic, or selection (P.[alpha]) consists of the choice of a subset A' [subset] A, as small as possible, composed of alternatives judged as most satisfactory. In this case, we want to select the set of R&D projects judged as most satisfactory.

Considering that the problem of prioritization of R&D projects is included in an enterprise planning approach, it makes no sense to utilize a prioritization methodology without making a connection with a planning process visualizing enterprise results.

R&D Investment decision is closely linked to multicriteria decision problems (STEWART T. J., 1991,). Many studies show that criteria used in investment problems can be considered as criteria in R&D problems. These criteria represent the multiple objectives in the decision process.

This study has been developed with MAUT and ELECTRE methods, and other classic multicriteria decision aid methods that can be utilized for prioritization of R&D projects. However, in this paper a case study is presented exploring the ELECTRE I method.

ELECTRE I is a method which aims to eliminate obsolete alternatives with a set of rates/indexes which assess degrees of outranking among alternatives. This method is aimed at problems that involve selection (Problematic of Choice P.[alpha]), and aims to reduce the size of the non-dominated set by means of concordance and discordance indexes that measure the advantage and equal relative disadvantage to pair among the alternatives.

The decision maker establishes relative weights for criteria and the evaluation of each alternative according to each criterion. The decision maker also establishes the limit for the rates of concordance index p and discordance index q.

The concordance index refers to a value which represents a proportion of weight so that alternative a is preferable to b. In ELECTRE I, this index is calculated by the following formula:

(1) C(a,b) = [SIGMA]([W.sup.+] + .5[W.sup.*])/[SIGMA]([W.sup.+] + [W.sup.*] + [W.sup.-])

where, W+--the sum of objectives' weight where a is preferable to b. W--the sum of objectives' weight where b is preferable to a. W*--the sum of objectives' weight where a = b (indifferent)

On the other hand, the discordance rate measures the relative disadvantage between two alternatives a and b, being defined as the maximum reason for each criterion (criteria difference where b is preferable to a) divided by the distance of possible criteria difference of the discordance rate. The discordance rate calculation formula is as a follows:

D(a,b) = Max {[Z.sub.gk] - [Z.sub.ak]/[Z.sup.*.sub.k] - [Z.sup.-.sub.k]} [for all]k, b > a

where: Zk* = excellent achievement for criteria, k- amongst the evaluations of criteria alternatives k, Zk* congruent to the best evaluation.

Zk- = Terrible achievement for criteria k-amongst the evaluation of criteria alternatives k, Zk- congruent to the worst evaluation.

In ELECTRE I, two parameters are indispensable: p and q. alternative a defeats alternative b if, and only if, C(a,b)>=p e D(a,b) <=q.

Kernel is determined in function of p- the minimum concordance rate required for outranking, and q- the maximum discordance rate needed for outranking. The kernel consists of elements that are not outranked by any other element in the kernel, and each element not in the Kernel is outranked by at least one element in the kernel. (OLSON 1996).

According to ROY (1996), these methods are based on a study of outranking relation in a non compensatory logic with the power of veto using concordance and discordance. Outranking relationships are constructed in such a way that any one alternative is as good as another under the following conditions: A criteria majority supports this proposal (concordance principle) and a minority opposition is not considered strong enough to disprove this proposal.

4. CASE STUDY WITH ELECTRE I METHOD

This case study in based on work developed in an electricity utility corporation. The team chosen to develop the study had previous experience in the R&D management process, in this company.

The following criteria were considered:

FIG. 4: CRITERIA CONSIDERED IN THE R&D PROJECTS SELECTION PROBLEM

CRITERIA

Project cost

Success probability associated with the project

Strategic impact level in the organization

Final product quality impact level in the organization

Operational productivity impact level in the organization

The performance of each alternative (R&D project) for each criterion is presented in table 1. These values are normalized between 0 and 1. The cost criterion is presented with an inverse scale, where 0 represents the biggest cost and 1 represents the lowest cost. The inversion was necessary to keep the same preference order in all of the criteria. Table 1 also shows the weight of the criteria.

Applying the concordance index p = 0.4 and the discordance index q = 0.3 we obtain the following projects:

P2, P5, P7, P9 and P10

A sensitive analysis of the concordance and discordance index p and q shows that the same recommendation remains when we apply a 10% reduction in the parameters p and q and an increase of 10% in p. When the q index is increased by 10% the project P5% is excluded. Other variations show different recommendations as follow:

* Parameter p plus 20% implies in the inclusion of P1 and P8 projects.

* Parameter p minus 20% implies in the exclusion of P2 project.

* Parameter q plus 20% implies in the exclusion of P5 project.

* Parameter q minus 20% implies in the inclusion of the P3 and P8 projects.

5. CONCLUSIONS

In the case study presented, an approach to the problematic choice was considered necessary to select a subset with the most satisfactory projects among all of the R&D projects proposed. Thus, the initial intention was the selection of a group of projects that had presented an outranking relation preference over the others.

Continuation of the study involves a critical evaluation of the current procedures utilized in prioritization problems and justification of investment in R&D projects. Furthermore, based on the context of the problem, alternative methodology to solve this problem will be presented.

The critical analysis of the current procedures will involve a revision of the criteria or business objectives utilized in the planning process. The whole study will be developed from the point of view of analysis of investment in R&D projects and principally of multicriteria decision aid.

TABLE 1: PROJECT PERFORMANCE FOR EACH CRITERION

           C1     C2      C3     C4      C5
Criteria
weigth     0, 3   0, 15   0, 2   0, 15   0, 2

P1         0.40   0.20    0.40   0.20    0.80
P2         0.20   0.70    1.00   0.90    0.00
P3         0.70   1.00    0.10   0.00    0.80
P4         0.65   0.40    0.30   0.10    0.70
P5         0.55   0.30    0.50   0.40    0.20
P6         0.10   0.00    0.90   0.70    0.30
P7         1.00   0.80    0.20   0.10    0.55
P8         0.40   0.65    0.45   0.30    0.65
P9         0.00   0.50    0.80   1.00    0.80
P10        0.60   0.30    0.00   0.10    1.00

REFERENCES

ESCOBAR-PEREZ, B. "Information systems investment decisions in business practice: the Spanish case", European Journal of Information Systems, 7: 202-209, 1998.

GREINER, M. A.; FOWLER, J. W.; SHUNK, D.L.; CARLYLE, W. M.; MCNUTT, R. T. "A Hybrid Approach Using the Analytic Hierarchy Process and Integer Programming to Screen Weapon Systems Projects", IEEE TRANSACTIONS ON ENGINEERING MANAGEMENT, 50 (2): 192-203 MAY 2003.

GROVER, V.; TENG, J. T. C. and FIEDLER, K. D. "IS investment priorities in contemporary organizations", Association for Computing Machine--Communications of the ACM, New York, Feb, 1998.

HAN, C. H.; KIM, J. K.; CHOI, S. H. and KIM, S.H. "Determination of Information System Development: Priority Using Quality Function Development", Computers Ind. Engineering, Great Britain, 35 (12): 241-244, 1998.

IVES, B. and LEARMONTH, G. P. "The Information Systems as a Competitive Weapon", Association for Computing Machinery--Communications of the ACM, 1193-1201, Dez, 1984.

KEENEY, R. L. and RAIFFA, H. Decision with Multiple Objectives: Preferences and Value Trade-offs, John Wiley & Sons, 1976.

MARTISONS, M.; DAVISON, R. and TSE, D. "The balanced scorecard: A foundation for the strategic management of information systems", Decision Systems, Amsterdam, 25(1): 71-89, Feb, 1999.

MEADE, L. A.; PRESLEY, A. "R&D Project Selection Using the Analytic Network Process", IEEE Transactions on Engineering Management, 49 (1): 59-66 FEB 2002.

OLSON, D. L. Decision Aids for Election Problems, Springer, 1996.

PANT, S. and HSU, Co "An integrated framework for strategic information systems planning and development", Information Resources Management Journal, Middletown, 12(1): 15-25, Jan-Mar, 1999.

ROY, B. Multicriteria Methodology for Decision Aiding, Netherlands, Kluwer Academic Publishers, 1996.

SPRAGUE, R. H. and WATSON, H. J. Decision Support Systems--Putting Theory Into Practice, Prentice-Hall International. 1989.

STEWART T. J. "A Multicriteria Decision Support System for R&D Project Selection", Journal of the Operational Research Society, 42 (1): 17-26 JAN, 1991.

VANDYK E. and SMITH D. G. "Research-And-Development Portfolio Selection By Using Qualitative Pairwise Comparisons", Omega-lnternational Journal of Management Science, 18 (6): 583-594 1990.

VINCKE, P. Multicriteria Decision-Aid, John Wiley & Sons, 1992.

Author Profiles:

Dr. Adiel Teixeira de Almeida earned his PhD at the University of Birmingham, England in 1994. Currently, he is a Professor of Production Engineering at The Federal University of Pernambuco and deals with information and decision systems, multicriteria decision aid, quality and, reliability and maintenance engineering.

Dr. Ana Paula Cabral Seixas Costa earned her PhD at The Federal University of Pernambuco, Brazil in 2003. She is a Lecturer at The Federal University of Pernambuco and deals with information and decision systems.

Ms Caroline M. G. de Miranda earned her MSc at The Federal University of Pernambuco, Brazil in 2002. Currently, she is doing the doctorate course in Production Engineering and is a member of a research group of information and decision systems at The Federal University of Pernambuco. She is graduated in civil engineer.