Printer Friendly

The performance evaluation of university scientific research project management based on the FAHP.

1. Introduction

The university research project management and performance evaluation is still in just the start stage, and the performance evaluation has some problems. Due to scientific research project evaluation is belong to the subjective evaluation, so there is no exact quantitative indexes of science and technology project quality. It is difficult to reflect the quality level completely and really, and this lead to inaccurate and incomplete information when evaluating the science and technology project [1].

Scientific research project management performance evaluation is to point to use scientific and standardization methods to have a comprehensive evaluation for performance management, personnel performance, results performance of scientific research project and the goal realization degree following certain principles, procedures and standards and organized by research management department and provide scientific basis for scientific research project decision-making, management and implementation of the future through the timely and effective information feedback at the same time [2-4].

At presently, there are a variety of university management performance evaluation methods, mainly including balance performance index method, the benchmark of analysis, layer analytical method, and fuzzy comprehensive analysis [5-6]. AHP method was put forward in the 1970s by famous operations research scholars Satty of university of Pittsburgh and it is a multi-objective decision analysis method combined with qualitative and quantitative analysis [7-8]. The difference between fuzzy analytic hierarchy process (FAHP) and analytic hierarchy process (AHP) is that the former simplifies complex degree of relative importance about judgment target and extends AHP method to fuzzy environment and realizes convenient and fast conversion from qualitative to quantitative about the decision with the aid of fuzzy judgment matrix combining the fuzzy mathematics and the analytic hierarchy process (AHP). It forms fuzzy consistent judgment matrix directly by the fuzzy judgment matrix and solves judgment uniformity problem [9-10].

This paper consider that scientific research fund performance evaluation system should be able to reflect the whole process of the establishment to examination and approval of the research fund, and evaluate research efficiency through the specific data. Because of varies sex of selection about quantitative analysis indicators of the process and results of scientific research activities and funds performance evaluation. This paper mainly uses fuzzy analytic hierarchy process to constitute a proper evaluation index system to make an objective, fair and accurate comprehensive evaluation for a certain period of university project efficiency and performance.

2. The mathematical model of fuzzy analytic hierarchy process

The basic idea of fuzzy analytic hierarchy process in general is first to research the decision making problems for hierarchical structure analysis, and then establish affect system index according to the structure of target strata, criterion layer and layer factors. Next, it builds triangular fuzzy matrix through expert scoring for comparison between two elements according to 1 ~ 9 digital scale method and establish fuzzy comprehensive judgment matrix to get the initial weight. It ensures the corresponding indexes weights of the factors of different levels after removing blur and standardization finally. The performance evaluation of scientific research project management using fuzzy analytic hierarchy process is shown in figure 1.

2.1 establish performance evaluation levels structure model

Fuzzy analytic hierarchy process (AHP) is derived by analytic hierarchy process (AHP), and it determines the index of the research object according to target strata, criterion layer, and layer factor in the analytic hierarchy process. There is only one element in the target layer which is the purpose of this question as the highest level structure. Rule layer is some main element which is important for target layer. They are measures and principles, policies, etc for achieving the goal. Factors layer is the specific index factor attached to the corresponding standards layer.

The performance evaluation of scientific research project is a complex system of multi-level and many factors and many stages, and there are many factors reflecting the level and affecting the change. In this paper, the performance evaluation of scientific research project is mainly according to the use and safeguard status of scientific research outlays to evaluate the quality of the project management. According to the funds investment of scientific research, production, and the output and project external overall condition and the security situation of the facilities, combining previous long-term studies, it takes the university scientific research project management performance evaluation for the purpose (O), and integrates relevant index about the scientific research project investment, production and output and project guarantee. It gradually forms the university scientific research project management performance effectiveness evaluation index system by Preliminary screening, adding and perfecting about these indexes. University scientific research project management performance evaluation is the target layer. And rule layer (S) includes project guarantee ([S.sub.1]), the fund budget ([S.sub.2]), process management ([S.sub.3]) and project performance ([S.sub.4]). There are 12 factor evaluation indexes including research team quality ([S.sub.11]), research performance ([S.sub.12]),scientific research conditions ensure ([S.sub.13]), project management status ([S.sub.14]), the fund budget audit ([S.sub.21]), budget approval ([S.sub.22]), budget budget implementation ([S.sub.23]), use of funds rules construction ([S.sub.31]), spending projects and access ([S.sub.32]), funds management and supervision ([S.sub.33]), the research results ([S.sub.41]), economic benefits ([S.sub.42]) and social effects ([S.sub.43]). As shown in figure 2.

2.1.1 Projects supportability index

The establishment of projects supportability index is because of the prospective, exploratory, risk and cannot foresight of scientific research activities. Evaluating the quality of the team and project can guarantee the smooth progress of scientific research project, such as construction equipment, technology application development laboratory level, including the quality of talents which will affects the project, and the specific content is as shown in table 1.

2.1.2 Funds budget sex index

About the fund budget index, research fund budget is throughout the whole in the spending process, and budget examination and approval lays the foundation of the research project and provides a range for the use of the implementation process of scientific research project. The implementation of the budget plan can supervises the reasonable amount of funds use effectively and provides strong guarantee for planning implementation of the research funding. Specific content is as shown in table 2.

2.1.3 Process managerial index

The distribution of the quantity of the research fund is controlled by the budget, and the next step is the supervision and administration in the process of using Setting up complete process management mechanism has effective monitoring for preventing the appearance of the abuse and divert in the use of funds and explicit the economic responsibility and makes accountability mechanism establishment and implementation play to utility. Specific content is as shown in table 3.

2.1.4 Project profitability index

When we pay attention to the process of the scientific research outlays management, we should pay attention to the effect at the same time. The use of funds for research in the universities is to create more benefits for the school and countries. From the resource allocation part to think, it is that to use fewer resources as possible to create maximization research influence. Specific content is as shown in table 4.

2.2 The determination of index weight

In the hierarchical analysis of a general problem, two and two comparison judgment matrix usually did not consider the fuzzy of the judgment and only considered the man's judgment of the two possible extreme: use membership 1 to choice one index and at the same time again use membership 1 to negative other scale value. In the fuzzy hierarchical analysis it used group decision to tectonic triangular fuzzy judgment matrix to confirm the weight of each variable in each layer variable by the judging of the expert assessment. Using triangular fuzzy evaluation method, we first need to determine the scale methods of the triangular fuzzy number, for example, using 1 ~ 9 scale method (table 5). Experts use triangular fuzzy number to mark the relationship between two variables when specific rating. The triangle fuzzy number M was expressed as (1, M, u), of which X = M, and X belongs entirely to M, and outside of the 1 and u completely do not belong to the fuzzy number M. So firstly it need expert to mark the median m, and it showed the basic relations between two variables. Then it marks 1 and u, which means the possible highest and lowest score. After structuring the triangular fuzzy judgment matrix of rule layer and factors layer, it gets the fuzzy matrix through the integration to get the initial weights, and then get the weight finally by removing blur.

2.3 The comprehensive evaluation of the scientific research project management performance

The fuzzy comprehensive evaluation method mainly use the fuzzy set theory, the fuzzy transform principle and maximum membership degree of the fuzzy mathematics and consider various factors related with the evaluation object to make reasonable comprehensive evaluation [11-12]. It has established evaluation levels structure and index weight of the university scientific research project management performance by fuzzy analytic hierarchy process (FAHP) in above. Next it construct scientific research project management performance evaluation model with fuzzy comprehensive evaluation method.

3. Case Analyses

3.1 Project Situation

This article selects A project as evaluation objects, and this project research industry cluster manufacturing services platform of the lace industry in Zhejiang province in order to improve scientific research and practical ability of the project team members. After the completion of the subject, it reflects research achievements with science and technology paper form and plans to publish 1 ~ 2 articles related to the project research contents in core journals or in a meeting and participate in the 1 ~ 2 times academic conferences, etc.

3.2 Build the triangular fuzzy matrix

Firstly, it use fuzzy analytic hierarchy process to select and division the index of the whole system according to the target strata, criterion layer and layer factors when evaluating the project management performance. It has established four rule layers including the project security, budget, process management, project benefits combined with the mentioned, and the factors layer in project guarantee divided into one class index: research team quality, scientific research conditions guarantee, the scientific research achievement, the project management condition, research team quality. In order to determine the specific weight of each index, it hires experts to carries on the comparison of all the variables in rule layer and layer factors to get the triangular fuzzy judgment matrix. It takes the triangular fuzzy judgment matrix of the rule layer and the index factors layer in the project guarantee, reference table 6 and table 7.

With multiple experts participate in scoring and so the comprehensive fuzzy triangle number is:

[a.sub.ij] = [[[a.sup.1.sub.ij] + [a.sup.2.sub.ij] + A[a.sup.m.sub.ij]]/m]

Among them, the m is the total number of experts [a.sup.m.sub.ij], means the triangular fuzzy number of the m. It gets table 8 and table 9 when taking comprehensive fuzzy triangle count from triangle judgment matrix of table 6 and table 7.

3.3 Calculate index synthesis weights

When it gets the fuzzy judgment matrix of the rule layer and the index of the [S.sub.1], it needs to calculate the initial weight index, calculated as below:

[D.sup.k.sub.i] = [n.summation over (j=1)] [a.sup.k.sub.ij] / ([n.summation over (i=1)] [n.summation over (j=1)] [a.sup.k.sub.ij]), i = 1, 2......n

The [D.sup.k.sub.i] means the comprehensive fuzzy value of i in K and the initial weight calculation are as follows:


Similarly, it gets the initial weight of [S.sub.11], [S.sub.12], [S.sub.13], [S.sub.14]


After calculating the initial weight of the above indexes, it needs to remove blur and calculate the finally the weight of the index. The comparison principle of the fuzzy number is:

Definition 1: [M.sub.1] ([l.sub.1], [m.sub.1], [u.sub.1]) and [M.sub.2] ([l.sub.1], [m.sub.2], [u.sub.2]) is the triangular fuzzy number. The possibility of [M.sub.1] [greater than or equal to] [M.sub.2] is defined by triangular fuzzy function as


Definition 2: The possibility that one fuzzy number greater than other K fuzzy number is defined as:

v(M [greater than or equal to] [M.sub.1], [M.sub.2] ... [M.sub.K]) = min V(M [greater than or equal to] [M.sub.I]), i = 1, 2 ... k

So remove blur of [S.sub.1], [S.sub.2], [S.sub.3], [S.sub.4], [S.sub.11], [S.sub.12], [S.sub.13], [S.sub.14] as:


It gets the minimum value through comparing digital of each line as

(0.514 0.762 0.247 1) (1.00 0.851 0.382 0.504)

With the standardization of the above weight values, it gets the final weight of each index.

Standardization point to make (a b c d) to

([a/a + b + c + d] [b/a + b + c + d] [c/a + b + c + d] [d/a + b + c + d])

The final weight of the index:


It gets the level index system of the performance evaluation of scientific research project management comprehensively as table 10.

3.3 Comprehensive evaluations

After determining the index weight, it scores to index factors according to comment set by experts. Suppose the evaluation of each factor in factor set has 4 levels, including well, good, medium and bad. The comment set: V = {well, good [v.sub.2], medium [v.sub.3], bad [v.sub.4]}, [u.sub.1] = {Research team quality, Scientific research conditions ensure, Research performance, Project management status} = {[R'.sub.1], [R'.sub.2], [R'.sub.3], [R'.sub.4]}. Experts scored one class index through the evaluation of the second index, then it random extract four score sheets to carry on the analysis from the form by visiting survey or telephone consultation the relevant experts. Among them, the evaluation of research team quality indexes is showed as the following table 11:

From the table, it get that for the performance of research team quality, the expert evaluation 20% think well, 55% think good, 20% think general, 5% think that bad. There is one line matrix [R'.sub.1] to show the evaluation results:

[R'.sub.1] = (0.2, 0.55, 0.2, 0.55)

Similarly, assume that experts evaluate scientific research conditions guarantee, the scientific research achievement and the project management situation, then the evaluation results constitute the fuzzy relation matrix of project guarantee index valuation results:


Reference table 10, and multiply by so it get the fuzzy subset:


From the comprehensive evaluation results, it know that experts on the evaluation results about 15% is good, 69% think good and 14% think that moderate, 2% think the poor about the project guarantee index of the project. In the same way, it takes other indicators of scientific research project performance management as scientific research budgets, research fund internal control and research fund output to evaluate according to the above method, and through the fuzzy computation it get comprehensive evaluation results for [B.sub.2] = (0.2, 0.7, 0.1, 0.02) [B.sub.3] = (0.1, 0.7, 0.16, 0.04) [B.sub.4] = (0.1, 0.8, 0.1, 0). By the "project guarantee, research fund budget, research fund internal control, research fund output" evaluation result for the line consists of the fuzzy relation matrix is for


Then it makes weight distribution coefficient vector A and the fuzzy relation matrix R for multiplication, and The comprehensive fuzzy evaluation set B is defined as shown below.


According to the maximum subjection principle in fuzzy math, and the maximum subjection principle is close to the good, so this research project performance management situation is good.

4. Conclusions

It is very important for the management of scientific research projects to establish a performance evaluation model which can evaluate the projects effective and reasonable. This paper is based on the actual demand of the scientific research project management performance, and put forward the fuzzy hierarchy performance method which combined Analytic hierarchy process and fuzzy evaluation method. We transform the complex problem about scientific research project management into several factors, though the qualitative and quantitative analysis. Then, we invite experts to score the related factors, and construct the fuzzy triangular matrix to get the ultimate weight through to blur and standardization, so that it can makes the actual system situation and the experts' professional knowledge or experience quantitative. This method can reflect the influence of indexes objectivity, avoiding the randomness of subjective evaluation. In addition, it is easy to judge the benefits of the various projects from the score in the form of evaluation, and this will helps us to evaluate the project performance effective.

5. Acknowledgment

This research is supported by Foundation of Zhejiang Education Committee(Grant No.Y201122787), Foundation of Ministry of Education of China(Grant No.KJTJ-201045), National Natural Science Foundation of China (Grant No. 71071140), National Natural Science Foundation of Zhejiang Province (Grand No. Y1090617) as well as Modern business centre of Zhejiang GongShang University.

Received: 11 September 2013, Revised 24 October 2013, Accepted 30 October 2013


[1] Na, Ma Li., Hua, Li Jian. (2008). Hierarchy gray evaluation model applied research of the evaluation of science and technology project. Science and Technology Management, 5, 277-279.

[2] Qing, Dai Guo., Ya, Li Li. (2005). Foreign science and technology project performance evaluation research and reference. Chinese science and technology, BBS, 5.

[3] Cerci, A. B., Ekin. (2010). Performance evaluation of a 250 L household refrigerator. Energy Education Science anTechnology Part A, 26, 25-26.

[4] Kizilkan. (2011). Performance evaluation of variable speed refrigeration system with artificial neural networks. Energy education science and technology part a. 26, 239-250.

[5] Buckley, J. (1985). Fuzzy hierarchical analysis. Fuzzy Sets and Systems, 17, 233 ~ 247.

[6] Zhao Zhen Wu, Tang Wan Sheng, Ning Yu Fu. (2006). Based on the AHP of fuzzy simulation in venture investment project evaluation. Fuzzy system and mathematics, 4.

[7] Zhang Wei Xuan. (2008). Thermal power production enterprise performance management system research based on the balanced scorecard. Southwest traffic university.

[8] LiXiao Yan, FengJunWen. (2009). Benchmarking establish research based on the DEA and AHP. The economy, 4.

[9] Xu Zhao Yong. (2009). Scientific research personnel performance evaluation model research based on the analytic hierarchy process (ahp). Management of scientific research, 30 (S1) 115-118,147.

[10] Celen, Aydin. (2012). Performance evaluation of Turkish electricity distribution market using a combined FAHP/TOP SIS method. Energy Educ Sci Technol Part A 29,1263-1276.

[11] Xie JiJ ian, Liu Cheng Ping. (2006). Fuzzy mathematics method and its application (third edition).wuhan: Huazhong University of science and technology press.

[12] Fu, M., Xie, Z., Chen, Y., Zhang. (2012). Social comprehensive evaluation for the economic profits of railway in China. Energy education science and technology part B, 1169-1176.

Wang Bei (1), Liu Dongsheng (2)

(1) Zhejiang Gongshang university Office of Science and Technology Hangzhou, China

(2) Zhejiang Gongshang University College of Computer Science & Information Engineering China

Author Biographies

Wang Bei received her Master degree in the School of Information Engineering of Zhejiang Gongshang University (ZJGSU), China, in 2005. She is currently a Research Associate in Office of Science & Technology in ZJGSU. Her research interests include Administrative Management on Science Research and Management Information System.

Dongsheng Liu received his PHD degree in the school of information engineering, in 2008, Zhejiang Gongshang University (ZJGSU), China. He is currently an Associate Professor in the school of information engineering at ZJGSU. His research interests include data mining, electronic commerce and wireless network.

Table 1. The evaluation index and evaluation content of the project

Rule layer    One class index        Evaluation content

Project       Research team          The academic level of the whole
guarantee     quality                team, excellent young proportion,
                                     master degree or above proportion

              Scientific research    Technology application
              conditions             development laboratory level,
                                     equipment construction

              Research performance   The main research results

              Project management     Project management team
              status                 condition, the construction of
                                     university management department

Table 2. The evaluation index and evaluation content of the funds

Rule layer     One class index         Evaluation content

Funds budget   He fund budget audit    Budget material quality, budget

               Budget approval         Approved Progress, declare

               Budget implementation   Compliance of Project
                                       implementation and budget,
                                       Funds planning implementation

Table 3. The evaluation index and evaluation content of the process

Rule layer    One class index        Evaluation content

Process       Use of funds rules     Constitution execution dynamics,
management    construction           Use of funds statute

              Spending projects      Spending powers and economic
              and access             responsibility, spending projects
                                     and amount management

              Funds management and   The periodic goal completion
              supervision            rate, expenditure executive

Table 4. The evaluation index and evaluation content of the project

Rule layer     One class index        Evaluation content

Project        The research results   Patent number, number and level
performance                           of papers Achievement transfer
               Economic benefits      income, the scientific research
                                      technical gains

               Social effects         Actual use efficiency of the
                                      results, talent cultivation

Table 5. 1 ~ 9 scale method

Scale                               Implication

1             Compared, the same important
3             Compared, the former slightly important than the latter
5             Compared, the former obvious important than the latter
7             Compared, the former intensity important than the latter
9             Compared, the former extremely important than the latter
2, 4, 6, 8    Middle value of the above adjacent judgment
countdown     If the importance ratio of element i and element j is
              a.j, then the importance ratio of element j and element
              i is a.j = 1/a.j

Table 6. Triangular fuzzy judgment matrix of the rule layer

                  [S.sub.1]              [S.sub.2]

[S.sub.1]          (1, 1, 1)             (1, 2, 3)
                   (1, 1, 1)         (0.2, 0.25, 0.33)

[S.sub.2]       (0.33, 0.5, 1)           (1, 1, 1)
                  (3, 4, 5)              (1, 1, 1)

[S.sub.3]       (0.33, 0.5, 1)       (0.17, 0.25, 0.33)
                  (3, 4, 5)            (0.33, 0.5,1)

[S.sub.4]     (0.143, 0.2, 0.33)       (0.33, 0.5, 1)
                  (2, 3, 4)              (6, 7, 8)

                  [S.sub.3]              [S.sub.4]

[S.sub.1]         (1, 2, 3)               (3,5,7)
              (0.2, 0.25, 0.33)      (0.25, 0.33, 0.5)

[S.sub.2]         (3, 4, 6)              (1, 2, 3)
                  (1, 2, 3)          (0.13, 0.14, 0.17)

[S.sub.3]         (1, 1, 1)            (0.5, 1, 1.5)
                  (1, 1, 1)          (0.14, 0.17, 0.25)

[S.sub.4]        (0.67, 1, 2)            (1, 1, 1)
                  (4, 6, 7)              (1, 1, 1)

Table 7. Triangular fuzzy judgment matrix of [S.sub.11],
[S.sub.12], [S.sub.13], [S.sub.14]

                  [S.sub.11]             [S.sub.12]

[S.sub.11]         (1, 1, 1)             (2, 3, 4)
                   (1, 1, 1)         (0.25, 0.33, 0.5)

[S.sub.12]    (0.25, 0.335, 0.5)         (1, 1, 1)
                  (2, 3, 4)              (1, 1, 1)

[S.sub.13]    (0.17, 0.25, 0.33)        (1, 1.5, 2)
                  (3, 4, 5)            (0.5, 0.67,1)

[S.sub.14]     (0.2, 0.25, 0.5)        (0.33, 0.5, 1)
                  (2, 3, 4)            (0.33, 0.5, 1)

                  [S.sub.13]             [S.sub.14]

[S.sub.11]        (3, 4, 6)               (2,4,5)
                 (1, 1.5, 2)           (0.5, 0.67, 1)

[S.sub.12]      (0.5, 0.67, 1)           (1, 2, 3)
                 (1, 1.5, 2)             (1, 2, 3)

[S.sub.13]        (1, 1, 1)            (0.5, 1, 1.5)
                  (1, 1, 1)            (0.5, 0.67, 1)

[S.sub.14]       (0.67, 1, 2)            (1, 1, 1)
                 (1, 1.5, 2)             (1, 1, 1)

Table 8. Comprehensive fuzzy judgment matrix of the rule layer

                   [s.sub.1]               [s.sub.2]

[s.sub.1]    1.00    1.00    1.00    0.60    1.13    1.67
[s.sub.2]    1.67    2.25    3.00    1.00    1.00    1.00
[s.sub.3]    1.67    2.25    3.00    0.25    0.38    0.67
[s.sub.4]    1.07    1.60    2.17    3.17    3.75    4.50

                   [s.sub.3]               [s.sub.4]

[s.sub.1]    0.60    1.13    1.67    1.63    2.67    3.75
[s.sub.2]    2.00    3.00    4.50    0.57    1.07    1.58
[s.sub.3]    1.00    1.00    1.00    0.32    0.58    0.88
[s.sub.4]    2.33    3.50    4.50    1.00    1.00    1.00

Table 9. Comprehensive fuzzy judgment matrix of [S.sub.11],
[S.sub.12], [S.sub.13], [S.sub.14]

                     [S.sub.11]             [S.sub.12]

[S.sub.11]    1.00    1.00    1.00    1.13    1.67    2.25
[S.sub.12]    1.13    1.67    2.25    1.00    1.00    1.00
[S.sub.13]    0.33    0.47    0.67    0.75    1.08    1.50
[S.sub.14]    0.60    0.88    1.25    0.33    0.50    1.00

                     [S.sub.13]             [S.sub.14]

[S.sub.11]    2.00    2.75    4.00    1.25    2.33    3.00
[S.sub.12]    0.75    1.08    1.50    1.00    2.00    3.00
[S.sub.13]    1.00    1.00    1.00    0.50    0.83    1.25
[S.sub.14]    0.83    1.25    2.00    1.00    1.00    1.00

Table 10. Hierarchy evaluation index system of the University
scientific research project management performance

Target         Rule layer    Weight  Factor layer               weight

The            Project        0.20   Research team quality       0.37
performance    guarantee             Research performance        0.31
evaluation of                        Scientific research         0.14
scientific                             conditions ensure
research                             Project management status   0.18
management     Fund budget    0.30   Fund budget audit           0.45
                                     Budget approval             0.25
                                     Budget implementation       0.30

               Process        0.10   Funds rules construction    0.30
               management            Spending projects and       0.30
                                     Funds management and        0.40

               Project        0.40   Research results            0.46
               performance           Economic benefits           0.26
                                     Social effects              0.28

Table 11. Evaluation of research team quality indexes

Evaluation    Expert 1    Expert 2    Expert 3    Expert 4    average

Well            0.15        0.28        0.19        0.23       0.2125
Good            0.57        0.55        0.59        0.47       0.545
Medium          0.23        0.15        0.19        0.22       0.1975
Bad             0.05        0.02        0.03        0.08       0.045
COPYRIGHT 2014 Digital Information Research Foundation
No portion of this article can be reproduced without the express written permission from the copyright holder.
Copyright 2014 Gale, Cengage Learning. All rights reserved.

Article Details
Printer friendly Cite/link Email Feedback
Author:Bei, Wang; Dongsheng, Liu
Publication:Journal of Digital Information Management
Article Type:Report
Date:Feb 1, 2014
Previous Article:The influence of government intervention on logistics enterprise's adoption of information technology.
Next Article:Study on college student credit evaluation and prediction based on RF algorithm.

Terms of use | Privacy policy | Copyright © 2018 Farlex, Inc. | Feedback | For webmasters