Customer satisfaction evaluation (CSE) for knowledge based services from public funded (S&T) R&D organisations.

ABSTRACT

Council of Scientific & Industrial Research (CSIR), India has undertaken Customer--Supplier (C-S) approach in its eight constituent Chemical Sciences Laboratories to evaluate the customer satisfaction originating from the "What-Based" factors of projects undertaken in terms of objectives, schedules, output and utilities. CSE enabled Indian Institute of Chemical Technology (IICT), Hyderabad, India a constituent Laboratory of CSIR to obtain direct and valuable feed back from its customers to improve the planning, execution and delivery of the projects undertaken by the institute. CSE study revealed that an average CSE of IICT is around 3.32 on the scale of 4.0 points for the past six years (1997-2003). This paper analyses CSE methodology adopted and results obtained pertaining to CSE grades awarded by IICT customers, category of user agencies by investment basis, type of agency, types of projects undertaken etc. Conclusions drawn indicate that this CSE methodology can be adopted for knowledge-based services in chemical sciences laboratories for improving its customer satisfaction levels.

KEYWORDS: Customer Satisfaction Evaluation, Knowledge Based Services, Public Funded (S&T) R&D organizations, Technology Management.

1. INTRODUCTION

Knowledge generation and its management in public funded S&T (R&D) organization in present day world of market economy has become more important due to business driven R&D approach. R&D organizations continuously need to create, capture, harness and share knowledge with user agencies with custom centered approach through most of the times importance of customer is realized yet many a times unknowingly customer is taken for granted by many public funded R&D organizations. An attempt is made in this article to analyze and share the experiences of Indian Institute of Chemical Technology (IICT), which is pursuing R&D in chemical technology areas and catering to the needs of different sectors of agencies (national & overseas).

2. INDIAN INSTITUTE OF CHEMICAL TECHNOLOGY (IICT)

IICT, a constituent laboratory of Council of Scientific & Industrial Research (CSIR), New Delhi, has been pursuing R&D in Chemical Sciences & Technology for the last five decades and developed expertise in several R&D areas and set up centers of excellence. IICT major R&D areas are drugs & pharmaceuticals, natural products chemistry, agrochemicals & pheromone chemicals, flouro organics, specialty & fine chemicals, bulk organics & intermediates, polymers and surface coatings, lipid science & technology, coal, gas & energy, biochemical and environmental engineering, chemical engineering, mechanical design & engineering. IICT R&D areas of excellence are organic chemistry, nano technology, catalysis, lipid chemistry, novel drug delivery systems, polymers, chemical process design, mechanical & design engineering. (www.iictindia.org)

3. IICT CUSTOMER SPECTRUM & NETWORK:

IICT customer's family is networked with wide spectrum of industries such as national (Public, Private, Govt.), overseas clients and social status. However, IICT does not differentiate between the services it offers to different customers. In India industries, on investment basis, are categorized as SSI (Plant & Machinery cost up to Rs.100 Lakhs), MSI (Project cost less than Rs.500 lakhs), LSI (Project cost more then 500 lakhs)

IICT undertakes wide range of assignments for its customers. At any point of time about 60-70 projects are in pipeline and almost equal number are completed every year. The major assignments/projects of IICT may be categorized as. Contract R&D projects (Sponsored/Collaborative/Grant-in-Aid), consultancy projects (Scientific, Engineering, Technical etc) custom synthesis, S&T Services (Trouble Shooting, Testing & Evaluation, Technical Reports, Training etc).

The duration of IICT projects vary from 3 months to 36 months and project cost ranges from Rs.0.50 lakhs to Rs.2000 lakhs (National) US$50,000 to US$300,000 (Overseas). External cash flow of IICT during last 5 yeas (2000-05) was Rs.71 crores (USD 16 Million), which is about 40% of budget received by IICT from CSIR. Wide variation in value of projects undertaken is due to the commitment of IICT to (a) cater to the needs broad spectrum of clients and (b) to meet societal obligations being a public funded R&D organization.

4. CUSTOMER SATISFACTION EVALUATION (CSE)

CSIR has initiated a study of CSE among eight chemical sciences laboratories. The main objective of the study is to obtain the right kind of feed back from the customers for further improvement of R&D services. To fulfill this objective, a format is prepared and sent to the customers of all completed projects during previous years starting from 1997-98 to obtain feedback. Customer--Supplier (C-S) approach is followed to obtain customer satisfaction originating from the "What--Based" factors of projects undertaken in terms of project objectives, schedules, output and utility. CSE enabled IICT to obtain through CSIR CSE cell valuable feed back from its customers to improve the project planning, execution and delivery of the projects undertaken by the institute. Based on the customer feed back, Customer Satisfaction Index (CSI) is arrived on 4x4 point scale. The CSI data of IICT for five years (1997-98 to 2002-03) revealed that most of the IICT customers are satisfied. Further analysis of the data is discussed below in detail and conclusions are drawn.

5. ANALYSIS AND DISCUSSION ON OUTCOME OF CSIR CUSTOMER FEED BACK

5.1. Customer Satisfaction Index (CSI) Feed back From Customers.

Total projects evaluated for IICT for CSI is 101 for the period of six years. CSI ranges from 3.25(1998-99) to 3.47 (2002-03). CSI per project ranges from 0.55 (1997-98) to 0.13 (2002-03). However, the percentage of CSI on four point scale is consistently above 80% and ranges from 86.7 (2002-03) to 80% (1998-99). Cumulative CSI i.e., CSI multiplied by total No. of project in corresponding year gives reasonable indication of customer satisfaction of the institute. Cumulate CSI ranges from 79 (2002-03) to 19.8 (1997-98). Table-1 gives details of CSI of IICT and Chemical Science Laboratories of CSIR.

The four factors i.e. (1) Project Objectives (2) Time schedules (3) Output accomplishments (4) Usability of outputs received from customers are analyzed and discussed in Table-2.

5.1.1. Achieving the Project Objectives

The objective of the each project is discussed with the customer and finalized before signing the agreement or undertaking the assignment. The feed back from 101 projects shows that 33 customers (23 SSI, 4 MSI, 6 LSI) awarded Grade-A (more than stated), 59 (23 SSI, 14 MSI, 22 LSI) awarded Grade-B (as stated), 9 (4 SSI, 2 MSI, 3 LSI) awarded Grade-C (less than stated but significant) and none awarded Grade-D (insignificant achievement). This indicated that out of 101 customers assessments 33 customers were more than satisfied, 59 were more satisfied and 9 were less satisfied. Fig. 1 gives trends of this data.

[FIGURE 1 OMITTED]

5.1.2. Time schedules of projects

Time schedule of each project is finalized in consultation with the customer. Time schedules have of late become, very significant, as the market conditions are fluid for introducing products/goods into the market as per the demand and supply. The customer feed back reveals that out of 101 customers, 33 (21 SSI, 5 MSI, 7 LSI) awarded 'A' grade (ahead of schedule), 52 (23 SSI, 12 MSI, 7 LSI) have awarded grade 'B' (on schedule), 12 (6 SSI, 3 MSI, 3 LSI) have awarded Grade 'C' (behind schedule due to major constrains) and 4 (4 LSI) awarded grade 'D' (behind schedule). There are internal and external reasons for delay in completing the project work due to technical and scientific reasons. However, IICT endeavors to meet the time schedules as per the agreement terms due to market push and pull reasons. Fig. 2 gives trends of the data.

[FIGURE 2 OMITTED]

5.1.3. Accomplishment of Project Outputs

Scope of work is clearly defined in the agreement or assignment document based on the discussions with customer. The customers are very specific and keen about the project outputs as they want to get back the goods/services for which they are paying consideration. Out of 101 customers assessed, 68 (36 SSI, 13 MSI, 9 LSI) awarded 'A' grade (accomplishment: as is committed: it meets the customers business need and with its application clearly defined), 16 customers (8 SSI, 2 MSI, 6 LSI) awarded 'B' grade (accomplishment: as is committed: it needs the customers business need but its application is not clearly defined), 14 customers (5 SSI, 5 MSI, 4 LSI) awarded 'C' grade (accomplishment: as is committed: but it does not meet the customers business needs and only 3 customers (1 SSI, 2 LSI) awarded 'D' grade (accomplishment : in the report is not as committed in the proposal). Fig. 3 gives trends of this data

[FIGURE 3 OMITTED]

5.1.4. Usability of the Innovation/Output of the Project

The customers feed back reveals that 63 (36 SSI, 9 MSI, 18 LSI) awarded grade 'A' (readily usable: has significant impact and for a prolonged period), 17 (5 SSI, 5 MSI, 7 LSI) awarded grade 'B' (readily usable: has significant impact but not for a prolonged period), 9 awarded grade 'C' (readily usable: but its use does not have significant impact) but 12 (4 SSI, 4 MSI, 4 LSI) customers awarded grade 'D' (innovation or output is not readily usable). Fig. 4 gives trends of this data.

[FIGURE 4 OMITTED]

6. SUM UP OF CSE FEED BACK

The total grades awarded by customers for 101 projects are 404 'A' grades accounted for 49%, 'B' grades 36%, 'C' grades 11% and 'D' grades accounted for 5%. This shows about 16% of the customers are not fully satisfied.

6.1. Customers Profile

[a] Economic size of units (Investment basis)

Customers catered by IICT belonged to three sectors based on economic size of units. Out of 101 customers, 50 were small-scale sector, 20 were medium scale sector and balances 31 were large-scale sector.

[b] User agencing basis

IICT catered to public, private, government and international agencies. Out of 101 customers, 6 international agencies, 76 private sector, 13 public sector and 6 were government organisations.

[c] Categories of projects undertaken

Out of 101 projects analyzed 58 consultancy projects, 39 sponsored projects and 4 were collaborate projects.

[d] Customers category R&D area wise

IICT being multidisciplinary R&D institute customers approached for different S&T projects. Major R&D areas catered to customers are Organic Coatings & Polymers 33%, Chemical Engineering 17%, Analytical Services 14%, Pharmacology 11%, Organic Synthesis 15%, Catalysis 4% and other areas including Agrochemicals 6%.

6.2. Customer Relationship Management For Customer Loyalty

Knowledge based R&D services (KRDS) are different from other services as KM is based on tacit assets. KRDS are customized, individualized and are provided through two-way interaction. IICT developed its customer portfolio by Customer Relation Management (CRM) programs through interaction, relationships, networks and R&D communication channels i.e., papers, patents, seminars/workshops etc., Though IICT customer spectrum is very wide and complex institute could develop loyal customer base. During study period 2% customers approached IICT more than 3 times, 15% more than twice and 83% customers approved once. Out of which 55% customers were old customers. IICT loyal customer base is fairy good and requires to keep quality KRDS to retain them and attract them again and again.

7. DERIVATION OF HYPOTHESIS ON CUSTOMER SATISFACTION

IICT, as an R&D organization, is providing technologies and S&T services to wide spectrum of customers and variety of R&D services. Customer satisfaction Index alone may not give real picture of quality and quantity of R&D services. Hence, it is proposed to test through the following hypothesis after normalizing grades awarded by customers on total customer sample of 101.

H: CSI trends of all customers are same irrespective of the customer status (investment basis) for total four features put together.

Basis: (i) Sample size 101 customers

(ii) Normalization of grades weightage A: 4 Points, B: 3 Points, C: 2 Points, D: 1 point

(iii) Features: (a) Project objectivity (b) Time Schedules (c) Accomplishment of outputs (d) Innovative capability.

7.1. Testing of Hypothesis

Computation of Variance for their means (F-Test)

1.  Sum of Squares (SS) Total    :   1,49,615

2. Sum of Squares (SS) Between   :   1,843

3. Sum of Squares (SS) Within    :   1,47,772

Calculated value F = 0.056, Table Value F (2,9) = 4.25, At 5% of significance. Calculated Value is less than table value. Hence, the proposed hypothesis is accepted (i.e) customer satisfaction trend is same irrespective of their status based on size of firm (investment basis)

8. CONCLUSIONS

IICT since last six decades established a close interaction with its customers. Customer's Satisfaction Index (CSI) of IICT ranged from 3.25 to 3.47 on a 4 point scale where as CSI of chemical sciences laboratories ranged from 3.10 To 3.35, 49% of IICT customer's awarded 'A' grade i.e. 4 out of 4 points, 36% of IICT customer's awarded 'B' grade, i.e., 3 out of 4 points for all the factors put together in CSE study. Below 15% of customers awarded C&D grades, which needed to be worked for improving performance to preferably to 'A' grades. And hypothesis concluded that trends of customer satisfaction are same irrespective of their status (based on investment) when overall customer satisfaction on 4 features is considered.

9. ACKNOWLEDGEMENTS

Author acknowledges Director, IICT, Hyderabad for permitting to use the IICT CSE information for case study and to publish the paper. Author also acknowledges IICT colleagues who are compiling the CSE information for official records.

REFERENCES

CSIR's R&D project assessed by "Customer Assessor Approach Report", March 2000 CSE unity CSIR. "Internal records" of IICT CSE 1997-98 to 2002-2003

Sampath Kumar, G., "M.Phil dissertation (2002-2003)", Alagappa University entitled: "Performance evaluation of public funded Science & Technology (R&D) Organizations"

Jerry Honeyaitt: "Knowledge Management Strategies," Prentice Hall India, New Delhi 2001

Ganesh Natarajan, Sandhya Shekahr : "Knowledge Management", Tata McGraw-hill Publishing Company Ltd. New Delhi 2000

Guus Schreber et al: "Knowledge Engineering and Management", University Press India Ltd., Hyderabad 2001

Griffin, A. and J. Hanser, "The voice of the customer" Report November 92-106, Marketing Science Institute, Cambridge, Massachuellt, March 1922

Sampath Kumar G, Indian Institute of Chemical Technology, Hyderabad, INDIA

Vidyadhar Reddy A, University College of Business Management, Osmania, Hyderabad, INDIA

Sampath Kumar, earned his M.Phil (Management) from Alagappa University in 2003 and M.Tech (Chemical Plant Engineering) from Regional Engineering College Warangal India in 1973. Currently he is senior scientist and Head, Business Management Division of IICT Hyderabad, India. He is catalytic in transfer of more than 100 technologies form IICT to Industry for commercial exploitation. He is visiting faculty in several academic / organizations. He is member of All India Management Association (MIMA), Fellow, Institute of Marketing & Management (FIMM), Fellow, Institute of Engineers (India) (FIE)

Dr. Vidyadhar Reddy Aileni earned Ph.D at Kakatiaya University Warangal, India in 1990 and F.D.P from Indian Institute of Management (IIM), Ahmedabad, India in 1994. Currently he is professor & Dean Dept. of Business Management Osmania University, Hyderabad, India. He is Chairman, Board of Studies Dept. Of Business Management, Osmania University, Hyderabad. He is Member, Association of Indian Management Schools and Fellow, Academy of Marketing Science, USA. Presently consultant for BSNL, a Govt. of India Telecommunications company on media research.

TABLE 1: THE CSI OF IICT DURING 1997-98 TO 2002-03

S.No.   Year       CSI of      No. of     CSI of
                  Chemical    Projects     IICT
                  Sciences    Evaluated
                Laboratories

1      1997-98      3.10          6        3.30
2      1998-99      3.15         24        3.25
3      1999-00      2.98         17        3.32
4      2000-01      3.20         17        3.29
5      2001-02      3.16         14        3.30
6      2002-03      3.35         23        3.47

S.No.   Year    CSI/Project      CCSI      % of CSI
                              (CSI * No.   on 4 point
                             of Projects)    scale

1      1997-98     0.55          19.8        82.5
2      1998-99     0.13           78         80.0
3      1999-00     0.19           56         83.0
4      2000-01     0.19           55         81.1
5      2001-02     0.23           46         82.5
6      2002-03     0.15           79         86.7

TABLE 2: OUTCOME OF IICT CUSTOMER FEED BACK

S.No   Factors          Overall  More than    As stated
                        grade    stated

1      Objectives       33       23   4   6   23  14  22
2      Time schedule    33       23   5   7   23  12  17
3      Output           68       36   13  19  8   2   6
       accomplishments
4      Usability of     63       36   9   18  5   5   7
       innovation /
       output
Total                   197      116  31  50  59  33  52

                                 (Total No. of customer assessed: 101)

S.No   Factors          Overall  Less than        Insignificant
                        grade    stated but
                                 significant
1      Objectives       59       4   2   3   9    --
2      Time schedule    52       6   4   2   12   --  --  4
3      Output           16       5   5   4   14   1   --  2
       accomplishments
4      Usability of     17       5   2   2   9    4   4   4
       innovation /
       output
Total                   144      20  12  12  44   5   4   10

                        (Total No. of customer assessed: 101)

S.No   Factors          Overall
                        grade

1      Objectives       --
2      Time schedule    4
3      Output           3
       accomplishments
4      Usability of     12
       innovation /
       output
Total                   19

TABLE 3: NORMALIZED GRADE POINTS OF CUSTOMER SATISFACTION (INVESTMENT
BASIS)

Grades awarded  Small Scale  Medium Scale  Large Scale   Total
by Customer      Industry      Industry     Industry

A                   116           31           50         197
B                   59            33           52         144
C                   20            12           12          44
D                    5            4            10          19
[SIGMA]x            200           80           124        404
[SIGMA]x2         40,000        6,400        15,376     1,63,216

TABLE 4: SUMMARY CALCULATION OF 'F'

Source           SS        df      MS =SS/df      F

Between Groups  1,843     3-1=2      921.5    921.5/16419

With in Groups  1,47,772  12-3=9     16419    = 0.056

Total           1,49,615  12-1=11