Web-based decision support system for the public sector comprising linguistic variables.

Decision support systems are often demanding in terms of modelling and use, while purchasing software can also represent too large an investment for many organizations. The level of maturity of an organization influences the use (or non-use) of methods and tools for decision support, which is definitely lower in the public sector than in the commercial sector. The public sector uses considerable assets in its operation and investments, and therefore good decisions are of crucial importance for further development (at state, regional and local levels). In this article we present a decision support system which incorporates a process, approach and web-based software that is simple enough to use and be accepted in organizations and environments less inclined to use a systematic approach to decision making. The system is implemented as a web application, and the simplicity of the system is enhanced by the use of fuzzy logic. With this article we are opening discussion on the question of implementing management support systems in the public sector, where, with a suitable approach and the support of responsible persons, such solutions could play an important role. The results of the case studies on the use of the system in Slovenian municipalities indicate that the task will not be an easy one, because the opinions of the participants concerning a systematic approach to decision making are widely divergent.

Keywords: decision support system, decision-making, municipality, fuzzy logic, linguistic variable

Povzetek: V prispevku je predstavljen internetni sistem za podporo odlocanju v javnem sektorju.

1 Introduction

Decision support systems are gaining recognition in the public sector, which seeks solutions to various problems in a number of diverse areas. Many solutions are closely tied to individual fields, such as medicine [1], ecology [2] and spatial planning [3]. Others, in a more general way, are directed towards support in strategic planning and solving problems in management [4], [5]. Lately, due to the redirection of politics away from ascertaining public opinion about the functioning of the public sector towards public engagement and cooperation in decisionmaking processes, the number of solutions in the area of e-democracy is increasing [6], [7], [8], [9]. Support systems and cooperation in decision making are, however, still used mainly in narrow professional circles and have not found their way to political decision makers or to the public [10]. The challenge of successful implementation of a decision support system in the public sector, with engagement over the whole spectrum of decision making, is still unmet.

An important negative effect is also the conviction that there are great differences in decision making in the private and public sectors. This conviction is perpetuated by stereotypes of decision-making processes in both sectors, as shown in Table 1 [11]. The authors of a comparison, Bots and Lootsma [11], argue that all the mentioned approaches, with regard to the areas of operation and specifics of the branch of activity, can occur in either the private or public sector. For this reason, the question of decision making cannot be clearly separated into public and private decision making, yet we must take into account that the public sector has numerous specific features. If we add to this the increasingly emphasized demand for the engagement and co-deciding of the civil society, it becomes clear that in the development of decision support systems for the public sector, as well as in cases of direct transfer of solutions for the private sector to the public sector, we must also take into consideration the specific needs and demands of the public sector. Certain of these demands can be addressed by adapting existing solutions, but there are also numerous issues which demand special treatment and the development of a specific solution adapted to the environment.

When we speak about decision support in the public sector today, it is best to observe the issue in its most general form. Representative democracy, as we know it today, has a range of shortcomings. For this reason and thanks to the development of information and telecommunication technology, the public sector and politicians are seeking possible solutions to enable an approach to participatory democracy of an Athenian type. It requires that citizens be involved in all phases of decision making. They need, therefore, to learn about the problem, its alternative solutions and their implications, and about their own and other participants' interests and constraints. Since these interests may produce conflicts, the citizens need to he able to identify these conflicts and resolve them. It is also necessary that they be able and willing to take responsibility for their decisions [8]. E-democracy is today one of the principal challenges in the development of e-government [10], [15], [16].

Any consideration of decision making in the public sector must take into account that events take place in a triangle--politics ([left and right arrow]) civil society ([left and right arrow]) administration, where the civil society should be understood in the broadest sense as non-political and non-administrative (Figure 1) [7].

[FIGURE 1 OMITTED]

In the figure, arrows indicate influence and circles indicate domains of control. Intersections indicate "transaction zones" where control is negotiated by lobbyists and media, for example, on the left-hand side, intermediary service deliverers on the right-hand side and professional interaction in government boards and committees on the top [17].

Joint decision making by all three groups of participants is possible only if all of them are sufficiently acquainted with the subject of their decision making. Decision making should therefore be treated comprehensively as a process which implements all the phases necessary for high-quality decisions. The general process framework of decision making must take into consideration at least three phases (Figure 2) [18]:

* Formulating: Actors become aware of a decision problem at the "Doing" level, which represents the implementation phase, and initiate a decision process instance. Depending on their own background, experiences and agenda, as well as predefined goals and constraints, they formulate alternatives and criteria while seeking and filtering information about the problem. The produced alternatives and values are then passed to the appraising stage.

* Appraising: The role of actors at this stage is to assess the alternatives produced in the previous stage. Input that has been passed from the "Formulating" phase is evaluated and alternatives examined by the decision makers.

* Evaluating: The actors who are involved in this stage devise a framework for the evaluation of alternative interventions. Decision analysts or expert decision makers calculate the consequences of alternatives and choose a technique for the appraisal of alternative interventions.

Support for the decision-making process must be ensured with appropriate information and telecommunication technology and tools. The question of decision making in the public sector motivated our work and research with the aim of contributing to solving the issue and to adding a new solution to the range of current tools, specific to the environment in question. We are not alone in this, since the necessity of solving this type of problem is recognized throughout the world [11], [15], [16], [19], [20], [21].

[FIGURE 2 OMITTED]

We have established improving the quality of decision making as our foremost goal. The concept of quality decision making is defined by efficiency, effectiveness, future-influencing capacity and legitimacy [22], [11]. Efficiency is the ratio between invested effort and achieved results. In decision making, the internal efficiency that we would normally find in the production of products and services has little significance, since the number of decisions with respect to the time they take can only be an informative indicator, not to specify effectiveness. For this reason, effectiveness is linked to the decision-making results, that is, the effectiveness of reaching goals.

If we wish to achieve the long-term positive effects of the decision, we must set long-term goals, given that such goals are an important element of effectiveness. This especially holds true for the public sector. Legitimate decisions are those which the participants accept, and therefore the views of the participants must be incorporated in the goals, meaning that in assessing the quality of decisions in the public sector we are dealing with only two aspects: invested effort or the time consumed, and effectiveness, which is exemplified in achieving goals and legitimacy. We must chose suitable goals within a reasonable time and then select the optimum path towards reaching them.

The paths to high-quality decisions can be very diverse. In our case we must answer to various desires and the needs of a large number of participants, and therefore cooperation and reaching a consensus is undoubtedly the correct path. Experience indicates that effort invested in finding a consensus is rewarded with better, more innovative and efficient solutions, which are willingly accepted by the key participants [23]. Innes and Booher [24] specified three classes of the effects of the process of finding a consensus, which promise that efforts invested in consensus will be richly rewarded Table 2):

Table 2: Potential outcomes of consensus building [24]

First-order effects

* Social Capital: Trust, Relationships
* Intellectual Capital: Mutual Understanding, Shared
* Problem Frameworks, Agreed Upon Data
* Political Capital: Ability to Work Together for
  Agreed-Upon Ends
* High-Quality Agreements
* Innovative Strategies

Second-order effects

* New Partnerships
* Coordination and Joint Action
* Joint Learning Extends into the Community
* Implementation of Agreements
* Changes in Practices
* Changes in Perceptions

Third-order effects

* New Collaborations
* More Coevolution, Less Destructive Conflict
* Results on the Ground: Adaptation of Cities,
  Resources, Services
* New Institutions
* New Norms and Heuristics
* New Discourses

2 DSS in public sector

Before we start with detailed aspects of the issue, it would be prudent to devote a few words to the definition of decision support systems [12].

Decision Support Systems (DSSs) are interactive computer-based systems intended to help decision makers utilize data and models to identify and solve problems and make decisions. The "system must aid a decision maker in solving unprogrammed, unstructured (or 'semistructured') problems ... the system must possess an interactive query facility, with a query language that ... is ... easy to learn and use" [13]. DSSs help managers/decision makers use and manipulate data, apply checklists and heuristics, and build and use mathematical models. According to Turban [14], a DSS has four major characteristics: it incorporates both data and models; it is designed to assist managers in their decision processes in semistructured (or unstructured) tasks; it supports, rather than replaces, managerial judgment; and its objective is to improve the effectiveness of decisions, not the efficiency with which decisions are being made. The five types of Decision Support Systems are:

* Communications-Driven DSS--uses network and communications technologies to facilitate collaboration and communication;

* Data-Driven DSS--emphasizes access to and manipulation of a time-series of internal company data and sometimes external data;

* Document-Driven DSS integrates a variety of storage and processing technologies to provide complete document retrieval and analysis;

* Knowledge-Driven--intended to suggest or recommend actions to managers. These DSSs are personal computer systems with specialized problem-solving expertise;

* Model-Driven DSS or Model-oriented DSS emphasizes access to and manipulation of a model, e.g. statistical, financial, optimization and/or simulation. Simple statistical and analytical tools provide the most elementary level of functionality.

Most current advanced DSSs are combinations of all, or nearly all, five generic types. In the public sector, as a result of problem scope, social diversity and dynamics, the stakeholder network is generally more complex and less transparent, and its interests are more diverse. The variety of interests in particular seems to favor multi-criteria decision analysis (MCDA) approaches to decision support [11]. Thus at least two of the types of DSSs listed above (communications-driven and model-driven DSSs), especially for the public sector, should be able to handle a multi-criteria decision analysis approach.

Group Decision Support Systems (GDSSs) are interactive, computer-based systems that facilitate the solution of unstructured and semi-structured problems by a set of decision makers working together as a group. A GDSS aids groups in analyzing problem situations and in performing group decision-making tasks. Any of the five generic types of DSSs can be built as a GDSS.

Usually a DSS is tailored to either a specific application area (e.g. strategic planning, water management or policy making) or a particular decision-making phase (e.g. problem fi'aming or decision-tree development), or both. According to Bots and Lootsma [11], they can be divided into three categories.

* Generic DSSs. These decision-support information technology (IT) applications are domain independent (spreadsheets, generic DSSs for conceptual modeling, based on a particular problem solving method, and generic DSSs consisting of electronic meeting systems that support problem solving in a group).

* Domain-specific DSSs. The core of these systems is a model that computes the impact of measures on a given subsystem (economic, biological, or other) and presents the results in tabular or graphic form.

* Phase-specific DSSs. These applications aim to support one particular phase in the decision-making process (problem formulation phase, choice phase, negotiation support systems).

The aim of our research is the development of a model of a decision support system for the public sector and usable solutions for a chosen research environment. In this we have bound ourselves to the principle that the solution must be as general as possible. However, given the fact that for the development of e-democracy the local environment is the most suitable [16], we have decided to focus on local self-government and its key development problem: deciding on investment projects in local communities, with the aim of ensuring good decisions, which is related to the quality of selection of such projects. As the key point in ensuring the quality of decision making, we have focused on cooperation and reaching a consensus in determining that well prepared investment projects will be selected [24] and that they will be possible to realize within the set framework. It will in turn have a beneficial effect on efficient use of the local community's budget and ensure that the chosen investment projects will bring the participants long-term positive results.

A systematic approach to decision making is new to most Slovenian municipalities. For this reason, and because of time and financial limitations, we have limited the scope of the planned model and solutions in the sense of the typology of the projects [11] and a multiphase approach [10], taking into consideration only two groups of participants. The subject of the research was the decision-making process concerning investment projects which are included in the Plan for Programme Development, as well as the annual budget in the local community. The aim of the research was to shape a decision-making model and the use of support system in a chosen environment within the following framework:

* Model-Driven Group Decision Support System based on Multi-Criteria Decision Analysis with suitable elements of Communications-Driven DSS:

* Domain-specific DSS, with the intention nearing a generic DSS, covering local government decision making on investment projects (plans for development programs and investments from the municipal budget);

* Phase-specific DSS with emphasis on the choice phase with the intention to cover some aspects of the problem-formulation phase and negotiation support systems;

* Considering two basic areas of democratic governmental systems (formal politics and administration) with the intention to make the participation of the civil society possible.

The research thesis refers to the feasibility of the model in given circumstances in a given environment and asserts: "The decision support system in a chosen environment and set framework enable simple expressions of appraisal and balanced participation in decision making for all participants and ensure a final solution which the decision makers and responsible persons consider to be suitable". We have checked this with case studies in three Slovenian municipalities.

3 Research design

We have addressed the issue by a review of the literature and by setting basic guidelines for a solution. We then studied the environment for dealing with the issue, i.e. municipalities. Within the aim of our research, that is, finding a solution to the issue, which encompassed a study of documentation and interviews with participants in the decision-making process, we sought answers to the questions of how decision making progresses and why undesired results occur.

We found answers to the following questions: How ...

* does decision making progress in including investment projects in the municipal budget?

* are the interests of various political options asserted?

* are various expert opinions and interests asserted in decision making?

* do expert opinions affect political decisions?

* does adjustment of opinions about individual projects and groups of projects that have been selected take place?

Why ...

* do the selected projects often fail to meet expectations?

* can evidently less suitable projects dominate clearly more suitable ones?

* are attempts in adjusting opinions often unsuccessful?

On the basis of the case study we defined a solution model and developed web-based software for support of the model specified in the decision-making process. The research theses were analysed in case studies in three Slovenian municipalities within which we verified the suitability of the model, functioning of the software and the response of decision makers to this new, systematic approach to decision making.

We first presented the solution and its goals to the leaders in the municipality. We then analysed the situation in the area of investment projects and chose projects (five to seven projects) about which decisions had been made. In two cases the responsible persons in the municipality invited experts and municipal counsellors, while in one case, due to local elections, we decided to first make an appraisal with the municipal expert services and postpone the counsellor appraisals to a time after the elections. The number of questionnaires handed in by representatives of expert services was six, nine and ten, while the municipal counsellors contributed two and four. The assessment was initially conceived as anonymous; however, in the first case it was decided to have personal signatures, and in the second case, a statement of affiliation to the expert services department. In the case of the counsellors the process was anonymous, while in the third case, appraisal by expert services was also fully anonymous.

We processed the results and presented them to the participants in the appraisals. During the presentation we initiated a discussion concerning the usefulness of the approach, and in two cases we made a survey by which we measured the opinions of the participants concerning the approach and the end results. On the basis of the results we have made an appraisal of the model and solutions, and developed guidelines for further steps.

4 Solution model design

4.1 Investigative case studies

The Local Government Act [25] and Municipal Statutes [26] regulate the functioning of municipalities. The statutes define the organisational structure of the municipality in detail (division into sub-units of local self-government- local communities, specification of the committees and boards of the municipal council and the organizational structure of the municipal government). In deciding on investment needs, the public participates through a council of presidents of the local communities, which is the mayor's counselling body, expert staff of the municipal government represented by heads of the department and the mayor's collegium, and municipal counsellors who work through committees and the municipal council. If we add to this the forms of direct public decision making (people's assembly, referendum and public initiative), we find that the organizational structure of decision making in the municipality is sufficiently comprehensive.

However, for efficient progress in decision making this is not nearly enough. We would need a firm framework for decision making which would define the procedures, roles, inter-relations and limitations of the decision-making process. Unfortunately, we have not found this in any of the processed cases.

Due to this, preparation of the projects and preliminary appraisal of the participants' response (civil society, politicians and municipal government) progresses in an unsystematic and non-transparent manner. Investment projects are prepared within narrow political or expert circles, and the number of people who are well acquainted with all the parameters is low. Cooperation and balance between the participants is lacking, and the opinions and arguments of those who think in a different way are often ignored. This causes frequent situations where poorly prepared projects without prior discussion within the civil society and/or interaction between expert staff of the municipal government and/or counsellors reach the phase of final decision making by a poorly informed municipal council.

This situation is an ideal environment for asserting informal or formal power over arguments and the needs and desires of different-thinking people. The municipal government is aware of this problem, but has neither the knowledge nor the motivation to alter the situation. Numerous urgent and "urgent" projects, the first based on the actual needs of the participants and the other supported by informal (political) power, give the government little hope that this situation can be changed.

On the basis of the collected answers in our research, we can confirm that the environment in question is relatively immature in the area of decision making and that municipalities are confronting numerous difficulties:

* unorganized progress of opinion adjustment and deciding on preparing investment projects and their inclusion in the development plan or municipal budget

* powerful influence of the distribution of informal and formal powers in the municipality on the selection of possible investment projects and inclusion of approved projects in the budget

* difficulties in balancing opinions between professional fields, between political options and between or with the civil society

* the mayor's great direct influence on shaping expert opinion and political decisions

* absence of a comprehensive overview of the development of the local community

* poorly informed decision makers and public concerning the plans, realization and effects of the projects

* lack of qualifications and lack of motivation of the municipal government to improve the current situation

* unawareness and lack of motivation of the politicians to cooperate in solving issues a low level of public involvement in the preparation of decisions concerning solutions

Despite the relatively poor state of affairs, we were encouraged by the fact that leading staff of the municipal government understand the problem and are willing to invest the time and effort to find a solution.

4.2 Solution framework

The basic goal of our work is to improve the quality of decision making with the aid of a tool for decision support. Apart from ensuring the best possible participation of well-informed participants in the decision-making process, consensus is the central point of quality and success in decision making [23]. Implementing the principles of new public management in local self-government requires greater differentiation and responsibility in decision making, for which reason the motivation and qualification of professional decision makers and especially municipal councillors is of key importance for successful development of the local community [27].

This is why the model, and with it the solution, had to ensure a decision-making framework in accordance with the conclusions expressed thus far, with a definition of the procedure and roles in decision making through the following steps:

* specification of the selection of alternatives,

* specification of the participants in decision making,

* specification of criteria and limitations,

* appraisal and choice,

* iteration (in case the solution is not satisfactory),

* documentation and archiving, and a simple approach to enable appraisal without special knowledge in the area of decision-making or intuitive response, and by this to attract decision makers to cooperate and to appreciate the results of this method of decision making. Here we must point out the importance of knowledge exchange and constant adjustment, and improvement of the solution with regard to the capacity and motivation of the chosen environment, since this is the only correct way to achieve better decisions [28].

4.3 Solution model

The solution consists of the framework for decision making and web-based software for decision support within the following scope:

* specification of alternatives

* specification of the participants in the decision making

* specification of attributes, criteria and limitations

* designing questionnaires

* appraisal

* analysis of the results and level of consensus

* selection

* export of the results by various cross-sections of the given structure.

The definition of a multiple-attribute decision problem encompasses the following:

* a set of attributes (parameters, factors, viewpoints, views, ranges) C={[c.sub.1], ..., [c.sub.n]};

* a set of alternatives (possibilities, projects, scenarios, actions, goals, purposes) A={[a.sub.1], ..., [a.sub.m];

* specific information in each pair ([a.sub.i], [c.sub.j]);i [member of] {1, ..., n}, j [member of] {1, ..., m}, ascertaining the relative importance of each attribute [c.sub.j]--weight [w.sub.j];

* suitability [r.sub.ij], which is the decision maker's appraisal of the alternative [a.sub.i] with regard to the attribute [c.sub.j];

* the merging function U, by which the appraisals of criteria [r.sub.ij] for individual alternatives are aggregated into joint alternative appraisals;

* in group decision making, the given alternatives are appraised by the set of individuals D--{[d.sub.1], ..., [d.sub.k]}.

The core of the solution is a three-dimensional group multi-attribute decision space:

* the basic structures are decision trees for each individual alternative, where the appraisals of the attributes [c.sub.ij] (leaves) join into the appraisal of the alternative [a.sub.i] by the aggregating function [U.sub.a],

* the individual alternatives [a.sub.i] join into subsets of the set A, [V.sub.k] [member of] P(A) and the common appraisal for the subset of alternatives is given by the aggregating function [U.sub.v],

* the appraisals of individual appraisers [d.sub.l] are joined into group appraisals for all the nodes and leaves of the decision tree by all the alternatives and variants [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII].

For the sake of simplicity, we have upgraded the decision-making model with fuzzy logic methods and implemented the appraisal with linguistic variables [29].

One thinks in terms of descriptive categories for which reason the appraisal by descriptive values demands much less mental effort. An appraisal method that demands less mental effort will be more precise than a method that demands greater mental capacity [30]. We can therefore claim that a descriptive appraisal is more precise than a numeric one. Additionally, a definition of the appraisal by linguistic variables is easier for the appraiser [31], [32]. These are undoubtedly sufficiently substantial arguments to support our approach.

In fuzzy logic theory we can find suitable solutions for joining values, based on the mapping of linguistic values into fuzzy numbers and the use of aggregation operators for fuzzy numbers in making the calculation.

The starting point is Zadeh's definition of linguistic variables [33, 34]:

A linguistic variable is defined by a quintuple (H, T(H), U, G, [??]) in which H is the name of the variable; T(H) (or simplyT) is the term set of H, that is, the set of names for linguistic values H, with each value being a fuzzy variable denoted generically by X and ranging over a universe of discourse U which is associated with the base variable u; G is a syntactic rule (which usually has the form of grammar) for generating names X of values of H; and M is a semantic rule for associating each X with its meaning [??](X), which is a fuzzy subset of U. A particular X, that is, a name generated by G is called a term. A term consisting of a word or words which function as a unit (i.e. always occur together) is called an atomic term. A concatenation of components of a composite term is a subterm. An example of the term set (abbreviated by T instead of T(x)) is:

T = {Reject, Lowest, Very Low, Low, Middle, High, Very High, Highest, Must Be}.

The basic variable u is the assessed probability or degree of support, and comprises the values from the unit interval u [member of] [0,1] [35]. The general rule that assigns a fuzzy set to the term X can be written as: [??](X)= {(u,[[micro].sub.x](U)); u [member of] [0,1]}, which is for the term high:

[??](High) = {(u,[[mu].sub.High](u)); u [member of] [0,1]}.

What we have just written is in fact the definition of a fuzzy set:

Given a universe of discourse U, the fuzzy set [??] in U is given by its membership function [[mu].sub.[??]](u): U [right arrow] [0,1], in which the function [[mu].sub.[??]](u) is interpreted as the degree of membership of u in the fuzzy set [??]. Clearly, the fuzzy set [??] is fully determined by the set of ordered pairs [??] = {(u, [[mu].sub.[??]](u)); u [member of U]}.

Operations with fuzzy sets within given universe of discourse are operations with membership functions, which allows us relatively easy calculations with fuzzy sets defined with sufficiently simple membership functions. Expectations connected to this are fulfilled by Bonissone and Decker with the uniform scale for mapping linguistic conditional terms to fuzzy intervals [35], which are fuzzy subsets in the set &real numbers.

In this manner we have merged the advantages of using linguistic variables with the simplicity of mathematical operations over numeric variables. Thus we have avoided problematic and complex definitions of aggregation functions and unsurpassable limitations in respect of the branching out and size of the decision-making tree in aggregating linguistic values based on logical rules (rule-based aggregation). We have preserved the flexibility of modelling logical rules with a system of weights, which determine at each node the contribution of the child to the appraisal of the parent.

4.4 Software tool characteristics

The software itself consists of four modules (Figure 3). The module for implementing groups of cases is intended for defining the basic parameters of processing. With its help we can define the basic structure of the system and choose the methods of aggregation in tree structures. The number of questions is the number of the leaves of the appraisal tree; the content of the questions and the appraisal scale are determined with regard to the needs of each individual group of cases.

We first determine the decision-makers and the alternatives about which the decisions will be made for each separate case in the module for data collection, and if necessary we merge them into groups of decision-makers and subsets of projects--alternatives. The module also includes a user interface for appraisal. The job of calibrating the mapping is intended for registering the posture and mood of each individual decision-maker concerning the mapping of linguistic values into fuzzy numbers. With the help of the calculation module from the acquired appraisals, we fill in every point (leaf, node or root) of the given structure with three values: linguistic, fuzzy and real (Figure 4).

[FIGURE 3 OMITTED]

[FIGURE 4 OMITTED]

The module for presenting and exporting the results allows presentation and export along various cross-sections of the given structure.

The software tool is a web based application based on PHP 5 with database engine MySQL 5. It allows flexible settings of:

* structure of decision tree,

* appraisal scales (term sets) and mapping functions from linguistic values to fuzzy trapezoidal numbers,

* aggregation functions and distance measures.

With the intention of sharing ideas, knowledge and software itself after some tests the source code will be given in open source community.

The software tool is available at http://www.fu.unilj.si/bsc/. To obtain a user name and password, please contact the author of this article.

5 Case studies

5.1 Settings of the model

The research comprised three case studies in which we could, due to the similarity of the issues, use the same settings of the model, which include the decision-making tree, appraisal scale and functions of the mapping, aggregation, approximation and defuzzyfication.

Decision tree

Starting from the framework of deciding on capital investments in the public sector [36], legally prescribed definitions and the analysis of the method of decision making in local communities in Slovenia, we have determined the structure of the decision tree (Figure 5).

[FIGURE 5 OMITTED]

Appraisal scale and mapping

The appraiser approves each attribute with a linguistic appraisal, which represents the degree of trust in the suitability of the project in terms of the given attribute (Table 3 and Figure 6).

[FIGURE 6 OMITTED]

Arithmetic, aggregation function and distance measure

As a short break, have a look at a graph of a fuzzy number (more precisely, a fuzzy interval or trapezoidal fuzzy number, Figure 7):

[FIGURE 7 OMITTED]

For fuzzy numbers, the computation necessary for algebraic operations are considerably simplified. The calculations within the decision-making framework are only done with positive fuzzy numbers ([[mu].sub.[??]](x)= 0, [for all] x < 0), and therefore only the arithmetic for a positive fuzzy number will be introduced (the definitions (Table 4) comprise the fuzzy numbers [??] = (a,b, [alpha], [beta]) and [??] = (c, d, [gamma], [delta])):

Table 4: Arithmetic operations for trapezoidal fuzzy
numbers [35:230]

Operation                             Result

1/[??]        (1/b, 1/a, [beta]/b(b + [beta]), [alpha]/a(a - [alpha]))

[??] + [??]   (a + c, b + d, [alpha] + [gamma], [beta] + [delta])

[??] - [??]   (a - d, b - c, [alpha] + [delta], [beta] + [gamma])

[??] x [??]   (ac, bd, a[gamma] + c[alpha] - [alpha][gamma], b[delta]
                            + d[beta] - [beta][delta])

[??]/[??]     (a/d, b/c, a[delta] + d[alpha]/d(d + [delta]), b[gamma]
                            = c[beta]/c(c - [gamma]))

For the model to work, it will also need a aggregating operator. Following the simplicity principle, we have opted among the many operators for generalised operators of weighed mean expressed in the formula:

[h.sup.w.sub.[alpha]] ([a.sub.1], ..., [a.sub.n]) = [([[summation].sup.n.sub.i=1][w.sub.i][a.sup.[alpha].sub.i]).sup.1/[alpha]], [a.sub.i] [member of] [0,1], i [member of] [N.sub.n], [alpha] [member of] R ([alpha] [not equal to] 0)

where for the components of the vector [??] = ([w.sub.1], ..., [w.sub.n]) holds [[summation].sup.n.sub.(i=1)][w.sub.i] = 1, [w.sub.i] [greater than or equal to] 0 [for all] i [member of] [N.sub.n]. The vector [??] is termed the weighed vector, and its components [w.sub.i] the weights. In the simplest version (equal weights [w.sub.i] = 1/n and [alpha] = 1) it is simply the arithmetic mean.

The final results of these calculations, trapezoidal fuzzy numbers, are not suitable for the presentation of results to appraisers. We must therefore map them back to linguistic values. We must find the linguistic value of which the fuzzy equivalent is the closest to the given trapezoidal fuzzy number.

For this purpose we need a metric of the fuzzy sets. The Tran-Duckstein distance takes into account the fuzziness of the fuzzy numbers and is confirmed in practice in an environmental-vulnerability assessment. We have, therefore, decided to choose it for our framework. For trapezoidal fuzzy numbers the general definition is simplified as (f([alpha]) = [alpha], [??] = (a, b, [alpha], [beta]), [??] = (c, d, [gamma],[delta]6)):

Figure 8: Tran-Duckstein distance for generalized left-right
fuzzy numbers (GLRFN) [37:340].

[D.sup.2.sub.t] ([??],[??],[alpha]) = [(a + b/2 - c + d/2).sup.2]
                                      + 1/3 (a + b/2 - c +d/2)
                                        [beta] - [alpha] - [delta] +
                                        [gamma]]
                                      + 2/3 [(b - a/2).sup.2] + 1/9
                                        (b - a/2) [beta] + [alpha]]
                                      + 2/3 [(d - c/2).sup.2] + 1/9
                                        (d - c/2) [delta] + [gamma]]
                                      + 1/18 [[[beta].sup.2] +
                                        [[alpha].sup.2] +
                                        [[delta].sup.2] +
                                        [[gamma].sup.2]]
                                      - 1/18 [[alpha][beta] +
                                        [gamma][delta]]
                                      + 1/12 [[beta][gamma] + [alpha]
                                        [delta] + [beta][delta] +
                                        [alpha][gamma]]

Calculations

The linguistic values of the leaves are the direct result of the appraisal process, and the equivalent fuzzy numbers are the images of a simple mapping between them (Figure 6). The values of the parent nodes are calculated from the leaves towards the root of the tree as fuzzy arithmetic mean of fuzzy values of the children

[[??].sub.i,j] = 1/[K.sub.i,j] [[summation].sub.k] [[??].sub.i+1,j,k]; i = i = 1, ..., 1; j = 1, ..., [J.sub.j]; k = 1, ..., [K.sub.i,j];

where I is the number of levels of the tree, i is the current level of the tree, [J.sub.i] is the branching of the tree, j is the position of the node at the i-th level, [K.sub.ij] is the number of children of the parent in question at the level i+1, and k is the position of the child of the parent in question.

The calculated trapezoidal fuzzy numbers [??] are approximated back to linguistic values L so that the closest representative [??], of the linguistic values [L.sub.i] is found:

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII].

For higher granularity of the end results we introduced the approximation deviation. This is defined as the relative number of the difference in distance of the approximated fuzzy number and the fuzzy number image of the linguistic approximation and the difference between two adjacent linguistic values:

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]

The approximation with the deviation is then labelled as:

[left arrow] [L.sub.app], if dev < - 0,25

[L.sub.app], if - 0,25 [less than or equal to] dev [less than or equal to] 0,25

[L.sub.app] [right arrow], if Dev < 0,25.

This is our original solution, distinguished by its clarity, which allows precise and efficient presentation of the results to the decision makers [38].

The values of all the variables in the tree are merged into an appraisal of the variants V of the plan for the development programmes and into joint group appraisals of several appraisers a. This aggregation is also done with the calculation of the fuzzy arithmetic mean of the fuzzy numbers:

[[??].sub.i,j] = 1/[absolute value of G] x [absolute value of V] [[summation].sub.G] [[summation].sub.V] [[??].sub.i,j]; G [member of] P(D); V [member of] P(A); i = 1, ... I;j = 1, ... [J.sub.i];

for all subsets of the set of alternatives A and the set of appraisers D, for which it is reasonable in the given case.

5.2 Results

The case studies derive from the execution of the model in the chosen municipalities with the following steps:

* presenting solutions to the leaders (management director, heads of departments)

* preparation, adjustment and certification of the appraisal plan

* specification of the decision makers and groups

* specification of alternatives--projects

* presentation of the model and procedure to the appraisers

* executing the appraisal

* presentation of the results and discussions

* appraisal of the solution model

A detailed report about the progress and results of the case studies would unfortunately exceed the scope of this article, and we have therefore focused on the results, connected to the thesis of feasibility and usefulness of the model presented in the introduction. We verified the thesis by two methods:

* leading a discussion with the appraisers after presenting the results of the appraisal

* with a questionnaire about the progress and usefulness of the solution

The discussion revealed that the municipal government was well aware of the question of decision making. Municipal counsellors were less forthcoming.

They recognised the question of decision making, but they did not accept it as theirs.

The appraisers filled in the questionnaire in two cases, but only representatives of the municipal government responded. We have presented the appraisals in Appendix 1 at the end of this article.

The questionnaire referred to the following elements of the appraisal and attitude of the participants in the appraisal of the presented solutions:

1. Dissatisfaction with existing decision-making methods.

2. Willingness to cooperate in implementing new methods and approaches.

3. The method allows for easy expression of opinions about the projects and efficient cooperation in decision making.

4. The results will contribute to faster and better choices of projects.

5. The questionnaire is easily understood and allows for good expression of opinions.

6. I am content with the proposed decisions.

7. All the chosen projects are acceptable to me.

8. l wish to use the method in the future.

The results showed two very different facets of the municipalities. In the first municipality the collective satisfaction rating was High, an estimation given by all but one appraiser, who responded negatively (rejected) to the proposed choice of projects (Statement 7). Among other appraisers one could feel that all the proposed projects were not fully acceptable to them. Nevertheless, lower ratings than High were generally rare, so that the collective ratings, except two which refer to the results of the ratings themselves (Statements 6 and 7), were High. The highest estimated statement about willingness to cooperate in introducing new methods was Very High.

The results in the second municipality showed that the appraisers had different views of the presented method. The collective rating Medium was the result of three High and Medium ratings each and four ratings of Low. The collective rating Medium was given to almost all the individual statements, with the exception of willingness to cooperate in implementing new methods. Estimation of the method (Statements 3, 4 and 5) was in the opinion of most appraisers in accordance with their willingness to use the method in the future. Only one appraiser deviated from this pattern, who estimated willingness for future use of the presented method as High, but gave much lower ratings for the method.

The results of the case studies more or less confirm the thesis that the presented solution enables simple expression of the estimations in the municipalities, as well as balanced participation in decision making and ensures a final result which the decision makers and responsible persons accept as suitable. Of course, we cannot ignore the facts that necessitate an appropriate level of caution in confirming the thesis, since we have done the survey in only two municipalities. The appraisers in the second municipality were quite critical of the method, and we have yet to discover the reasons for such differences among appraisers in the same municipality. In any case we can claim that the method is suitable and enables, with correct preparation, swift execution of the appraisal procedure without major difficulties. Most of the participants in the three cases studies responded well to the method, and we thus can expect a positive response in the future.

We have attached the questionnaire and the appraisal results in both cases to this article.

6 Discussion

On the basis of the literature and studies of the situation and circumstances in the chosen environments we formulated certain principles and developed a decision-making model for investment projects in the public sector. We implemented a general model, which we have concluded must be simple for use in collecting data as well as in presentation of results, in a web-based software solution and tested it in three case studies. The study showed that we have fulfilled the requirement for simplicity and that the appraisers recognized the results as legitimate. We have confirmed the research statement within the given framework and have thus confirmed the approach and solution as a suitable tool for decision support in the public sector.

The formulated principles for the general decision-making model proved to be suitable and their realisation led to the successful execution of the appraisals. Our expectations concerning the approach to appraising and the quality of the results were met [30]. The chosen limitations described at the end of the first chapter prevented us from declaring the model a comprehensive solution to the question of decision making in the public sector. It will still have to be extended and generalized, which means that we must surpass the limitations and expand the model from the domain of local government to other domains, cover all the phases of decision making starting with recognition and definition of the problems and engage all participants involved in one way or another in the decision-making process.

The implemented solution showed that our approach is suitable and that we can ensure good-quality decision making. The approach with linguistic variables simplified the system, which is especially important from the point of view of presenting the results. A two-stage presentation of the results (with deviations and only with basic values of linguistic variables) allows overview on two levels of resolution (23 values and 7 values). Due to the limitations of the software, we could only test one setting of the model. This limitation will be rectified in the future version of the solution.

We have confirmed the research thesis in a given context, which is only the first step to comprehensive confirmation of the model as a good solution for the problem in question. Our expectations that the solution would make the decision process easier have been met. Unfortunately we were unable to attract a larger number of municipal counsellors. We therefore foresee additional presentations to the municipal council for the next appraisal, by which we will ensure the suitable participation of municipal counsellors. In addition, the expansion of the model to all phases of decision-making and to other domains awaits us in the future. We will devote special attention to the question of reaching a consensus and the quality of decision-making, and develop a method for assessment of the quality of decision-making in the public sector, which will enable us to prepare a comprehensive estimation of the circumstances of decision-making in given environments and the usefulness and quality of the presented decision support model.

7 Conclusion

Using fuzzy logic, we extended the selection of cases for the use of these methods. By mapping linguistic variables into fuzzy numbers we avoided the limitations connected to indirect aggregation of linguistic values [39], [40] and to the breadth of the tree structure stemming from the approaches based on defining the values of the parents with logical expressions for all possible values of the children [41]. In this manner we managed aggregation without limiting the number of leaves, levels, alternatives and appraisers. We also enabled the formation of subsets of alternatives--variants and subsets of appraiser groups. The calculations are simple enough so that the applied system is not demanding in terms of computer capacity. Of course new dangers and limitations accompany new approaches. Since there are no directly comparable systems we will need more time and effort to confirm the results. This is similar to mapping the values of linguistic variables into fuzzy numbers. There are only a few cases in use, and it will require additional time and effort for further development in this area.

Implementation of the model in an environment of local self-government is a contribution to the development of decision support in a local environment and thus a contribution to the development of e-democracy in the matter of public co-deciding. The possibility of formation of variants (subsets of alternatives) is especially important here, as well as the groups (subsets of appraisers) that can profitably be used in seeking a consensus. Activation of the public in recognising the problems of decision-making and engaging them in seeking a consensus can considerably contribute to better understanding between all the participants in the local community [24].

The proposed solution is suitable for any system in which we wish to aggregate and compare values of various types of variables in organizations and systems with a hierarchical structure of goals and indicators, since the new version will accept all three types of entry data (numeric, linguistic and fuzzy numbers) and convert them into the other two forms according to rules prescribed for implementation of the solution. This will enable not only comparison of various types of variables, but also a flexible adaptation of the conversions, which will additionally enable comparability between the same types of variables from various definition areas.

Our article is a small piece in a mosaic of activity and research in the area of e-democracy [15]. It upgrades electronic election systems in which the voters choose between confirming or rejecting an individual alternative and enables estimation of the level of agreement among the participants. Based on fuzzy logic, which facilitates the comparability of various indicators, the solution also becomes a tool for monitoring success and outcomes of the functioning of the public sector.

9 Appendix

The data captured in two case studies and the appraisal results are presented in two following tables. The first information in appendix is the legend of tables which contains the labels of input data (variables--questions, Table 5) and the labels of linguistic values (results, answers, Table 6).

The data of the case studies are given in the rows of the tables, while the results calculated are presented in the last columns (overall appraisal of the model given by a particular appraiser) and in the last rows (collective appraisal of an attribute), thus the collective overall appraisal occupies the outmost right cells in the last rows (Table 7 and Table 8).

Received: September 7, 2006

8 References

[1] Plaper, G., Piletic, M., Bohanec, M., Rajkovic, V., Urh, B. (1999): "Hierarchical multi-attribute models for decision support in the management of diabetic foot syndrome". In: Kokol, P., Zupan, B., Stare, J., Premik, M., Engelbrecht, R.: Medical Informatics Europe '99, Studies in Health Technology and Informatics, Vol. 68. Amsterdam [etc.]: IOS Press; Tokyo: Ohmsha, pp. 970-972.

[2] Hamalainen, R., Kettunen, E., Marttunen, M., Ehtamo, H. (2001): "Evaluating a Framework for Multi-Stakeholder Decision Support in Water Resources Management", Group Decision and Negotiation, Vol. 10, No. 4, pp. 331-353.

[3] Onate, E., Piazzese, J. (2003) Decision support system for risk assessment and management of floods. International Center for Numerical Methods in Engineering (CIMNE) Gran Capitan, s/n, Campus Norte UPC, 08034 Barcelona, Spain. <http://www.eu-lat.org/eenvirorn/Onate.pdf> (accessed July, 2006).

[4] Benkovic, J., Bohanec, B., Rajkovic, V., Vrtacnik, M. (1998): "Knowledge-based evaluation of higher education institutions", Preprints of the 6th IFAC-Symposium "Automated systems based on human skill", Kranjska Gora, Slovenia, Pergamon Press, pp. 157-160.

[5] Di Mauro, C., Nordvik, J.P., Lucia, A.C. (2002): Multi-criteria decision support system and Data Warehouse for designing and monitoring sustainable industrial strategies--an Italian case study. European Commission--Directorate General JRC Joint Research Centre, Ispra Institute for the Protection and Security of the Citizen (IPSC) Technological and Economic Risk Management Unit 1-21020 Ispra (VA), Italy <http://www.iemss.org/iemss2002/proceedings/pdf/ volume%20uno/434_dimauro.pdf> (accessed July, 2006).

[6] Bouras, C., Katris, N., Triantafillou, V. (2003): "An electronic voting service to support decision making in local government", Telematics and Informatics, Vol. 20, Issue 3, pp. 255-274.

[7] Gronlund, A. (2003): "e-democracy: in search of tools and methods for effective participation", Journal of Multi-Criteria Decision Analysis, Vol. 12, No. 2-3, pp. 93-100.

[8] Kersten, G.E. (2003): "e-democracy and participatory decision processes: lessons from e negotiation experiments", Journal of Multi-Criteria Decision Analysis, Vol. 12, No. 2-3, pp. 127-143.

[9] Walker, W.E. (2000): "Policy analysis: a systematic approach to supporting policymaking in the public sector", Journal of Multi-Criteria Decision Analysis, Vol. 9, No. 1-3, pp. 11-27.

[10] Gammack, J., Barker, M. (2003): E-democracy and public participation. a global overview of policy and activity. School of Management, Griffith University Queensland.

[11] Bots, P.W.G., Lootsma, F.A. (2000): "Decision support in the public sector", Journal of Multi-Criteria Decision Analysis, Vol. 9, No. 1-3, pp. 1-6.

[12] Power, D.J. (1999): "Decision Support Systems Glossary", DSS Resources, World Wide Web, <http://DSSResources.COM/glossary/> (accessed July, 2006)).

[13] Bonczek, R.H., Holsapple, C.W., Whinston, A.B. (1981): Foundations of Decision Support Systems. Academic Press, New York.

[14] Turban, E. (1995): Decision Support and Expert Systems. Management Support Systems. 4th Edition, Macmillan Publishing Company, New York.

[15] Coleman, S., Norris, D.F.: (2004); "A New Agenda for e-Democracy", Forum Discussion Paper No. 4, Oxford Internet Institute. www.oii.ox.ac.uk/resources/publications/FD4.pdf> (accessed July, 2006).

[16] Rosen, T. (2001): E-democracy in practice, Swedish experiences of a new political tool. <http://www.sociedadinformacion.unam.mx/reposit orio/documentos/E-democracySwedish.pdf> (accessed July, 2006).

[17] Molin, B., Mansson, L., Stromberg, L. (1975): Offentlig forvaltning, (Public Administration). Bonniers.

[18] Papamichail, K.N., Robertson, I. (2003): "Supporting societal decision-making: a process perspective", Journal of Multi-Criteria Decision Analysis, Vol. 12, No. 2-3, pp. 203-212.

[19] Perri, 6 (2004): Re-Wiring Decision-making for Local Government, Not Local Administration. <http://www.egovmonitor.com/features/perri6.html > (accessed July, 2006).

[20] Mittuniversitetet (2006): Decision analytic support tools in e-government. <http://www.miun.se/mhtemplates/MHPage_23 601.aspx> (accessed July, 2006).

[21] Podger, A. (2002): Improving Government decision-making. <http://www.apsc.gov.au/media/podger310502.htm > (accessed July, 2006).

[22] Dror, Y. (1997): "Strengthening government capacity for policy development", International Journal of Technical Cooperation, Vol. 3, No. 1, pp. 1-15.

[23] Harter, P.J. (1997): "Fear of commitment: an affliction of adolescents", Duke Law Journal, No. 46, pp. 1389-1428.

[24] Innes, J.E., Booher, D.E., (1999): "Consensus Building and Complex Adaptive Systems--A Framework for Evaluating Collaborative Planning". Journal of the American Planning Association, Vol. 65, No. 4, pp. 412-423.

[25] Local Government Act (officially approved text) (2005)/ZLS-UPB 1/(Ur.1. RS, No. 100/2005).

[26] Municipality of Koper (2006): Statute of the Municipality of Koper (unofficially approved text). <http://www.koper.si/dokument.aspx?id=11415> (accessed July, 2006).

[27] Hansen, K. (2001): "Local Councillors: Between Local Government and Local Governance", Public Administration, Vol. 79, No.1, 2001 (105-123).

[28] Roberto, M.A. (2005): "Why making decisions the right way is more important than making the right decisions", Ivey Business Journal Online, Ivey Management Services, Richard Ivey School of Business. <http://www.iveybusinessjournal.com/view_article. asp?intArticle_ID=578> (accessed July, 2006).

[29] Zimmerman, H.J.: Fuzzy Set Theory and Its Applications, 4th ed. Kluver Academic Publishers, Boston/Dordrecht/London, 2001, p. 514.

[30] Zimmer, A.C. (1986): "What Uncertainty Judgement Can Tell About the Underlying Subjective Probabilities". In: Uncertainty in Artificial Intelligence. Machine Intelligence and Pattern Recognition, Kanal, L.N., Lemmer, J.F., Amsterdam, Elsevier Science Publisher B.V., Vol. 4, pp. 249-258.

[31] Chin, W.C., Ramachandran, V. (2000): "Fuzzy Linguistic Decision Analysis for Web Server System Future Planning", International Journal of the Computer, the Internet and Management, Vol. 8, No. 3.

[32] Herrera, F., Herrera-Viedma, E., Lopez, E. (1996): "On the Linguistic Approach in Multi-Person Decision-Making", International Conference on Intelligent Technologies in Human-Related Sciences, 1THURS'96. Leon (Spain), pp. 205-213.

[33] Zadeh, L. A. (1975): "The concept of a linguistic variable and its application to approximate reasoning--I", Information Sciences, No. 8, pp. 199-249.

[34] Zadeh, L. A. (1973): The concept of lingustic variable and its application to approximate reasoning. Memorandum ERL-M 411, Berkeley, October 1973.

[35] Bonissone, P.P., Decker, K.S. (1986): "Selecting Uncertainty Calculi and Granularity: An Experiment in Trading-off Precision and Complexity". In: Uncertainty in Artificial Intelligence. Machine intelligence and Pattern Recognition, Kanal, L.N. and Lemmer, J.F., Amsterdam, Elsevier Science Publisher B.V., Vol. 4, pp. 217-247.

[36] GAO/AIMD-98-110 (1998): Leading Practices in Capital Decision-Making, U.S. General Accounting Office, Washington, http://www.gao.gov/special.pubs/ai99032.pdf (accessed, June 2006).

[37] Tran, L., Duckstein, L. (2002): "Comparison of fuzzy numbers using a fuzzy distance measure", Fuzzy Sets and Systems, No. 130, pp. 331-341.

[38] Bencina, J. (2004): Optimisation of the selection of investment projects in the public sector by methods of mathematical programming and fuzzy logic. Doctorial dissertation. Ljubljana: Faculty of Economics.

[39] Herrera, F., Herrera-Viedma, E. (1997): "Aggregation operators for linguistic weighted information", IEEE Transactions on Systems, Man, and Cybernetics, Vol. 27, pp. 646-656.

[40] Herrera, F., Martinez, L. (1999): A 2-tuple Fuzzy Linguistic Representation Model for Computing with Words. Dept. Computer Sciences and A.I., Granada University, Technical Report#DECSAI-990102. <http://citeseer.nj.nec.com/herrera99tuple.html]> (accessed May, 2006).

[41] Von Altrock, C. (1997): Fuzzy Logic and NeuroFuzzy Applications in Business and Finance. Prentice Hall, Upper Saddle River, New Jersey.

Joze Bencina

University of Ljubljana, Faculty of Administration

Gosarjeva 5, Ljubljana

E-mail: [e-mail]

Table 1: Perceptions of decision-making processes

Private sector              Public sector

Decisions are made by a     Decisions are not made but
single agent (individual    "happen" as a result of a
manager or management       complex interaction
team) whose authority is    between administrators,
defined by a hierarchical   trade unions, pressure
organization structure,     groups, etc.

Decisions are dominated     Decisions involve many
by a single interest,       and often divergent
typically the competitive   interests of a society, and
position of the company,    aggregation into such

Table 3: Linguistic values and equivalent fuzzy
trapezoidal numbers

Term           Fuzzy number         Label

Reject            0 0 0 0         [L.sub.1]
Lowest        .01 .02 .01 .05     [L.sub.2]
Very Low      .l .18 .06 .05      [L.sub.3]
Low           .22 .36 .05 .06     [L.sub.4]
Medium        .41 .58 .09 .07     [L.sub.5]
High          .63 .80 .05 .06     [L.sub.6]
Very High     .78 .92 .06 .05     [L.sub.7]
Highest       .98 .99 .05 .01     [L.sub.8]
Must Be           1 1 0 0         [L.sub.9]

Table 5: Labels of input variables

[c.sub.1]     Dissatisfaction with existing decision-making
              methods.

[c.sub.2]     Willingness to cooperate in implementing new
              methods and approaches.

[c.sub.3]     The method allows for easy expression of
              opinions about the projects and efficient--cooperation
              in decision-making.

[c.sub.4]     The results will contribute to faster and better
              choices of projects.

[c.sub.5]     The questionnaire is easily understood and allows
              for good expression of opinions.

[c.sub.6]     I am content with the proposed decisions.

[c.sub.7]     All the chosen projects are acceptable to me.

[c.sub.8]     I wish to use the method in the future.

Table 6: Labels of linguistic values

Label          Term

[L.sub.1]      Reject
[L.sub.2]      Lowest
[L.sub.3]      Very low
[L.sub.4]      Low
[L.sub.5]      Medium
[L.sub.6]      High
[L.sub.7]      Very high
[L.sub.8]      Highest
[L.sub.9]      Must be

Table 7: Data and results of the first case study

            [C.sub.1]       [C.sub.2]        [C.sub.3]

1               L8              L5              L6

2               L8              L8              L7

3               L5              L8              L6

4               L7              L7              L6

5               L3              L6              L5

6               L7              L8              L6

[SIGMA]         L6              L7              L6

            [C.sub.4]       [C.sub.5]        [C.sub.6]

1               L6              L7              L6

2               L6              L6              L6

3               L6              L6              L5

4               L6              L5              L5

5               L5              L7              L7

6               L6              L7              L6

[SIGMA]         L6              L6              L6
                                           [left arrow]

            [C.sub.7]       [C.sub.8]         [SIGMA]

1               L1              L6              L6
                                           [left arrow]

2               L5              L6         [right arrow]
                                                L6

3               L5              L7              L6

4               L5              L7              L6

5               L5              L7         [right arrow]
                                                L5

6               L6              L7              L7
                                           [left arrow]

[SIGMA]         L5              L7              L6
           [left arrow]    [left arrow]

Table 8: Data and results of the second case study

            [C.sub.1]       [C.sub.2]        [C.sub.3]

1               L5              L5              L6

2               L1              L7              L3

3               L5              L8              L6

4               L8              L5              L6

5               L5              L5              L4

6               L5              L7              L5

7               L2              L6              L4

8               L7              L7              L1

9               L7              L7              L8

10              L7              L6              L7

[SIGMA]         L5              L6         [right arrow]
                                                L5

            [C.sub.4]       [C.sub.5]        [C.sub.6]

1               L5              L5              L5

2               L7              L5              L5

3               L5              L5              L5

4               L6              L6              L5

5               L4              L5              L5

6               L5              L6              L5

7               L2              L5              L5

8               L1              L1              L5

9               L7              L7              L5

10              L7              L7              L5

[SIGMA]         L5              L5              L5

            [C.sub.7]       [C.sub.8]         [SIGMA]

1               L5              L5              L5

2               L5              L5              L4
                                           [left arrow]

3               L5              L7         [right arrow]
                                                L5

4               L5              L6              L6
                                           [left arrow]

5               L5              L4         [right arrow]
                                                L4

6               L5              L5              L5

7               L5              L6         [right arrow]
                                                L4

8               L5              L2              L4

9               L5              L7              L6

10              L5              L7              L6

[SIGMA]         L5              L5              L5