# Modelling and optimizing software quality.

IntroductionLarge software systems are increasingly used in all areas of modern technology including space vehicles, defense systems, manufacturing concerns, and in the service sector. The rise in demand for software in turn requires increases in productivity and quality, and the general success of software development. Effective project management is important to successful development of software. The following definition of software project development is quoted from the Institute of Electrical and Electronics Engineers (IEEE) standards[1]: "Software project management is the process of planning, organizing, staffing, monitoring, controlling, and leading a software project". Project managers deal with important issues such as cost, quality and schedules from the inception of the project. In particular they would like to know how these are interrelated so that efficient resource allocation decisions may be made. This results in customer satisfaction as they would be delighted to have a product delivered reliably, on schedule, and in a cost-effective manner.

Problems with software quality and performance can be minimized by implementing software quality management techniques within the software development process. Good management practices have a definite contribution to the achievement of software quality. These decisions include the distribution of effort throughout the development cycle in order to accomplish the tasks within the budget and on schedule.

Research literature on software project management has addressed time, cost, and quality. But an integrated quantitative approach incorporating these three objectives simultaneously would form a useful management tool. To conduct a successful software development project the project manager should know the levels of resources and effort that should be spent on each stage of the development to produce quality software within cost and time constraints.

The objective of this paper is to develop a model to optimize software quality within budget limitations. Software quality is described through a number of factors such as reliability, maintainability and portability. A model assuming linear relationship between a software quality factor and the efforts spent in the development phases was developed by Suresh and Babu[2]. In this paper we consider a nonlinear relationship between quality factors and the effort spent in the development phases. This model enables a software manager to determine optimal levels of resource allocation to each of the phases of the software development process to maximize the software quality subject to a budget constraint. We first discuss the software development process, software quality factors, and software quality metrics, and then move on to the development of the model. We give an illustrative example for the model developed and demonstrate the sensitivity analysis by determining changes in the optimal objective function values with respect to changes in data. The paper concludes with a summary.

Software quality

Development phases

Software development consists of the following phases: application requirement specifications, software requirement specifications, design and analysis, software implementation, unit testing, integration testing, validation testing and system testing. Each one of these phases includes review, iteration, and documentation.

Software quality factors

A commonly used definition of quality given in ISO 8402 is: "Quality is the totality of features and characteristics of a product or service that bear on its ability to satisfy stated or implied needs". Software quality is defined as "conformance to explicitly stated functional and performance requirements, explicitly documented development standards and implicit characteristics that are expected of all professionally developed software"[3]. Software quality is measurable and varies from system to system and program to program[4]. The measures differ among projects, and criteria vary as a function of the specific characteristics of the project, the needs of the user, and the application requirements of the system and software. Software quality is described by quality factors which are based on the perceived quality concerns of the user and program/project personnel. McCall et al. described the quality factors in[5]. Each of these quality factors is a function of software attributes. These attributes are called metrics. A description of the quality factors and the quality metrics are given in Arthur[4]. It is very difficult to measure these quality factors directly. Metrics are convenient means to quantify measurable aspects of quality. Each quality factor is a function of the metrics which affect that quality factor[4]. Relationship between a quality factor and quality metrics is discussed in Arthur[4]. In this paper, we assume a linear relationship between them and consider quality factor as a convex combination of quality metrics. McCall proposed the grading scale, 0 (low) to 10 (high) to measure a quality metric, but in this paper we adapt a continuous version of this scale, ranging from 0 to 1, which more closely parallels the concept of probability.

Description of the model

In this section we develop optimization models to allocate optimally the effort in man-years to various phases of the software development. The objective functions considered here are to maximize: individual quality factors, average of quality factors, and the minimum of quality factors. While the effort allocated for each phase is bounded below, the total effort is bounded above by s a limited budget constraint. Additional constraints include lower bounds on quality metric levels attained. Putnam[6] utilized the Rayleigh-Norden curve to estimate the software development time as a function of total effort for a given number of delivered lines of code. Using Putnam's model a due date constraint is incorporated into the model.

Notation

The following notation applies:

B budget available;

[C.sub.i] cost of unit effort at phase i;

[K.sub.lj] regression coefficient of jth metric for quality factor l;

L the number of delivered lines of code;

[l.sub.i] lower bound for [X.sub.i];

L[M.sub.j] acceptable lower limit of jth quality metric;

[m.sub.ij] the rate of contribution of effort at ith phase to the complement of jth quality metric;

T the development time (calendar time in years);

[X.sub.i] number of units of effort in man-years allocated for phase i;

[Xi] state of technology constant.

Assumptions

It is assumed that:

* The contribution to the complement of a quality metric attained at phase i is a function of [X.sub.i], the effort spent at that phase. A quality metric is an increasing concave function of [X.sub.i]. This function increases from zero to one asymptotically.

* For a given number of delivered lines of code (source statements), the effort and the development time are related by Rayleigh-Norden curve[6].

* Each quality factor is a convex combination of quality metrics that affect the quality factor. This assumption seems reasonable albeit a slight departure from lineanty assumption suggested by McCall et al.[5].

Development of the model

Now we develop a nonlinear programming model to maximize the quality of software within budget and development time restrictions. For i = 1, ..., M, [X.sub.i], the amount of effort spent on phase i, are the decision variables. Let us assume that there are M phases in a software development process, P quality factors, and N quality metrics. It is desirable to maximize the levels attained for the quality factors. Based on the first assumption let the level of quality metric j attained throughout the development of software be given by:

Q[M.sub.j] = 1 - [product of] (exp(-[m.sub.ij][X.sub.i])) where i=1 to M (1)

for j = 1, ..., N. In the above equation, a quality metric tends to zero as all [X.sub.i] tend to zero and to one as all [X.sub.i] tend to infinity. Smaller values of [m.sub.ij] need larger effort to attain the same quality metric level. Other suitable functions could be hypothesized and studied as well.

For l = 1, ..., P, the level of quality factor l attained throughout the development is given by a linear combination of the quality metrics as follows:

Q[F.sub.l] = [summation of] [K.sub.lj] Q[M.sub.j] where j=1 to N = [summation of] [K.sub.lj] where j=1 to N {1 - [product of] (exp(-[m.sub.ij][X.sub.i])) where i=1 to M}. (2)

Since larger values of Q[F.sub.l] are desirable, we consider maximizing: average of Q[F.sub.b] minimum of Q[F.sub.l], and Q[F.sub.l] for l = 1, ..., P.

The decision variables [X.sub.i] are restricted by the following constraints. Since the total cost cannot exceed the available budget, we have:

[summation of] [C.sub.i] [X.sub.i] [where] i = 1 to M [less than or equal to B. (3)

The minimum acceptable level for quality metric j is assumed to be L[M.sub.j]. The level attained throughout the development for each quality metric has to be greater than or equal to L[M.sub.j], that is for j = 1, ..., N,

1 - [product of] (exp(-[m.sub.ij][X.sub.i])) where i=1 to M [is greater than or equal to] L[M.sub.j]. (4)

Generally, a project has to be completed within a due date. The development time may be estimated using Putnam's formula[6]. For a given number of lines of code, the development time is a function of total effort. This due date constraint can be written as:

[L.sup.3/4]/[[Xi].sup.3/4][([Sigma][X.sub.i]).sup.1/4] [less than or equal to T

which on simplification gives

[Sigma][X.sub.i] [greater than or equal to] [L.sup.3]/[[Xi].sup.3][T.sup.4] (5)

where [Sigma][X.sub.i] is the total effort, T the available calendar time, [Xi] a state of technology constant, and L the lines of code. [Xi] reflects "throughput constraints that impede the progress of the programmer"[3]. Typical values of [Xi] might be 2,000 for a poor software development environment, 4,000 for a good software development environment, and 11,000 for an excellent environment. [Xi] can be derived from local conditions using historical data collected from past development efforts. For a given budget, this constraint checks if it is feasible to s develop the software within the due date.

Finally we specify minimal levels of effort that need to be spent on each phase i. Thus the decision variables have lower bounds [l.sub.i]:

[X.sub.i] [greater than or equal to] [l.sub.i], i = 1, ..., M. (6)

Then the nonlinear programming models considered are given below.

For l = 1, ..., P,

Model (l): Maximize Q[F.sub.l]

subject to the constraints (3)-(6)

For a given l, this model would optimize Q[F.sub.l] without regard to other quality factors.

Model(AVG): Maximize Average of Q[F.sub.l], l = 1, ..., P

subject to the constraints (3)-(6)

In this model each of the quality factors is given equal weight. Where the weights are known, a corresponding weighted average of the quality factors could be considered instead for maximization.

Model(MIN): Maximize Minimum of Q[F.sub.l], l = 1, ..., P

subject to the constraints (3)-(6)

In this model, the optimal objective function value denotes the maximum of the minimum over all the quality factors.

In each of the above models we are maximizing a strictly concave function on a convex set. Hence the local optimal solution is the unique global optimal solution[7].

An illustrative example

In this section we present an illustrative example and discuss its solution. While developing a software system, suppose high reliability, maintainability and testability are specified to be of importance. These quality factors are functions of the following quality metrics: concision, consistency, modularity, instrumentation, self-documentation, simplicity, accuracy, complexity, error tolerance and auditability[4].

For this example the data were generated arbitrarily. The unit of effort is taken as man-year.

Indices:

i index for phases i = 1, ..., 8 j index for quality metrics j = 1, ..., 10 l index for quality factors l = 1, 2, 3

Data:

[Xi] = 4,000

L = 10,000

T = 1.0 year

[C.sub.i] = (30,000; 43,750; 37,500; 38,750; 28,750; 33,750; 26,250; 35,000)

L[M.sub.j]= (0.7, 0.7, 0.7, 0.7, 0.7, 0.7, 0.7, 0.7, 0.7, 0.7)

The values of [m.sub.ij] and [K.sub.lj] are given in Tables I and II respectively.

[TABULAR DATA FOR TABLE I OMITTED]

[TABULAR DATA FOR TABLE II OMITTED]

These models were solved using GAMS 2.04[8]. Several runs of the example are made by varying the budget B in the constraint (3). The results are summarized in Tables III, IV and V.

As the budget increases, the quality factors increased monotonically. As shown in Figure 1, the rate of increase is not significant after a certain level of budget. This observation could assist the manager in the resource allocation decision-making process in a software development environment. Model(l) focuses on maximizing quality factor l. Hence the value of Q[F.sub.l] in Table V would be no less than those in Tables III and IV.

Table III. Optimum values of average of Q[F.sub.l] Budget Q[F.sub.1] Q[F.sub.2] Q[F.sub.3] OBJAVG 1,000,000 0.8883 0.8183 0.8251 0.8349 1,500,000 0.9643 0.9327 0.9372 0.9447 2,000,000 0.9870 0.9746 0.9765 0.9794 2,500,000 0.9951 0.9903 0.9911 0.9922 3,000,000 0.9981 0.9963 0.9966 0.9970 3,500,000 0.9993 0.9986 0.9987 0.9988 4,000,000 0.9997 0.9994 0.9995 0.9995 4,500,000 0.9999 0.9998 0.9998 0.9998 5,000,000 0.9999 0.9999 0.9999 0.9999 Table IV. Optimal values of minimum of Q[F.sub.l] Budget Q[F.sub.1] Q[F.sub.2] Q[F.sub.3] OBJAVG 1,000,000 0.8791 0.8210 0.8243 0.8210 1,500,000 0.9564 0.9346 0.9370 0.9346 2,000,000 0.9841 0.9754 0.9764 0.9754 2,500,000 0.9940 0.9906 0.9911 0.9906 3,000,000 0.9977 0.9964 0.9966 0.9964 3,500,000 0.9991 0.9986 0.9987 0.9986 4,000,000 0.9996 0.9995 0.9995 0.9995 4,500,000 0.9999 0.9998 0.9998 0.9998 5,000,000 0.9999 0.9999 0.9999 0.9999 Table V. Optimal values of Q[F.sub.l] Budget Q[F.sub.1] Q[F.sub.2] Q[F.sub.3] 1,000,000 0.8883 0.8210 0.8251 1,500,000 0.9649 0.9346 0.9379 2,000,000 0.9884 0.9754 0.9768 2,500,000 0.9961 0.9906 0.9912 3,000,000 0.9986 0.9964 0.9967 3,500,000 0.9995 0.9986 0.9987 4,000,000 0.9998 0.9995 0.9995 4,500,000 0.9999 0.9998 0.9999 5,000,000 1 0.9999 0.9999

Sensitivity analysis

The models were developed in the previous sections under deterministic assumptions. This means that we assume complete certainty in the data and the relationships involving the decision variables. In practice these are changing over time and difficult to estimate. One way to handle this apparent discrepancy is by determining how sensitive the solution is to model assumptions and data. We illustrate here the sensitivity analyses by determining the changes in objective function with respect to changes in [X.sub.i] (Table VI) and [m.sub.ij] (Table VII). In Table VI, the optimal solution 0.9677 of Model(AVG) is displayed. The sensitivity of the optimal solution is determined by changing one [X.sub.i] at a time. For instance, in the first row of the table, the value of 0.9675 is obtained by running the Model(AVG) by setting [X.sub.1] at 25 per cent higher than its value in the original optimal solution. Even though the model is, in general, not very sensitive to changes in [X.sub.i], the sensitivity varied widely among [X.sub.i]. For instance, the sensitivity with respect to [X.sub.2] is 24 times as much compared with that of [X.sub.3]. In Table VII, the values of [m.sub.ij] are changed by factor of a [Alpha] and the corresponding optimal values of various models are displayed. As [Alpha] increases, an increase in the optimal objective values is noted though the rate of increase steadily decreases.

Conclusions

In this paper we proposed an optimization approach to offer support in making resource allocation decisions in a software development environment. Software quality factors are viewed as multiple objectives for maximization. Three ways of dealing with these objectives are discussed using nonlinear programming models. An illustrative example is solved using GAMS 2.04[8]. The results indicated a monotonic increase in quality factors with increase in budget. It is found that the marginal increase in quality factors are not significant at higher budget values. The sensitivity analyses are demonstrated by varying [X.sub.i] and [m.sub.ij]. The sensitivity of the optimal objective value of Model(AVG) varied widely over individual [X.sub.i]s. When [m.sub.ij]s are multiplied by a (ranging from 0.5 through 1.5), the optimal objectives of the models increased with a at a steadily decreasing rate of increase.

Table VI. Sensitivity analysis on [X.sub.l]s Variables Change in [X.sub.i] Decrease in changed (percentage) OBJAVG OBJAVG X(1) 25 [up arrow] 0.9675 0.0002 X(2) 25 [up arrow] 0.9653 0.0024 X(3) 25 [up arrow] 0.9676 0.0001 X(4) 25 [up arrow] 0.9676 0.0001 25 [down arrow] 0.9677 0.0000 X(5) 25 [up arrow] Infeasible - 25 [down arrow] 0.9674 0.0003 X(6) 25 [up arrow] 0.9662 0.0015 X(7) 25 [up arrow] 0.9661 0.0016 X(8) 25 [up arrow] 0.9655 0.0022 Note: At optimal solution (2, 2, 2, 3.24, 16.07, 2, 2, 2) the objective function value OBJAVG of Model(AVG) is 0.9677 Table VII. Sensitivity analysis on [m.sub.ij]s [Alpha] OBJAVG OBJMIN OBJ1 OBJ2 OBJ3 0.5 0.8439 0.8210 0.8883 0.8210 0.8251 0.6 0.8880 0.8701 0.9234 0.8701 0.8735 0.7 0.9188 0.9047 0.9466 0.9047 0.9081 0.8 0.9407 0.9297 0.9622 0.9297 0.9328 0.9 0.9563 0.9478 0.9730 0.9478 0.9507 1.0 0.9677 0.9611 0.9805 0.9611 0.9637 1.1 0.9759 0.9709 0.9859 0.9709 0.9731 1.2 0.9820 0.9782 0.9898 0.9782 0.9800 1.3 0.9869 0.9836 0.9925 0.9836 0.9851 1.4 0.9898 0.9876 0.9945 0.9876 0.9888 1.5 0.9923 0.9907 0.9960 0.9907 0.9916

References

1. IEEE Standards Software Engineering Collection, 1987 ed., IEEE Standards Press, Piscataway, NJ.

2. Suresh, N. and Babu, A.J.G., "Software quality optimization: a linear programming model", Association of Management Proceedings, Vol. 10 No. 2, 1992, pp. 168-74.

3. Pressman, R.S., Software Engineering: A Practitioner's Approach, 2nd ed., McGraw-Hill, London, 1987.

4. Arthur, L.A., Measuring Programmer Productivity and Software Quality, Wiley Interscience, Chichester, 1985.

5. McCall, J., Richards, P. and Walters, G., Factors in Software Quality, 3 Vols, NTIS AD-A049-014, 015, 055, November 1977.

6. Putnam, L., "A general empirical solution to the macro software sizing and estimating problem", IEEE Trans. Software Engineering, Vol. 4 No. 4, 1978, pp. 345-61.

7. Bazaraa, M.S. and Shetty, C.M., Nonlinear Programming: Theory and Algorithms, John Wiley & Sons, Chichester, 1979.

8. Brooke, A., Kendrick, D. and Meeraus, A., GAMS: A User's Guide, The Scientific Press, San Francisco and Redwood City, CA, 1988.

Printer friendly Cite/link Email Feedback | |

Author: | Babu, A.J.G.; Suresh, Nalina |
---|---|

Publication: | International Journal of Quality & Reliability Management |

Date: | Mar 1, 1996 |

Words: | 3347 |

Previous Article: | A software reliability growth model. |

Next Article: | Strategic quality management, Malcolm Baldrige and European quality awards and ISO 9000 certification: core concepts and comparative analysis. |

Topics: |