Printer Friendly
The Free Library
22,741,889 articles and books

Service level agreements as vehicles for managing acquisition of software-intensive systems.

Service level agreements (SLAs) can be used as a means to manage the acquisition of software-intensive systems. The SLAs support performance-based acquisition by stating in measurable terms the service to be performed, the level of service that is acceptable, the way in which the service level is to be measured, and the incentives for the provider of information technology products and services to meet the agreed-to target levels of quality. The SLAs are traditionally used in outsourcing contracts for postproduction post·pro·duc·tion  
A final stage in the production of a film or a television program, occurring after the action has been filmed or videotaped and typically involving editing and the addition of soundtracks.
 support. This article proposes a new approach by using SLAs in software acquisition to support quality and process control throughout the entire lifecycle of a software-intensive system. This article defines SLAs, discusses software quality, and describes how SLAs can be utilized to incorporate requirements pertaining per·tain  
intr.v. per·tained, per·tain·ing, per·tains
1. To have reference; relate: evidence that pertains to the accident.

 to product, process, project, and deployment quality throughout the software lifecycle.


As advances in information technology (IT) increase worker productivity and encourage the adoption of new ways of conducting business, organizations and end users become ever increasingly dependent upon that technology. Consequently, the strategic and tactical advantages that IT affords an organization places pressure on the organization's IT department to provide quality services and products. Interruptions to software-intensive systems are having a far greater impact than before in terms of opportunity loss, revenue loss, customer dissatisfaction, and inefficiency. As managers realize that their mission-critical processes are tied to IT services and products, they are demanding correspondingly greater levels of both performance and quality in these services and products, despite the fact that acquiring and maintaining software-intensive systems is increasingly challenging from both a managerial and technical perspective.

In addition, an organization may not have enough personnel with the right mix of IT skills, knowledge, and experience within their organization to provide high-enough quality IT products and services for the organization's employees and business partners. Rather than hire IT specialists or invest in training for its internal staff, the organization may have the option to outsource to external service providers (ESPs) their IT services and products. This strategy has been followed by the U.S. Department of Defense (DoD) in procuring large software-intensive systems such as the well-known Navy/Marine Corps Intranet (NMCI NMCI Navy/Marine Corps Intranet
NMCI National Multi-Cultural Institute
) and Ballistic Missile Defense System Noun 1. missile defense system - naval weaponry providing a defense system
missile defence system

naval weaponry - weaponry for warships
 (BMDS BMDS Ballistic Missile Defense System
BMDS Base Manpower Data System
). The hope is that ESPs can readily supply different mixes of IT specialists to meet the ever-changing requirements an organization has for IT products and services, at a lower cost than enlarging, shrinking, or retraining re·train  
tr. & intr.v. re·trained, re·train·ing, re·trains
To train or undergo training again.

 its internal IT staff. In addition to providing access to cutting-edge technology and skilled staff, ESPs share the project risk and make it possible for organizations to concentrate on core competencies A core competency is something that a firm can do well and that meets the following three conditions specified by Hamel and Prahalad (1990):
  1. It provides customer benefits
  2. It is hard for competitors to imitate
  3. It can be leveraged widely to many products and markets.
 (King, 2001).

Reliance on outsourcing adds another layer of complexity onto the management of information systems. Outsourcing efforts require additional discipline and management oversight that may not be necessary with in-house development or maintenance of information systems. Outsourcing requires skill in software acquisition as well as project management. Program managers need to be accomplished in software-requirements engineering, software development, the contracting process, requirement-change management, contract management and oversight, and contractor-relationship management.

When outsourcing software-intensive programs, the program manager must be explicit in stating the services to be performed, in addition to identifying the metrics and means to determine whether the contractor has satisfied those requirements. However, it is challenging to develop, manage, and enforce a contract for software services that will ensure that the contractor delivers the specified services or end product within prescribed levels of performance and quality.

Managing software-intensive information systems can also be challenging. Utilizing the latest technology to exploit information requires highly developed intellectual and managerial skills. The difficulty in managing these systems has been demonstrated by the numerous system development and maintenance projects within the U.S. Department of Defense. Many of these projects lacked sound planning, had inadequate controls, were without effective measurements for success, and failed to meet user expectations (U.S. General Accounting Office [GAO], 2001; Department of Defense, Office of the Inspector General Office of the Inspector General (or OIG) is a common sub-agency within cabinet-level agencies of the United States federal government and serves as auditing and investigative arm of the agency's programs focused on identifying waste, fraud and abuse.  [DoD OIG Noun 1. OIG - the investigative arm of the Federal Trade Commission
Office of Inspector General

independent agency - an agency of the United States government that is created by an act of Congress and is independent of the executive departments
], 2000). However, the private sector also has problems managing IT projects. The 2003 Standish Group's CHAOS research report indicates that of the 13,522 IT projects studied; only 34 percent of projects were considered a success, and only 52 percent of required features were incorporated into the released product (Standish Group, 2003).

Despite software's increased importance to organizations, the quality of software is often lacking. Some would argue, such as Mann (2002), that despite the advances in software engineering theory, processes, methodology, and tools, software quality is actually getting worse. Mann's reasons center around four main points: 1) developers are rushing software to market, 2) software is poorly designed, 3) developers and testers lack the requisite skills, and 4) programs are not managed well. Given the difficulties in developing software-intensive systems, performance and quality requirements should be well defined and monitored throughout the software's lifecycle. In this paper we explain how service level agreements (SLAs) can be utilized in software acquisition to improve the quality and management of software throughout its lifecycle.

SLAs are a means of incorporating Performance-based Service Contracting (PBSC PBSC Peripheral Blood Stem Cell
PBSC Performance-Based Service Contracting
PBSC Pro Bono Students Canada
PBSC Polar Bear Software Company
PBSC Public Buildings and Site Commission
) into software acquisition. SLAs are similar to a Quality Assurance Plan (QAP QAP Quality Assurance Program
QAP Quadratic Assignment Problem
QAP Quality Assurance Plan
QAP Quality Assurance Procedure
QAP Quality Assurance Personnel
QAP Quality Assurance Provision
QAP QoS Access Point
QAP Queue Adjustment Policy
) in that they define quality and performance requirements, they contain penalties for non-performance, and they establish quality control methods to ensure compliance.


In this article we identify areas throughout the lifecycle (requirements generation through post-production support) of software-intensive systems in which SLAs can be used to manage the contractual provisions of insourced or outsourced IT services and products. In this section we define terms associated with SLAs and quality, in addition to describing how SLAs can be utilized in requirements engineering (programming) Requirements Engineering - The task of capturing, structuring, and accurately representing the user's requirements so that they can be correctly embodied in systems which meet those requirements (i.e. are of good quality).

DOORS is one product to help with this task.
, design, post-production support, and program management to achieve target levels of performance and quality for IT services and products.

An SLA (1) (StereoLithography Apparatus) See 3D printing.

(2) (Service Level Agreement) A contract between the provider and the user that specifies the level of service expected during its term.
 is a contractual mechanism between a provider of services and a customer that defines a level of performance (Sturm, Morris, & Jander, 2000). This agreement defines in measurable terms the service to be performed, the level of service that is acceptable, the means to detetmine if the service is being provided at the agreed upon Adj. 1. agreed upon - constituted or contracted by stipulation or agreement; "stipulatory obligations"

noncontroversial, uncontroversial - not likely to arouse controversy
 levels, and incentives for meeting agreed upon service levels. SLAs contain quality-of-service (QoS) requirements, including how QoS is to be measured. SLAs also set forth the roles and responsibilities--in contractual form--of both the organization and the provider of the services and products.

SLAs are not a new concept: they have been around since the 1960s. However, ever-growing reliance on software-intensive systems for conducting business and recent advances in both techniques and tools for specifying SLAs and monitoring compliance to SLAs have all contributed to the rise in the acceptance of SLAs within the public and private sectors. If SLAs are not included in a system-acquisition contract, the acquiring organization has little leverage over the provider of the IT services or products to ensure that the organization's performance and QoS requirements are met.

There is a subtle distinction between an SLA and a traditional contract requirement. SLAs are also requirements, but they are different contractually: SLAs contain more detail, describe incentives for meeting thresholds, and specify incentives for meeting performance and QoS requirements. When SLAs are used, the contracting officer A US military officer or civilian employee who has a valid appointment as a contracting officer under the provisions of the Federal Acquisition Regulation. The individual has the authority to enter into and administer contracts and determinations as well as findings about such contracts.  can withhold incentive payments or levy penalties if quality levels are not met during the time period stated in the SLA (usually a month or a quarter). Requirements are generally only measured at the end of a milestone decision. In most contracts, the only recourse if a requirement is not met is to cancel the contract, terminate any ongoing contractor support, or give the contractor a poor performance rating. Terminating a project can be difficult, especially if the software-intensive system to be acquired is mission-critical. SLAs specify the degree of recourse if a requirement is not met.

One of the ways that SLAs can help improve software quality is that they focus the attention of the acquisition professional on the nonfunctional requirements (e.g., reliability, security, testability) that otherwise might be ignored until the test phase in the absence of SLAs (Bass, Clements, & Kazman, 1998). SLAs contractually mandate the performance and quality attributes that users, program managers, and software engineers consider essential for a system to support the underlying business process. They help make explicit many of the quality factors that users may implicitly assume. SLAs also specify the quality metrics by which the software quality requirements are measured. Measuring and monitoring performance and quality of services and products makes it possible for an organization to judge whether performance and quality requirements have been met. Measurements also support early detection and resolution of problems with quality.

Before discussing SLAs and how they can improve the performance and quality in the various phases of a software-intensive system's lifecycle, we first define the term software quality. There are numerous definitions of quality. The International Standards Organization See ISO.  (ISO (1) See ISO speed.

(2) (International Organization for Standardization, Geneva, Switzerland, An organization that sets international standards, founded in 1946. The U.S. member body is ANSI.
) standard 9126 defines software quality as the totality of features and characteristics of a software product that bear on its ability to satisfy stated or implied needs. Another definition is that software quality is conformance to explicitly stated functional and performance requirements, explicitly documented development standards, and implicit characteristics that are expected of all professionally developed software (Pressman, 2001). Others believe that an IT system will not be perceived as a quality product if the product does not perform according to according to
1. As stated or indicated by; on the authority of: according to historians.

2. In keeping with: according to instructions.

 the end-user's perspective (Wiegers, 2002). The Institute of Electrical and Electronics Engineers Not to be confused with the Institution of Electrical Engineers (IEE).

The Institute of Electrical and Electronics Engineers or IEEE (pronounced as eye-triple-e
 (IEEE (Institute of Electrical and Electronics Engineers, New York, A membership organization that includes engineers, scientists and students in electronics and allied fields. ) standard 610-1990 does incorporate user needs: it defines software quality as the degree to which a system, component, or process meets specified requirements and meets customer or user needs and expectations (Schmidt, 2000).

In this article we adopt the IEEE definition and specialize it to encompass four specific areas of focus: 1) product quality, 2) project quality, 3) process quality, and 4) post-production quality. Each of the areas has quality attributes associated with it that describe the degree to which the software possesses certain characteristics. SLAs utilize quality metrics and threshold levels Noun 1. threshold level - the intensity level that is just barely perceptible
intensity, intensity level, strength - the amount of energy transmitted (as by acoustic or electromagnetic radiation); "he adjusted the intensity of the sound"; "they measured the
 to assign quantitative measurements to the quality attributes. The quality metrics are then monitored to ensure compliance.

Quality can be viewed from numerous perspectives, and certain attributes are more preferable to others depending on the mission the software-intensive system is supposed to support. Numerous quality attributes and quality models have been developed to measure or predict software quality. Selecting the appropriate quality attributes for the project is the responsibility of the SLA development team, the program manager, and stakeholders Stakeholders

All parties that have an interest, financial or otherwise, in a firm-stockholders, creditors, bondholders, employees, customers, management, the community, and the government.


Product quality is concerned with the relationships between the characteristics and attributes of a product and its ability to satisfy stated requirements and implicit needs; this area can also be referred to as end-product quality. In addition to the functional aspects of a system, the end users want the product to exhibit certain quality characteristics that will assist them in performing their task. From a user's perspective, some of the common quality attributes include availability, usability, integrity, interoperability, and reliability. Personnel involved in the development of software or its maintenance may be more concerned with the software attributes such as portability, testability, maintainability, and reusability (Wiegers, 2002).

There are numerous quality models that can be incorporated into SLAs that assign quantitative measurements to various product quality attributes. Some of the better-known quality models include those proposed by McCabe in which software complexity is a function of the number of conditional statements in the code, and those of Halstead in which software complexity is a function of the number of operators and operands (Ogasawara, Yamada, & Kojo, 1996). The ISO standard 9126-1 quality model incorporates the following quality factors: functionality, reliability, usability, efficiency, maintainability, and portability (Cross, 2002). There are also numerous software quality models that concentrate on specific quality factors such as complexity. Zuse (1991) identifies over 90 models for describing software complexity. Other quality models are specific to object-oriented systems (Coppick & Cheatham, 1992), specific languages (Pritchett, 2001), commercial off-the-shelf Commercial off-the-shelf (COTS) is a term for software or hardware, generally technology or computer products, that are ready-made and available for sale, lease, or license to the general public.  (COTS) components (Bertoa & Vallecillo, 2002), and others, are only applicable at run-time (Bass et al., 1998).


Project quality is concerned with metrics that allow an organization to manage, track, and improve the quality of the software development effort; this area could be viewed as in-process quality management (Hilburn & Townhidnejad, 2000).

Some of the common project quality metrics that Motorola used to measure its software development projects included software defect density (programming) defect density - The ratio of the number of defects to program length. , adherence to schedule, estimation accuracy, reliability, requirements tracking, and fault-type tracking (Daskalantonakis, 1992). Another metric used in assessing project quality is risk, which can be defined as any variable within a project that results in project failure. General risk areas are schedule risk, requirements risk, budget risk, and personnel risk (Padayachee, 2002). There are a number of risk-assessment models including Gilb's (1993) risk heuristics heu·ris·tic  
1. Of or relating to a usually speculative formulation serving as a guide in the investigation or solution of a problem:
, Boehm's (1991) classification of risk, Noguiera de Leon's (2000) risk assessment model, Keil's identification of risk factors (Keil, Cule, Lyytinen, & Schmidt, 1998), and risk models associated with enterprise software projects (Sumner, 2000).


Process quality is concerned with the processes, planning, and controls used to develop and manage the software product. It could be argued that the quality of the development process is the best predictor of software product quality (Fenton, Krause, & Neil, 2002). Repeatable software processes such as the Software Engineering Institute's Capability Maturity Model for software (SW-CMM SW-CMM Software Capability Maturity Model (Software Engineering Institute)  also known as CMM (Capability Maturity Model) A process developed by SEI in 1986 to help improve, over time, the application of an organization's supporting software technologies. ), which lists five levels of organizational maturity, and the ISO 9001, are designed to improve software quality, productivity, predictability, and time to market (McGuire, 1996). There is empirical evidence that supports the relationship between process maturity and software quality (Harter & Slaughter, 2000).

Other models of process quality include the Software Engineering Institute's new Capability Maturity Model Integration (CMMI See CMM. ). CMMI integrates three CMM models into one to eliminate problems with different architecture, semantics, and approaches. Humphery (1996) also developed a process model called the Personal Software Process (PSP (PlayStation Portable) See PlayStation. ) to assist software engineers in producing quality software. Other process models include cleanroom engineering that has shown reduced errors per KLOC (Kilo Lines Of Code) One thousand lines of programming source code. See lines of code.

(unit, programming) KLOC - Thousand (kilo-) Lines of code.
 for small projects (Fenton & Neil, 1999), and the quality management metric (QMM QMM Quick March Medley
QMM Quasi-Multiple Medium Method
QMM Quadratic Minimax
QMM Quadrature Multipath Monitor
QMM Question Mark and the Mysterians (band)
QMM Quest for the Magic Marbles
) (Osmundson, Michael, Machniak, & Grossman, 2003). There are also numerous IEEE and ISO standards This is a list of ISO standards that are discussed in Wikipedia articles. For a list of all the more than 16,000 ISO standards (as of 2007), see the ISO Catalogue.

About 300 of the standards produced by ISO and IEC's Joint Technical Committee 1 (JTC1) have been made freely/publicly
 that provide processes on everything from software engineering product evaluation (e.g., ISO/IEC ISO/IEC International Organization for Standardization/International Electrotechnical Commission (ITU-T M 3000)  14598) to selecting appropriate quality metrics (e.g., IEEE Std. 1061-1998).


The last area of focus is on post-production quality or deployed-application management. Many of the quality models involving deployed applications are concerned with software maintenance and the quality factors that make maintenance cost-effective. Some of the maintenance quality factors deal with ease of change (Royce, 1990), architectural design This article or section may contain original research or unverified claims.

Please help Wikipedia by adding references. See the for details.
This article has been tagged since September 2007.
 to promote maintenance (Garlan, 2000), defect management (1) The elimination of bugs in software and flaws in hardware. Defect management is part of a software or hardware development project.

(2) The prevention of data errors in a storage medium by invalidating bad sectors.
 (Kajko-Mattsson, 1998), organizational structure This article has no lead section.

To comply with Wikipedia's lead section guidelines, one should be written.
 (Briand, Melo, Seaman SEAMAN. A sailor; a mariner; one whose business is navigation. 2 Boulay Paty, Dr. Com. 232; Code de Commerce art. 262; Laws of Oleron, art. 7; Laws of Wishuy, art. 19. The term seamen, in it most enlarged sense, includes the captain a well as other persons of the crew; in a more confined , & Basili, 1995), and change management (Bennett & Rajlich, 2000).

Quality does not only apply to the application itself, it is also concerned with the IT system as a whole, across distributed components. Part of that distributed system See distributed computing.

distributed system - A collection of (probably heterogeneous) automata whose distribution is transparent to the user so that the system appears as one local machine.
 is the network. There are numerous quality metrics that can be applied to network QoS (Moeller et al., 2000). Quality metrics are also applied to the host server. Quality metrics such as application resource utilization (Aries, Banerjee, Brittan, Dillon, Kowalik, & Lixvar, 2002), bandwidth utilization (Eager, Vernon, & Zahorjan, 2001), concurrent user In computer science, the number of concurrent users for a resource in a location, with the location being a computing network or a single computer, refers to the total number of people using the resource at the same time.  management (Aweya, Ouellette, Montuno, Doray, & Felske, 2002), and server performance (Gama, Meira, Carvalho, Guedes, & Almeida, 2001) are also utilized to address system-level quality. Other areas where quality metrics are applied include total cost of operation (TCO (1) (Total Cost of Ownership) The cost of using a computer. It includes the cost of the hardware, software and upgrades as well as the cost of the inhouse staff and/or consultants that provide training and technical support. See ROI. ), help desk support metrics, backups, storage, configuration management, and security.

There are numerous software quality models and metrics that can be incorporated into SLAs. The models or quality factors chosen will depend upon those quality attributes that best support the underlying business processes of the organization acquiring the software-intensive system. SLAs should only incorporate software metrics Software measurements. Using numerical ratings to measure the complexity and reliability of source code, the length and quality of the development process and the performance of the application when completed.  that are meaningful, quantitative, and measurable.

When developing the SLAs, it is essential that all stakeholders be represented in this effort, as the system requirements To be used efficiently, all computer software needs certain hardware components or other software resources to be present on a computer system. These pre-requisites are known as (computer) system requirements and are often used as a guideline as opposed to an absolute rule.  need to represent all of their needs for quality. The development team, consisting of representation for all stakeholders, must resolve a number of issues. These include, for instance, identifying quality requirements, determining the various levels of service that are needed, detailing quality metrics that are meaningful and measurable, assessing risk, resolving conflicting quality requirements, and prioritizing the quality requirements. When quotations are received from vendors (i.e., the providers of IT services and products), the group also needs to perform a cost-benefit analysis cost-benefit analysis

In governmental planning and budgeting, the attempt to measure the social benefits of a proposed project in monetary terms and compare them with its costs.
 based on the levels of quality thresholds desired and the budget allotted al·lot  
tr.v. al·lot·ted, al·lot·ting, al·lots
1. To parcel out; distribute or apportion: allotting land to homesteaders; allot blame.

 for the acquisition. The group development of SLAs helps the various stakeholders understand each others' biases, viewpoints, concerns, terminology, and perceptions--that understanding is essential in requirements engineering.


Requirements engineering provides the building blocks for all other efforts in the software-engineering process, so if quality is not addressed in the initial requirements analysis (project) requirements analysis - The process of reviewing a business's processes to determine the business needs and functional requirements that a system must meet. , it is usually addressed at the end of the project in the form of testing. If quality requirements are implemented too late, the architecture that was already developed will dictate the solution space for addressing quality problems that are discovered, or the acquisition authority will need to approve major contract modifications to permit changes to the requirements, architecture, and other high-level system artifacts artifacts

see specimen artifacts.

The process of developing the SLAs is part of the larger requirement-engineering process and fosters discussion of the following: the goals of the system that must be met, the processes and tasks that the system must perform to meet those goals, and any operational and organizational needs, policies, standards, and constraints. Discussions necessary to develop the SLAs will generate information about the application domain, business and organization processes/culture, and the intended operating environment In computing, an operating environment is the environment in which users run programs, whether in a command line interface, such as in MS-DOS or the Unix shell, or in a graphical user interface, such as in the Macintosh operating system.  that the system will be placed in. The discussions will help the requirements engineer capture tacit knowledge The concept of tacit knowing comes from scientist and philosopher Michael Polanyi. It is important to understand that he wrote about a process (hence tacit knowing) and not a form of . , identify constraints, prioritize requirements, and justify how quality factors support business needs. Additionally, SLAs require quantifiable quality metrics, so those quality requirements that cannot be measured, do not provide value, or are not realistic should not be accepted by the SLA development team or the requirements engineer.

In many organizations, senior management's involvement in requirements engineering can be low (Bubenko, 1995), resulting in requirements that may not be related to business visions and objectives. SLAs help mitigate this problem because they define specific performance parameters that are tied to business processes. As such, every SLA performance requirement must be analyzed, and validated to ensure that it is meaningful, cost-effective, and that they add to improving overall performance. Senior management may also take more notice when there are contractual repercussions repercussions nplrépercussions fpl

repercussions nplAuswirkungen pl 
 (i.e., penalties or rewards) associated with the requirements.

By defining meaningful and measurable metrics in the SLAs, the end-users, business managers and software engineers have realistic quantifiable requirements that can be used to develop architectures, designs, and other software artifacts.


The real contribution of SLAs in obtaining quality software is that the quality factors addressed in the SLAs drive significant architectural and design decisions. If developers know which of the characteristics are most critical to project success, they can select the architecture, design, programming, and verification and validation Verification and Validation (V&V) is the process of checking that a product, service, or system meets specifications and that it fulfills its intended purpose. These are critical components of a quality management system such as ISO 9000.  approaches that best achieve the specified quality goals. When customer requirements have been collected and specified, architecting and designing translates the requirements into a blueprint that programmers can use to build the product. That design can be evaluated and monitored to ensure quality factors are adequately addressed. In this way, quality is designed in the beginning phases of the lifecycle where it is most cost-effective to do so.

The architecture of a software system models its structure and behavior (Shaw & Garlan, 1996). Architecture also shows the correspondence between the system requirements and the elements of the constructed system; so by analyzing the architecture it is possible to predict the quality of a product before it has been built. There are a number of methods to analyze architectures (Bass et al., 1998). However, these methods do not produce quantifiable quality measurements (Dobrica & Niemela, 2002), although they can provide an estimate of how well the design will satisfy a particular quality factor.

SLAs also improve software quality in the development phase by contractually mandating that certain quality-control measures (e.g., adhering to specified standards and processes) be performed. There are numerous industry-approved standards that can be incorporated into SLAs. Incorporating standards in SLAs provides a common methodology that makes management easier for program managers and contracting officers as they provide the basis against which activities can be measured and evaluated (Horch, 1996). Standards can be applied to all aspects of development, maintenance, and operation of an information system.

Development processes can be specified in an SLA. Specifying specific processes has many of the same advantages of specifying compliance to standards. Applying well defined, standardized software-development processes can increase software quality and make the development effort more cost effective and predictable (Gnatz, Marschall, Popp, Rausch, & Schwerin, 2003). Specifying the processes in the SLA helps to ensure that they are recognized and adhered to.

SLAs also assist the testing effort by identifying business-critical processes, defining quantitative metrics to measure quality factors, identifying testing procedures, and ensuring testing is conducted throughout the system's lifecycle. SLAs can ensure that other audits such as documentation reviews, requirements reviews, design reviews, test plan reviews, user documentation reviews, and implementation reviews are conducted (Horch, 1996).


Program managers have to ensure that quality considerations are addressed early in the lifecycle and they must provide the proper amount of oversight to ensure those quality factors are incorporated into the final product. SLAs can assist program managers in many of the tasks necessary to ensure quality is delivered in the final and future versions of a product.

In order to reduce and manage risk, program managers need to measure or monitor contractor, project, and system performance throughout the project's lifecycle. This will help ensure that standards, processes, and quality requirements are being met. SLAs mandate monitoring of the quality requirements associated with product, process, project, and post-production quality. If quality levels are not met, program managers and the contractor are informed of the violation and potential risks; that knowledge may lead to closer monitoring or corrective action A corrective action is a change implemented to address a weakness identified in a management system. Normally corrective actions are instigated in response to a customer complaint, abnormal levels if internal nonconformity, nonconformities identified during an internal audit or  to reduce risk and improve quality.

The development of SLAs provides information that will assist the program manager in managing the project's finances. SLAs help financial management in determining the scope of the project (e.g., determining areas of responsibilities in an end-to-end performance SLA), identifying business-critical processes and functions, they help to allocate costs among business units, they provide justification for service-related expenditures, and they help coordinate the IT strategy with business strategies (e.g., applying funds toward services to support critical processes).

Quality control consists of the actions necessary to certify that desired standards, processes, and quality requirements are adhered to during design, implementation, and production (Tricker & Sherring-Lucas, 2001). SLAs are a quality-control mechanism. SLAs help the program manager by contractually mandating that certain quality-control methods be implemented.

Contract oversight is easier when SLAs are established because they can be used to define the levels of service expected, explain how the measurement will be conducted, identify responsibilities, and set forth escalation procedures. The monitoring mandated by SLAs helps the program manager determine whether the contractor is meeting the quality requirements. When determining what quality metrics are included in the SLAs, it is helpful to determine the behavior you want from the contractor, and determine what measurements will most likely encourage that behavior (Kendrick, 2003).

SLAs help institutionalize in·sti·tu·tion·a·lize
To place a person in the care of an institution, especially one providing care for the disabled or mentally ill.

 a change review board to continually review the SLAs, evaluate new requirements, and ensure maintenance actions do not affect the SLAs. The change review board ensures that only authorized changes are enacted, that changes are tested against quality levels, that changes conform to Verb 1. conform to - satisfy a condition or restriction; "Does this paper meet the requirements for the degree?"
fit, meet

coordinate - be co-ordinated; "These activities coordinate well"
 architectural constraints, and those changes are properly documented. SLAs can also be written to specify quality requirements that deal specifically with the accuracy or effectiveness of configuration identification, configuration control, configuration accounting, and configuration audits.

In addition, SLAs provide a basis for common understanding of the services that will be performed, the levels of service to be expected, how they will be measured, as well as define the responsibilities of both parties to an SLA. Both parties must mutually agree upon contractual SLAs, or there will never be a contract. It is common-place to negotiate on the services and the performance levels that are requested and ultimately agreed upon. An SLA should contain a definition of service requirement that is both achievable by the provider and affordable by the customer. The customer and the ESP (1) (Enhanced Service Provider) An organization that adds value to basic telephone service by offering such features as call-forwarding, call-detailing and protocol conversion.  must also define a mutually acceptable set of indicators of the quality of service (Sturm et al., 2000). Note that SLAs can and should be modified throughout the lifecycle of a system as requirements change, technology improves, and efficiencies are gained.

The use of SLAs helps to ensure that customer expectations are being met by establishing performance and quality objectives, and providing the metrics and a measurement methodology to ensure that those requirements are being met. An important part of customer service is monitoring the performance of the system to ensure that it is supporting the critical business processes in a manner acceptable to the customer. The fact that SLAs specify what is acceptable makes it easier for program managers to manage expectations.

SLAs can be valuable to organizations that lack the proper IT skills to contract with ESPs. Template SLAs, or pre-existing SLAs, which represent best business practices, can be used as a starting point Noun 1. starting point - earliest limiting point
terminus a quo

commencement, get-go, offset, outset, showtime, starting time, beginning, start, kickoff, first - the time at which something is supposed to begin; "they got an early start"; "she knew from the
 in formulating SLAs that are unique to specific applications. SLA templates are a starting point for program managers to develop their own SLAs. The templates are helpful in generating questions regarding services and service levels that should be provided to support or develop applications. The program managers only have to modify the existing SLAs to best support their application.


Quality control does not stop once a software product has been deployed. Quality requirements still need to be applied to the application performance, maintenance efforts, and hosting services throughout its lifecycle. Monitoring the performance of the application once it is deployed is essential in quality control and maintaining customer satisfaction. Much of the application performance monitoring in the initial phases of deployment is to validate product-quality requirements identified in the initial requirements. However, in the deployment environment there is also an emphasis on monitoring system performance in terms of resource utilization, application security, application reliability, application maintainability, database performance, and server performance.

Quality requirements associated with deployed software take a holistic view of the entire system, including distributed components. Quality requirements should be applied to network performance, host environment security, disaster recovery, storage solutions, problem management, concurrent user management, as well as end-to-end performance. Monitoring of the network, hardware, operating system operating system (OS)

Software that controls the operation of a computer, directs the input and output of data, keeps track of files, and controls the processing of computer programs.
, and the application not only assists in problem resolution, but trend analysis can indicate potential problems before they become unwieldy.

SLAs can be utilized for maintenance actions when patches or new versions need to be developed and deployed. SLAs utilized in the development phase may be used in some circumstances with maintenance actions.


SLAs are performance-based instruments for acquiring software. SLAs can be used to specify software quality, but they are also useful for specifying performance requirements. We have described how SLAs can drive product, process, project, and deployment quality solutions. SLAs can help ensure that quality requirements are identified and established early in the development cycle in order to be incorporated into preliminary designs. SLAs can help program managers establish quality controls to monitor and manage the various aspects of the projects. SLAs also carry sufficient weight through incentives to focus management and contractor attention on the quality issues for a software-intensive system that affect the ability of the acquiring organization to perform its mission.

Future research that can build upon the foundations set forth in this paper include evaluating SLAs in actual contracting to analyze quality improvements, determining which quality models and quality attributes are best suited for different types of IT systems, and evaluating or generating tools necessary for measuring end-to-end SLA measurements, such as response time and availability.

All organizations want world-class quality levels, but achieving those quality levels requires a holistic view of quality that incorporates leadership support, repeatable and measurable quality processes and controls, resource planning Resource planning may refer to:
  • Enterprise resource planning (ERP)
  • Manufacturing resource planning (MRP and MRPII)
  • Distribution Resource Planning (DRP)
  • Human resources (HR)
, vision, customer support, and service-level management. Organizations must do more than identify and incorporate quality attributes in their requirements: they must also monitor quality metrics to ensure those quality requirements are being met. SLAs are instruments that can be used to establish quality controls. Quality is not something that is inherent in the development process; it must be planned, monitored and incorporated as part of standard business practices.


Aries, J., Banerjee, S., Brittan, M., Dillon, E., Kowalik, J., & Lixvar, J. (2002, June). Capacity and performance analysis of distributed enterprise systems. Communications of the Association of Computing Machinery (ACM (Association for Computing Machinery, New York, A membership organization founded in 1947 dedicated to advancing the arts and sciences of information processing. In addition to awards and publications, ACM also maintains special interest groups (SIGs) in the computer field. ).

Aweya, J., Ouellette, M., Montuno, D., Doray, B., & Felske, K. (2002, January/ February). An adaptive load balancing The fine tuning of a computer system, network or disk subsystem in order to more evenly distribute the data and/or processing across available resources. For example, in clustering, load balancing might distribute the incoming transactions evenly to all servers, or it might redirect them  scheme for web servers. International Journal of Network Management, 12, 3-39.

Bass, L., Clements, P., & Kazman, R. (1998). Software architecture in practice. Reading, WA: Addison-Wesley.

Bennett, K., & Rajlich, V. (2000, June). Software maintenance and evolution: A roadmap. Proceedings of the Conference on the Future of Software Engineering, Limerick Limerick, city, Republic of Ireland
Limerick, city (1991 pop. 56,083), seat of Co. Limerick, SW Republic of Ireland, at the head of the Shannon estuary. The city has a port with two docks.
, Ireland, 73-87.

Bertoa, M., & Vallecillo, A. (2002, November). Quality attributes for COTS components. Retrieved October 12, 2004, from

Boehm, B. (1991, January). Software risk management: Principles and practice. IEEE Software IEEE Software is an IEEE Computer Society practitioner-oriented magazine targeting software engineers and managers. It contains peer-reviewed articles, regular columns, and interviews. , 91, 32-41.

Briand, L., Melo, W., Seaman, C., & Basili, V. (1995, April). Characterizing and assessing a large-scale maintenance organization. Proceedings of the Seventeenth International Conference on Software Engineering The International Conference on Software Engineering, or (ICSE), is one of the largest annual Software Engineering conferences. The first ICSE conference was in 1978 in Atlanta, Georgia. , Seattle, WA, 133-143.

Bubenko, J. (1995, March). Challenges in requirements engineering. Second IEEE Symposium on Requirements Engineering, York, England, 160-162.

Coppick, C., & Cheatham, T. (1992, March). Software metrics for object-oriented systems. Proceedings of the 1992 ACM Annual Conference on Communications, Kansas City Kansas City, two adjacent cities of the same name, one (1990 pop. 149,767), seat of Wyandotte co., NE Kansas (inc. 1859), the other (1990 pop. 435,146), Clay, Jackson, and Platte counties, NW Mo. (inc. 1850). , MI, 317-322.

Cross, S. (2002, December). A quality doctrine for software: Do it right the first time. Proceedings of the ninth Asia-Pacific Software Engineering Conference, Gold Coast, Australia, 187-194.

Daskalantonakis, M. (1992, November). A practical view of software measurement and implementation experiences within Motorola. IEEE Transactions on Software Engineering The IEEE Transactions on Software Engineering (TSE) is a monthly journal published by the IEEE Computer Society. It contains peer-reviewed articles and other contribitions in the area of software engineering by computer scientists, covering theoretical results and empirical studies. , 18, 998-1010.

Department of Defense, Office of the Inspector General. (2000, July 13). Summary of audits of acquisition of information technology. (DoD OIG Report D-2000-162). Washington, DC: U.S. Government Printing Office.

Dobrica, L., & Niemela, E. (2002, July). A Survey on software architecture analysis measures. IEEE Transactions on Software Engineering, 28(7), 638-653.

Eager, D., Vernon, M., & Zahorjan, J. (2001, September/October). Minimizing bandwidth requirements Bandwidth requirements (communications)

The channel bandwidths needed to transmit various types of signals, using various processing schemes. Every signal observed in practice can be expressed as a sum (discrete or over a frequency continuum) of sinusoidal
 for on-demand data delivery. IEEE Transactions on Knowledge and Data Engineering, 15(5), 742-757.

Fenton, N., Krause, P., & Neil, M. (2002, July/August). Software measurement: Uncertainty and causal modeling. IEEE Software, 116-122.

Fenton, N., & Neil, M. (1999, September/October). A critique of software defect prediction models This article outlines the various propagation models currently used by the wireless industry for signal transmission at both 900 MHz and 1800 MHz. We start with the foundation of free-space transmission, followed by Picquenard’s multiple knife edge diffraction model. . IEEE Transactions on Software Engineering, 25(5), 675-689.

Gama, G., Meira, W., Carvalho, M., Guedes, D., & Almeida, V. (2001, November). Resource placement in distributed e-commerce servers. Proceedings of Globecom 2001, San Antonio San Antonio (săn ăntō`nēō, əntōn`), city (1990 pop. 935,933), seat of Bexar co., S central Tex., at the source of the San Antonio River; inc. 1837. , TX, 1677-1682.

Garlan, D. (2000, June). Software architecture: A roadmap. Proceedings of the Conference on the Future of Software Engineering, Limerick, Ireland, 93-101.

Gilb, T. (1993). Risk management: A practical toolkit for identifying analyzing and coping with project risk. Retrieved October 12, 2004, from

Gnatz, M., Marschall, F., Popp, G., Rausch, A., & Schwerin, W. (2003, June). The living software development process. Software Quality Professional, 4-16.

Harter, D., & Slaughter, S., (2000). Process maturity and software quality: A field study. Proceedings of the Twenty-first International Conference on Information Systems International Conference on Information Systems (ICIS) is an annual international conference for academics and research-oriented practitioners in the area of Information Systems. , Brisbane, Australia, 407-411.

Hilburn, T., & Townhidnejad, M. (2000, March). Software quality: A curriculum postscript. Proceedings of the Thirty-First SIGCSE SIGCSE Special Interest Group on Computer Science Education  Technical Symposium on Computer Science Education, Austin, TX, 167-171.

Horch, J. (1996). Practical guide to software quality management. Boston, MA: Artech House Publishers.

Humphrey, W. (1996, May). Using a defined and measured personal software process. IEEE Software, 77-88.

Kajko-Mattsson, M. (1998, April). A conceptual model of software maintenance. Proceedings of the Twentieth International Conference on Software Engineering, Kyoto, Japan, 422-425.

Keil, M., Cule, P., Lyytinen, K., & Schmidt, R. (1998, November). A framework for identifying software project risk. Communications of the ACM (publication) Communications of the ACM - (CACM) A monthly publication by the Association for Computing Machinery sent to all members. CACM is an influential publication that keeps computer science professionals up to date on developments. , 76-83.

Kendrick, T. (2003). Identifying and managing project risk: Essential tools for failure-proofing your project. New York New York, state, United States
New York, Middle Atlantic state of the United States. It is bordered by Vermont, Massachusetts, Connecticut, and the Atlantic Ocean (E), New Jersey and Pennsylvania (S), Lakes Erie and Ontario and the Canadian province of
: AMACOM AMACOM American Management Association .

King, W. (2001, February). Guest editorial developing a sourcing strategy for IS: A behavioral decision process and framework. IEEE Transactions on Engineering Management, 48(1), 15-24.

Mann, C. (2002, July/August). Why software is so bad. MIT MIT - Massachusetts Institute of Technology  Technology Review, 33-38.

Nogueira de Leon, J. (2000, September). A formal model for risk assessment in software projects. Unpublished doctoral dissertation, Naval Postgraduate School The Naval Postgraduate School is a graduate school operated by the United States Navy. Located in Monterey, California, it grants primarily master's degrees plus some doctoral degrees to its students, who are mostly active duty officers from U.S. and foreign military services. , Monterey, CA.

McGuire, E. (1996). Factors affecting the quality of software project management: An empirical study based on the capability maturity model. Software Quality Journal, 5, 305-317.

Moeller, M., Hochstetler, S., Glasmacher, P., Jewan, P., Pathre, M., Polacheck, J., Sampath, V., & Ziller, H. (2000). Tivoli Netview 6.01 and friends. Austin, TX: IBM (International Business Machines Corporation, Armonk, NY, The world's largest computer company. IBM's product lines include the S/390 mainframes (zSeries), AS/400 midrange business systems (iSeries), RS/6000 workstations and servers (pSeries), Intel-based servers (xSeries)  Redbook.

Ogasawara, H., Yamada, A., & Kojo, M. (1996, March). Experiences of software quality management using metrics through the life-cycle. Proceedings of the Eighteenth International Conference on Software Engineering, Berlin, Germany, 179-188.

Osmundson, J., Michael, J., Machniak, M., & Grossman, M. (2003, September). Quality management metrics for software development. Information and Management, 40(8), 799-812.

Padayachee, K. (2002, September). An interpretive study of software risk management perspectives. Proceedings of the Annual Research Conference of the South African Institute of Computer Science and Information Technologists on Enablement through Technology, Port Elizabeth Port Elizabeth, city (1991 pop. 670,653), Eastern Cape, SE South Africa, on Algoa Bay, an arm of the Indian Ocean. It is a tourist center and a major seaport that ships diamonds, wool, fruit, and other items. , South Africa South Africa, Afrikaans Suid-Afrika, officially Republic of South Africa, republic (2005 est. pop. 44,344,000), 471,442 sq mi (1,221,037 sq km), S Africa. , 118-127.

Pressman, R. (2001). Software engineering a practitioner's approach (5th ed.). New York: McGraw-Hill.

Pritchett, W. (2001, September). An object-oriented metrics suite for Ada 95. Proceedings of the Annual ACM SIGAda International Conference on Ada, Bloomington, MN, 117-126.

Royce, W. (1990, November). Pragmatic quality metrics for evolutionary software development. Proceedings of the Conference on TRI-ADA '90, Baltimore, MD, 551-565.

Schmidt, M. (2000). Implementing the IEEE software engineering standards. Indianapolis, IN: Sams.

Shaw, M., & Garlan, D. (1996). Software architecture perspectives on an emerging discipline. Upper Saddle River Saddle River may refer to:
  • Saddle River, New Jersey, a borough in Bergen County, New Jersey
  • Saddle River (New Jersey), a tributary of the Passaic River in New Jersey
, NJ: Prentice Hall Prentice Hall is a leading educational publisher. It is an imprint of Pearson Education, Inc., based in Upper Saddle River, New Jersey, USA. Prentice Hall publishes print and digital content for the 6-12 and higher education market. History
In 1913, law professor Dr.

Standish Group. (2003). Chaos. In The Standish Group report. West Yarmouth, MA: The Standish Group.

Sturm, R., Morris, W., & Jander, M. (2000). Foundations of service level management. Indianapolis, IN: Sams.

Sumner, M. (2000, April). Risk factors in enterprisewide information management system projects. Proceedings of the SIGCPR SIGCPR Special Interest Group on Computer Personnel Research  Conference on Computer Personnel Research, Chicago, 180-187.

Tricker, R., & Sherring-Lucas, B. (2001). ISO 9001:2000 in brief Oxford, MA: Butterworth Heinemann.

U.S. General Accounting Office. (2001). Major management challenges and program risks: Department of Defense. (U.S. GAO Report GAO 01-244). Washington, DC: U.S. Government Printing Office.

Wiegers, K. (2002). Software requirements (2nd ed.). Redmond, WA: Microsoft Press.

Zuse, H. (1991). Software complexity--measures and methods. Berlin, Germany: Walter De Gruyter & Co.


CDR (1) See CD-R and extension.

(2) (Call Detail Reporting) See call accounting.

(3) (Common Data Rate) A standard sampling rate for digital video for 480i and 576i systems. The rate is 13.5 MHz. See ITU-R BT.
 Leonard T. Gaines, USN is a Supply Corps Officer currently assigned to the Joint Logistics The art and science of planning and carrying out, by a joint force commander and staff, logistic operations to support the protection, movement, maneuver, firepower, and sustainmentof operating forces of two or more Military Departments of the same nation. See also logistics.  Operation Center at the Defense Logistics Agency Noun 1. Defense Logistics Agency - a logistics combat support agency in the Department of Defense; provides worldwide support for military missions
Defense Department, Department of Defense, DoD, United States Department of Defense, Defense - the federal department
. He has a master's degrees from the Naval Postgraduate School and the Industrial College of the Armed Forces The Industrial College of the Armed Forces (ICAF) is a U.S. military educational institution tasked with preparing military officers and civilian government officials for leadership and executive positions in the field of national security. . He has served on three ships and has had shore assignments in logistics, acquisition, and information technology. He is a member of the Acquisition Professional Community.

(E-mail address See Internet address.

e-mail address - electronic mail address

Dr. James Bret Michael is an Associate Professor of Computer Science and Electrical & Computer Engineering at the Naval Postgraduate School. His research and teaching interests center on distributed computing (1) The use of multiple computers networked throughout a wide geographical area, or the world via the Internet, in order to solve a single problem. See grid computing.

(2) The use of multiple computers in an enterprise rather than one centralized system.
 and software engineering as they apply to ballistic missile defense Missile defence is an air defence system, weapon program, or technology involved in the detection, tracking, interception and destruction of attacking missiles. Originally conceived as a defence against nuclear-armed ICBMs, its application has broadened to include shorter-ranged  and information warfare Also called "cyberterrorism," it refers to creating havoc by disrupting the computers that manage stock exchanges, power grids, air traffic control and telecommunications. While the term often deals with attacks against a nation, it may also refer to attacks on organizations and the . He serves on several government, editorial, and industry advisory boards. Michael has a doctorate degree from George Mason University Named after American revolutionary, patriot and founding father George Mason, the university was founded as a branch of the University of Virginia in 1957 and became an independent institution in 1972.  and is a senior member of the Institute of Electical and Electronic Engineers (IEEE).

(E-mail address:
COPYRIGHT 2004 Defense Acquisition University Press
No portion of this article can be reproduced without the express written permission from the copyright holder.
Copyright 2004, Gale Group. All rights reserved. Gale Group is a Thomson Corporation Company.

 Reader Opinion




Article Details
Printer friendly Cite/link Email Feedback
Title Annotation:RESEARCH
Author:Michael, James Bret
Publication:Defense A R Journal
Date:Dec 1, 2004
Previous Article:A performance-based technology assessment methodology to support DoD acquisition.
Next Article:Public and private partnerships in support of Performance-Based Logistics initiatives--lessons learned from Defense Logistic Agency partnerships.

Related Articles
DCX uses simulation for chassis development. (WIP).
EMC launches world's first integrated ATA networked storage solution.
IBM and Adobe join forces to deliver enterprise solutions based on document services.
Subject: Revitalizing the Software Aspects of Systems Engineering.
Defense ARJ executive editor.
REIT sells $1.7b portfolio.
Subject: Department of Defense (DoD) Support for the SmartBUY Initiative.
Ensuring the Navy's present and future technological edge.
IBM to accuire Internet Security solutions.
CompleteAuto Software Program EXPLODING With Over 10,000 Vehicles Listed on eBay Motors! Software Agreement Inquiries Pouring In On A Daily Basis!

Terms of use | Copyright © 2014 Farlex, Inc. | Feedback | For webmasters