Printer Friendly
The Free Library
22,710,190 articles and books

Determining quality in academic libraries.


This article summarize sum·ma·rize  
intr. & tr.v. sum·ma·rized, sum·ma·riz·ing, sum·ma·riz·es
To make a summary or make a summary of.

 the attempts to define and measure quality and effectiveness in academic libraries, from traditional evaluative studies to Total Quality Management (TQM (Total Quality Management) An organizational undertaking to improve the quality of manufacturing and service. It focuses on obtaining continuous feedback for making improvements and refining existing processes over the long term. See ISO 9000. ) and new research on user-defined criteria. Focusing on the organizational analysis of the library as a whole and the contribution it makes to the university or college, the article outlines a number of fundamental concepts and tools common to models of evaluation. Particular attention is then given to assessment in higher education higher education

Study beyond the level of secondary education. Institutions of higher education include not only colleges and universities but also professional schools in such fields as law, theology, medicine, business, music, and art.
 as a whole and ways in which determinants of library quality must be linked to educational outcomes. The concluding sections suggest several areas for future research and for collaboration among library managers, educational administrators, scholars, and measurement theorists.


Quality ... you know what it is, yet you don't know Don't know (DK, DKed)

"Don't know the trade." A Street expression used whenever one party lacks knowledge of a trade or receives conflicting instructions from the other party.
 what it is. But

that's self-contradictory. But some things are better than others, that

is, they have more quality. But when you try to say what the quality

is, apart from the things that have it, it all goes poof! (Pirsig, 1974, p.


To open with a quotation from Zen and the Art of Motorcycle Maintenance is more than just a literary conceit conceit, in literature, fanciful or unusual image in which apparently dissimilar things are shown to have a relationship. The Elizabethan poets were fond of Petrarchan conceits, which were conventional comparisons, imitated from the love songs of Petrarch, in which , for the book itself embodies the two levels on which one operates to understand quality in academic library services. Librarians must deal both with the nuts and bolts nuts and bolts
pl.n. Slang
The basic working components or practical aspects: "[proposing]
 of evaluating library services and with the elusive and challenging work of crafting a conceptual definition A conceptual definition is an element of the scientific research process, in which a specific concept is defined as a measurable occurrence. It is mostly used in fields of philosophy, psychology, communication studies. This is especially important when conducting a content analysis.  of effectiveness and a broad vision of information impact. Attempts to define effectiveness - an earlier term for quality - have been a strong thread running through the professional literature of U.S. and British librarianship since the 1960s. The current prominence of "quality," by which of course is meant high quality, emerges from the latest trend in business and industry but dovetails neatly with much existing research and practice in libraries. Guidelines guidelines, a set of standards, criteria, or specifications to be used or followed in the performance of certain tasks.
 and tools introduced in the guise Guise (gēz, gwēz), influential ducal family of France. The First Duke of Guise

The family was founded as a cadet branch of the ruling house of Lorraine by Claude de Lorraine, 1st duc de Guise, 1496–1550, who received
 of Total Quality Management (TQM) can be described as principles and techniques that, taken separately, have long been promoted as aspects of sound management. The emphasis on process rather than just measurement, however, has received new priority.

Few libraries exist in a vacuum, accountable only to themselves. There is thus always a larger context for assessing library quality, that is, what and how well does the library contribute to achieving the overall goals of the parent constituencies? The major objective for academic libraries, especially in an environment of increasing economic pressure, structural change, and technological innovation, must be to align themselves with the structures of higher education and the criteria by which those institutions are judged. The literature of educational effectiveness is enormous and, like library managers, higher education administrators have borrowed heavily from the recent business methods of Total Quality Management. The micro-evaluation of libraries has given countless opportunities for detailed studies, yet still lacking are agreed-upon and objective ways to measure and incorporate library value into such processes as academic accreditation, educational assessment, and ratings of graduate programs.

This article will briefly summarize the principal attempts to define and measure quality and effectiveness in academic libraries, focusing on the organizational analysis of the library as a whole and the contribution it makes to the university or college. While highlighting the major trends and recent new research, it is not intended to be a comprehensive history of the evaluation of academic libraries or even of the Total Quality Management movement in those libraries. Out of these highlights will be shown a number of fundamental concepts and problems common to many models of evaluation. It is primarily at the macro-evaluative level that one can distinguish key characteristics of academic libraries in contrast to other libraries that share many of the same functions. Although the organization-wide perspective must incorporate the detailed suborganizational approach of analyzing individual functions or departments, that voluminous work is not treated here, nor are the many pragmatic articles about ways to implement local evaluation and TQM projects. Despite the almost overwhelming amount of writing in all of these areas, the profession still lacks many essential models and forms of measurement; the concluding sections of this article will suggest several areas for future research and for collaboration among library managers, educational administrators, scholars, and measurement theorists.


What does the term quality (a positive adjective adjective, English part of speech, one of the two that refer typically to attributes and together are called modifiers. The other kind of modifier is the adverb.  is always assumed to be there) mean in recent library and education writings? Quality and quality control have been much used and too narrowly defined. The first implies an ultimate state of being, and the second seems to refer more to the process of getting there. Both have been tossed around a bit carelessly care·less  
1. Taking insufficient care; negligent: a careless housekeeper; careless proofreading.

 by those eager for a new tool to handle a tough problem. The determination of quality, however, does not automatically imply using any one type of measurement or analysis (e.g., TQM), nor is it just another synonym synonym (sĭn`ənĭm) [Gr.,=having the same name], word having a meaning that is the same as or very similar to the meaning of another word of the same language. Some are alike in some meanings only, as live and dwell.  for output or performance measures, though those must be part of any serious quality program.

Definitions of library effectiveness have ranged from technical efficiency measures to vague statements of goodness, but most have focused on goal achievement, efficiency, user satisfaction, personnel management, and ability of the organization to survive. Based on a reading of professional attempts to sort this out (see excellent summaries in Du Mont & Du Mont, 1979, pp. 107-10; McDonald & Micikas, 1994, pp. 7-19), it would appear that the terms quality and effectiveness are being used to mean the same thing: achieving a quality of service that satisfies to a high degree the information and research needs of faculty, students, and other users; that contributes demonstrably de·mon·stra·ble  
1. Capable of being demonstrated or proved: demonstrable truths.

2. Obvious or apparent: demonstrable lies.
 to the success of the institution's educational and developmental goals; and that accomplishes this in an operationally effective manner. When one tries to nail down the implications of this definition, roadblocks quickly appear - effective by what criteria, meeting what level of needs, at what cost, for what purpose?

These questions can be asked at several levels, for example, for an individual academic library or throughout higher education for understanding the systems of library services. An overall assessment of the quality of library service cannot be achieved without progressing through a series of basic steps that are common to almost all systems of evaluation. Too often, however, the first and the last of these get ignored or are hastily hast·y  
adj. hast·i·er, hast·i·est
1. Characterized by speed; rapid. See Synonyms at fast1.

2. Done or made too quickly to be accurate or wise; rash: a hasty decision.
 swept aside with platitudes.

1. What is the purpose in establishing library services? Is it enough to say


it is to meet the needs of the users" and to "support the institution?"

It may also involve understanding the program of the institution and

its philosophy of resource allocation resource allocation Managed care The constellation of activities and decisions which form the basis for prioritizing health care needs  in enough detail so that difficult

and perhaps venturesome choices can be made about what services

and materials to provide. 2. How does one know whether and when the mission is being accomplished?

Once the goals are understood, one must find ways to measure or

track them. This is the problem libraries have been struggling with

for decades. In academic libraries, the question is difficult to answer

because of the lack of performance measures that make sense across

institutions and that link library processes to educational and research

outcomes. 3. How do library managers and staff effect improvements to achieve quality and

effectiveness? Setting goals and measuring progress is only a part of the

process. To ensure quality and effectiveness, library managers and

staff must continually seek ways to move closer to the goals through

monitoring, feedback, and communication structures that address

problems, determine needs, and support change. Underneath it all is

the realization that "there" keeps on moving. 4. What is the ultimate evidence of success? Many statements about success

are more definitions of its meaning than a description of the evidence

one would use to prove its existence. To demonstrate success both in

providing service and in doing it efficiently will require deeper understanding

both of the "outcomes" question, and of the rather traditional

problem of measuring costs.

Academic library quality must be defined to fit local programs, yet it must also incorporate the contribution to the higher education system, which lends itself to being defined in terms of regional and national frameworks such as accreditation. Local evaluations have tended to focus on micro aspects of service, evaluating delivery systems and expressed patron needs; institutional-level assessments have either relied on traditional library data from national and peer sources that then are unlinked to local goals or on broad educational models that do not address support services support services Psychology Non-health care-related ancillary services–eg, transportation, financial aid, support groups, homemaker services, respite services, and other services  such as libraries. Academic librarians do not have concrete ways to assess what the library contributes to the delivery of effective educational and research services by the campus itself. This is referred to as "outcome" or "impact" assessment and will be discussed in a later section.

Development of Approaches to Academic

Library Effectiveness

As the focus on modern approaches to library management and research grew in the 1940s and 1950s, articles and studies on aspects of evaluation immediately began appearing. General summaries of this immense body of literature (Du Mont & Du Mont, 1979; McDonald & Micikas, 1994) cite articles back at least to 1954 in which definitional and professional quandaries were raised that are still being debated. Techniques and frameworks have been adapted from various disciplines outside the library science field: industrial process management, organizational research, institutional research, behavioral dynamics, social program review, and educational assessment, to name the most heavily used. Evaluation research in libraries draws most on major and ongoing contributions from Childers (1989), Hernon and McClure (1990), Lancaster (see Baker & Lancaster, 1991), Van House (1989), and Van House et al., 1990), citing here only a fraction of the output of these writers. Initially, effectiveness and user satisfaction were studied more thoroughly in public libraries, while much of the earlier literature on academic libraries seems to emphasize scientific measurement details more than concepts of quality: studies of catalog catalog, descriptive list, on cards or in a book, of the contents of a library. Assurbanipal's library at Nineveh was cataloged on shelves of slate. The first known subject catalog was compiled by Callimachus at the Alexandrian Library in the 3d cent. B.C.  use, operations research operations research

Application of scientific methods to management and administration of military, government, commercial, and industrial systems. It began during World War II in Britain when teams of scientists worked with the Royal Air Force to improve radar detection of
 for library internal functions, cost/time factors, and the design of information retrieval information retrieval

Recovery of information, especially in a database stored in a computer. Two main approaches are matching words in the query against the database index (keyword searching) and traversing the database using hypertext or hypermedia links.

Orr (1973) emerged from the special library field and published articles that remain milestones. He suggested a distinction between library quality (how good is the service) and value (how much good does it do), and four areas within which to define measurement variables (resources, capability, utilization, and beneficial effects). He implied that particular measures could be developed but, over twenty years TWENTY YEARS. The lapse of twenty years raises a presumption of certain facts, and after such a time, the party against whom the presumption has been raised, will be required to prove a negative to establish his rights.
 later, it is not clear that it has been accomplished even though the framework is still the same. Taylor (1972) stressed the need for academic libraries to move from measures of quantity to ones of process and user satisfaction, anticipating the ideas of TQM well before its arrival in most U.S. businesses and professions. In the same anthology, Dougherty (1972) called for quantification of outputs and their impact, and linked staff participation to library effectiveness in a systems management approach. Du Mont and Du Mont (1979) develop criteria and measurement techniques for assessing library effectiveness based on models of goal attainment, efficiency, user satisfaction, and behavioral factors; they also delineate the gaps in the varying approaches taken to library effectiveness and design a taxonomy taxonomy: see classification.

In biology, the classification of organisms into a hierarchy of groupings, from the general to the particular, that reflect evolutionary and usually morphological relationships: kingdom, phylum, class, order,
 that attempts to integrate the approaches.

The literature on performance and output measures is documented by Goodall (1988), Shapiro (1991), and Van House (1989) who clarifies that performance is a broader term that may actually be used with measures of input, process, output, and outcomes. Van House, Weil, and McClure (1990) provide a signal publication, an attempt to develop a practical manual that would actually make a difference in library statistics and evaluation. From a British perspective, Blagden (1980) and Allred (1979) both present compelling analyses of the problem of setting relevant criteria and the need to evaluate libraries based on performance, outcomes, and user satisfaction. The literature of organizational dynamics and behavioral styles of management and interaction also contributed to the evolving notions of what constitutes a well-running library. Thus, leading up to the seemingly seem·ing  
Apparent; ostensible.

Outward appearance; semblance.

seeming·ly adv.
 recent quality movement, one realizes that there is no dearth of research and writing on how to determine library goodness and how to manage for change and improvement.

As the Total Quality Management movement began to be adopted in libraries, there have been a rash of practical and theoretical publications outlining the basic concepts and how to apply them. Jurow and Barnard (1993), Siggins and Sullivan (1993), Riggs (1993), and Shaughnessy (1993) are but a few of the most useful examples. This literature in fact brings together many previous issues and approaches, for example Riggs, (1992a), Whitehall (1992), and Clack (1993) blend TQM, organizational development, and strategic planning Strategic planning is an organization's process of defining its strategy, or direction, and making decisions on allocating its resources to pursue this strategy, including its capital and people. . The rapidity and fervor with which TQM has swept organizations has led to misconceptions Misconceptions is an American sitcom television series for The WB Network for the 2005-2006 season that never aired. It features Jane Leeves, formerly of Frasier, and French Stewart, formerly of 3rd Rock From the Sun.  and skepticism. TQM does not imply a new kind of measurement, although it does urge the use of measurement tools for tracking processes and deducing performance problems. At the other extreme, TQM does imply some form of benchmarking or process control, more than just participatory management Participatory management is the practice of empowering employees to participate in organizational decision making. This practice grew out of the human relations movement in the 1920s, and is based on some of the principles discovered by scholars doing research in management and  or quality circles; some articles that purport To convey, imply, or profess; to have an appearance or effect.

The purport of an instrument generally refers to its facial appearance or import, as distinguished from the tenor of an instrument, which means an exact copy or duplicate.

PURPORT, pleading.
 to describe quality approaches reveal little more than traditional consultative and consensus management styles. The emphasis on user surveys is not new, but the reorienting of the whole organization toward a focus on customer satisfaction, the broadening of the definition of customer, and the evaluation of processes with this in mind goes beyond earlier views of how to solicit and interpret use and user data.

There are strong links between evaluative and planning frameworks, and the assessment of the director, of administrative style, and organizational structure This article has no lead section.

To comply with Wikipedia's lead section guidelines, one should be written.
. Leadership and management are key to quality at every stage. While not explicitly couched in terms of effectiveness criteria, much of the work analyzing the role of the library director and the shape of the internal organization suggests that these connections exist in the mind of higher administration and staff. Articles by Lewis (1986), Sweeney (1994), and Buschman and Stephen (1993) specifically discuss the implications of administrative leadership for the success of library operations. The literature of customer service (St. Clair, 1993; Millson-martula & Menon, 1995) bridges TQM and traditional management concepts. A series of articles by Martell and co-authors (Martell, 1983a, 1983b, 1985; Martell & Tyson, 1983; Martell & Untawale, 1983) used the quality of work life" model as a way of analyzing organizational structure and staff satisfaction, the implication being that this was an essential precondition pre·con·di·tion  
A condition that must exist or be established before something can occur or be considered; a prerequisite.

 for overall library effectiveness. While not the technical approach found in pure TQM and evaluation studies, this broad area of thought pertains closely to the professional discussion of how to implement mechanisms that will promote quality library service.

When scanning the above material, it is hard not to feel that librarians are constantly reinventing the wheel Reinventing the wheel is a phrase that means a generally accepted technique or solution is ignored in favor of a locally invented solution. To "reinvent the wheel" is to duplicate a basic method that has long since been accepted and even taken for granted. . For decades, we have had models of measurement directed toward helping assess effectiveness. There is a high degree of agreement among the models and concepts espoused by most of those who write on this topic, and yet practicing library managers still do not have such agreement among themselves as to what constitutes library quality. The difficulty lies in trying to find a single model or set of simple indicators that can be used by different institutions, and that will compare something across large groups that is by definition only locally applicable - i.e., how well a library meets the needs of its institution. Librarians have either made do with oversimplified o·ver·sim·pli·fy  
v. o·ver·sim·pli·fied, o·ver·sim·pli·fy·ing, o·ver·sim·pli·fies
To simplify to the point of causing misrepresentation, misconception, or error.

 national data or have undertaken customized local evaluations of effectiveness, but there has not been devised an effective way to link the two. Existing library and higher education processes and frameworks have tended to draw on both.

Processes and Frameworks

The organizational effectiveness Organizational effectiveness is the concept of how effective an organization is in achieving the outcomes the organization intends to produce. The idea of organizational effectiveness is especially important for non-profit organizations as most people who donate money to non-profit  literature, as reviewed most recently by Cullen and Calvert (1995), and McDonald and Micikas (1994), presents four major approaches to organizational evaluation: (1) the goal attainment model, (2) the system resource model, (3) the internal processes model, and (4) the constituency satisfaction model. These do not prescribe pre·scribe
To give directions, either orally or in writing, for the preparation and administration of a remedy to be used in the treatment of a disease.
 exact "measures," they are interpretive in·ter·pre·tive   also in·ter·pre·ta·tive
Relating to or marked by interpretation; explanatory.

in·terpre·tive·ly adv.
 contexts within which particular analyses are designed. Academic and public libraries have at times advocated and undertaken evaluations that fit within all four models. The purposes for evaluation emerge from institutional frameworks, however, not from the models themselves. These broader purposes or frameworks include internal library management and service planning; strategic planning, program reviews and self-studies (for the library or the institution); and accreditation reviews.

Total Quality Management and continuous quality improvement programs are universalizing schemes that offer a formal approach to the requirements of the broader level of accountability, while incorporating measurement and process techniques typical of all four models of evaluating effectiveness. In earlier years, librarians adopted other equally applicable management techniques such as MBO MBO

See: Management buyout
 (Management by Objectives), PPB (Programmers, Planning, Budgeting), and elaborate versions of strategic planning. The assessment of quality may be usefully situated within any of these models. Each still needs to have a clear conceptual structure (as outlined below), or else they are little more than meaningless exercises. There are many variations and considerable jargon jargon, pejorative term applied to speech or writing that is considered meaningless, unintelligible, or ugly. In one sense the term is applied to the special language of a profession, which may be unnecessarily complicated, e.g., "medical jargon.  and overlap among these management and planning approaches, and some or all are used as part of the others. It may be sensible simply to adopt whichever terminology and structure is currently prevalent on campus. Not only is it politically more expedient ex·pe·di·ent  
1. Appropriate to a purpose.

a. Serving to promote one's interest: was merciful only when mercy was expedient.

, but it may lessen less·en  
v. less·ened, less·en·ing, less·ens
1. To make less; reduce.

2. Archaic To make little of; belittle.

To become less; decrease.
 the amount of convincing needed among staff and will ultimately still cover all the important concepts.

There is great consistency throughout these articles, research projects, management schemes and standards, in and out of librarianship. Does the repetition suggest that the lessons have not yet been learned? Rather, it may be that there is no new "silver bullet silver bullet - magic bullet " or shortcut (1) In Windows, a shortcut is an icon that points to a program or data file. Shortcuts can be placed on the desktop or stored in other folders, and double clicking a shortcut is the same as double clicking the original file.  for academic libraries. Experience reveals that one may have the formal process without getting good results and vice versa VICE VERSA. On the contrary; on opposite sides. ; the determining factor is whether the library staff, managers, and stakeholders Stakeholders

All parties that have an interest, financial or otherwise, in a firm-stockholders, creditors, bondholders, employees, customers, management, the community, and the government.
 define certain fundamental assumptions about the nature of the enterprise. All the above have in common the following underlying components:

* the careful definition of goals, or of some kind of criteria against which

success can be assessed; * a focus on meeting the needs of the users, as defined by the library

and the institution; * leadership: a commitment from the top, conscious efforts at ensuring

communication, the provision of training and resources for the process

of evaluation, the active support of a process to promote shared

values; * the involvement of all levels of staff in goal-setting, evaluation, and the

improvement of processes and services; and

integrating a process of evaluation that is continuous and adaptive,

whether that process is based on the framework of TQM, strategic planning,

or another model.

Within the frameworks being used to assess quality, another consistent pattern is the set of organizational parameters that must be defined even before actual measures or assessments can be undertaken. These will have a fundamental impact not only on the choice of measures but also on the interpretation of the results. It is rarely possible to collect data that perfectly match the dimensions and timing of every situation, thus using figures and measures in assessments often requires making a compromise to help achieve specific goals. The nature of the compromise varies with the desired goal, for example, better internal library management, campus budget reallocation Noun 1. reallocation - a share that has been allocated again
allocation, allotment - a share set aside for a specific purpose

2. reallocation
, regional accreditation Regional accreditation is a term used in the United States to refer to the process by which one of several accrediting bodies, each serving one of six defined geographic areas of the country, accredits schools, colleges, and universities. , or institutional success in competing for external support. Given all the logistical lo·gis·tic   also lo·gis·ti·cal
1. Of or relating to symbolic logic.

2. Of or relating to logistics.

[Medieval Latin logisticus, of calculation
 and definitional problems in evaluating and improving libraries, effort need not be focused in areas that do not help target the overall purposes and the principal stakeholders, whatever and whoever those may be.

Key to being able to make claims about library goodness is a deconstruction deconstruction, in linguistics, philosophy, and literary theory, the exposure and undermining of the metaphysical assumptions involved in systematic attempts to ground knowledge, especially in academic disciplines such as structuralism and semiotics.  of the factors that go into answering the question, "Is a particular library meeting the needs of its institution?" Those needs may be many, they may relate to present and future generations and local and national roles, but they must be articulated and usually among many groups beyond library walls. Quality programs, strategic planning, and ongoing internal evaluation are all built, directly or indirectly, on the following: * Mission of the specific institution of higher education, and derived

from that, of the library. This may not be so simple as it sounds, and

much writing has gone into explaining how to craft a meaningful mission

statement. Meyer (1995) gives some straightforward examples of

phrasing the library mission in support of the broader educational

outcome. * Identification of user groups and their particular differing needs (for

example, faculty, undergraduate students, graduate students, distance

learners, administrators, the general public, alumnae, consortial

groups, even future users or the national posterity POSTERITY, descents. All the descendants of a person in a direct line. " if part of the mission

is to serve as a major research library). * Goals for accomplishing the mission and serving the users, which

should include language that can lead to criteria and measurement of

performance. While the simple goal-attainment model of evaluation

has been criticized, statements of goals are still needed as building

blocks for more multidimensional mul·ti·di·men·sion·al  
Of, relating to, or having several dimensions.

 determinations of effectiveness. * Determination of audiences and organizational processes to which the

library is accountable, that is, how and by whom will quality and effectiveness

be ascertained? This is not the same as identifying the user

groups, though there is overlap. Depending on the mission and governance

of the institution, accountability may relate to: user satisfaction;

budgetary performance; relevance of support to academic programs;

success in contributing to academic accreditation; success in

gaining state and legislative appropriations; and success in achieving

national participation in research or other roles.

Discerning dis·cern·ing  
Exhibiting keen insight and good judgment; perceptive.

dis·cerning·ly adv.
 the Quality of a Library: Tools and Models

Any library is working to mobilize mo·bi·lize
1. To make mobile or capable of movement.

2. To restore the power of motion to a joint.

3. To release into the body, as glycogen from the liver.
 resources to provide services that meet the needs of users and that fulfill ful·fill also ful·fil  
tr.v. ful·filled, ful·fill·ing, ful·fills also ful·fils
1. To bring into actuality; effect: fulfilled their promises.

 the overall mission of the institution. Is "service" quality the only important part of "library" quality? What is actually meant when referring to "library service?" The attainment of a high-quality library can be judged completely, subjectively, and individually, but ultimately most stakeholders want to know whether this mobilization mobilization

Organization of a nation's armed forces for active military service in time of war or other national emergency. It includes recruiting and training, building military bases and training camps, and procuring and distributing weapons, ammunition, uniforms,
 has been done in the most effective way with the most pertinent services and resources. This implies some kind of measurement, whether of a traditional or a more venturesome nature. But what does one want to measure and why? It sounds simple, but it should not be taken for granted Adj. 1. taken for granted - evident without proof or argument; "an axiomatic truth"; "we hold these truths to be self-evident"
axiomatic, self-evident

obvious - easily perceived by the senses or grasped by the mind; "obvious errors"
, and this question has a direct impact on what tools are used, where one gathers data, and how it is interpreted. The above structures for planning and evaluating help define the measurement context and, within the chosen context, a library might then choose from an immense and not always well-defined array of measurement tools and models.

Knightly (1979, p. 174) distinguishes clearly and simply among library inputs, process, outputs, and effect (impact) as the components of a system, and the four types of evaluation that may result: (1) effort evaluation (inputs), (2) process evaluation (appropriateness and efficiency of activities), (3) effectiveness (outputs and the accomplishment of objectives), and (4) impact (on the parent or broader community). As the development of TQM took hold over a decade later, it is apparent that the evaluative structures for looking at processes and effectiveness were long in place. Knightly further outlined the debate over measures themselves, enumerating seven types of measurement criteria: (1) assessment based on user opinion, (2) expert opinion, (3) standards, (4) peer comparisons, (5) quantifiable Quantifiable
Can be expressed as a number. The results of quantifiable psychological tests can be translated into numerical values, or scores.

Mentioned in: Psychological Tests
 outputs, (6) quantifiable processes, and (7) based on unit costs in combination with the other criteria. From these can be derived an enormous armada An earlier brand name for laptop computers from Compaq. The line was noted for its quality and innovative features.  of tools and data. The measurement that will lead to an assessment of quality should ideally draw on all seven categories, though frequently only one or two are used at a time.

King and Griffiths (1991) summarize their long record of evaluative research and outline four categories of generic measures: (1) input cost measures (staff, equipment, facilities, collections, the allocation among those, and their attributes); (2) output measures (quality of service, timeliness, availability, accessibility); (3) effectiveness measures (amount of use, user satisfaction, user-expressed importance of services, consequences of use of service), and (4) service domain measures (total population size and attributes, user population size and attributes). They further identify four kinds of derived indicators: operational performance, effectiveness, cost-effectiveness, and impact. While terminology differs, these same categories hold across many schemes of measurement: inputs, processes, outputs, outcomes. It is the relationships among the measures that provide a basis for decision-making, and what may start out looking like a quantitative measure can emerge as a qualitative indicator.

Measures of service have been confused with performance and also with "access." Access is a particularly mutable mu·ta·ble  
a. Capable of or subject to change or alteration.

b. Prone to frequent change; inconstant: mutable weather patterns.

 term in the profession right now; from some articles, it may be taken to mean something as simple as the degree of physical access (i.e., seats) or bibliographic access to materials owned (i.e., catalog entries), while it is also used to allude to allude to
verb refer to, suggest, mention, speak of, imply, intimate, hint at, remark on, insinuate, touch upon see see, elude
 the mix of services and systems that provide users with documents or electronic information not held on site. Performance, as stated earlier, is a dimension that may apply to inputs, processes, or outputs. Hemon and McClure (1990) and Van House, Weil, and McClure (1990) describe many performance measures, some of which are very simple. There are no iight or wrong performance measures, and a library needs to use several in combination with other categories of measurement. Performance is generally thought of as an internal benchmark, though there are also library directors who would like to see national standards emerge. The value of recent TQM efforts is the increased focus on evaluating local performance, on services as processes, and on measuring the effectiveness of operations by looking at customary factors such as speed, redundancy of tasks, costs, productivity, satisfaction, and "reach" into the user population.

Qualitative mechanisms for assessing library effectiveness include interviews, surveys, the use of consultants or external review teams ("experts"), unobtrusive studies, process analysis, job factor analysis, and organizational structure analysis. These are often brushed brushed  
Having a nap produced by brushing: a dress made of brushed cotton.


 aside as being too easily biased and not readily compared with results from other institutions. Such techniques may well be the best groundwork for confirming new measures, however, by using a qualitative approach consistently with certain groups of libraries, and seeking combinations or correlations with quantitative measures of inputs, outputs, and performance. TQM itself is a qualitative mechanism and an aspect of management process. In its purest form, it is based on the regular use of quantitative techniques, but organizations have implemented TQM processes and benchmarking independently of each other.

Academic libraries often have problems building any but the most rudimentary rudimentary /ru·di·men·ta·ry/ (roo?di-men´tah-re)
1. imperfectly developed.

2. vestigial.

 measurement process into routine operations or finding the time, money, and expertise to conduct special evaluations. Thus there is a preference for relying on gathering routine data and augmenting it with that from external sources. Some of the most common sources for academic library data are national associations like the American Library Association American Library Association, founded 1876, organization whose purpose is to increase the usefulness of books through the improvement and extension of library services.  and the Association of Research Libraries, bibliographic utilities, and serials vendors. These data usually reflect only some subset A group of commands or functions that do not include all the capabilities of the original specification. Software or hardware components designed for the subset will also work with the original.  of the academic library universe and may change from year to year. Data from vendors can be very detailed and revealing but may be proprietary and thus limited for peer-group and broader uses. Organizational and library school research projects have yielded special studies on valuable topics (e.g., alternative sources of revenue, foreign serials, access measures) but those too are not done every year or for consistent peer groupings.

The annual statistics from the Association of Research Libraries, by the very fact of their publication in "rank" order, have been used by many, both within libraries and in academia in general, as a de facto [Latin, In fact.] In fact, in deed, actually.

This phrase is used to characterize an officer, a government, a past action, or a state of affairs that must be accepted for all practical purposes, but is illegal or illegitimate.
 indicator of quality despite vigorous protestations from the members of ARL ARL - ASSET Reuse Library  and criticism from higher education analysts who have assumed without further inquiry that ARL really does Warren Trotter, better known as Really Doe, is an American rapper from Chicago, Illinois. He is affiliated with Kanye West and his G.O.O.D. Music family and label. Discography
  • "Day By Day"
  • "Plastic"
  • "The Love"
 promote the membership index as a "goodness" measure. The counts of volumes, expenditures, and other inputs and outputs (e.g., circulation) have been debated internally within ARL since the 1940s (Shaughnessy, 1990). Almost as early, the ARL attempted to devise models and measures that would provide comparable performance and output assessment in a form that might ultimately be capable of integration into the membership criteria (Shaughnessy, 1990; Shapiro, 1991). These efforts often appear as supplemental or occasional reports and may take years to be integrated into the larger surveys. Testing of new measures for use with the membership index continues but has not yielded anything statistically significant.

Another strategy is to make better use of government and higher education data that already explicitly include libraries, building toward models of the role of the library in the academic institution. These tend to still be fairly simple collections of input data for higher education, but using them is an important leap in overcoming professional insularity in·su·lar  
a. Of, relating to, or constituting an island.

b. Living or located on an island.

 and achieving cognizance The power, authority, and ability of a judge to determine a particular legal matter. A judge's decision to take note of or deal with a cause.

That which is cognizable to a judge is within the scope of his or her jurisdiction.
 of the measures key to academic administrators. These include the immense data sets generated by the Integrated Post-Secondary Education Data Survey (IPEDS IPEDS Integrated Postsecondary Education Data System
IPEDS Interactive Public Exhibits and Digital Signage
), the cost matrices of the Higher Education Price Index The Higher Education Price Index (HEPI) is a measure of the inflation rate applicable to United States higher education; more precisely, the increase in costs in a defined basket of goods and services typically purchased by institutions of higher education.  (HEPI HEPI Higher Education Price Index
HEPI Heavy Equipment Point of Impact (military paradrop)
HEPI High Explosive Penetrating Incendiary
), the recent benchmarking study launched by the National Association of College and University Business Officers (NACUBO NACUBO National Association of College and University Business Officers ), the annual surveys of information technology resources conducted by CAUSE, and the periodic rating of doctoral programs put together by the National Research Council, which includes factors for campus support resources such as libraries. Each of these suffers from the same limitations we find in data from ARL, ALA, and library-specific sources, but each organization has also overtly o·vert  
1. Open and observable; not hidden, concealed, or secret: overt hostility; overt intelligence gathering.

 approached library groups in search of ways to more clearly and consistently assess library effectiveness across programs and institutions.

The interpretation of the data depends on the original criteria, what is defined as important, and whose models are being followed. Arguments over how to establish criteria for success and even criteria for measurement can be traced throughout the literature of evaluation; attempting to state what is or is not "quality" will inevitably raise questions about the comparability of the variables, the philosophy of the data collection project, and the absolute definitions of success against which the data are being judged. Out of context, the data can be used in many different ways. There is nonetheless a stubbornly stub·born  
adj. stub·born·er, stub·born·est
a. Unreasonably, often perversely unyielding; bullheaded.

b. Firmly resolved or determined; resolute. See Synonyms at obstinate.

 resilient See resiliency.  notion that the "statistics" are to blame, that they are all worthless, and that adherence to them is what is slowing the quest for Verb 1. quest for - go in search of or hunt for; "pursue a hobby"
quest after, go after, pursue

look for, search, seek - try to locate or discover, or try to establish the existence of; "The police are searching for clues"; "They are searching for the
 better models of library effectiveness. While ARL and others work to piece together an approach to a difficult problem that goes well beyond their membership and resources, it may be that much of the criticism is a bit misplaced mis·place  
tr.v. mis·placed, mis·plac·ing, mis·plac·es
a. To put into a wrong place: misplace punctuation in a sentence.

. To meet the expressed goal of national-level data gathering, one is never going to be able to use the same kinds of data and instruments that one would use to determine whether an individual academic library is of high quality in the minds of its local users. Input and output statistics are still useful building blocks for looking at organizational performance Organizational performance comprises the actual output or results of an organization as measured against its intended outputs (or goals and objectives).

Specialists in many fields are concerned with organizational performance including strategic planners, operations,
 from year to year.

The most common contexts for bringing meaning to these measures are library and educational standards and guidelines; for example, the standards of the Association of College and Research Libraries ACRL ACRL Association of College and Research Libraries
ACRL Administrative Cost Reimbursements to Localities
, 1989, 1995) and the documents of the regional accrediting agencies. Notably, the Association of Research Libraries does not issue standards, and it makes no claim that its membership index is other than an internal mechanism for comparing potential members to the existing group. Academic libraries may also wish to develop measures related to specific standards for branch libraries, distance learning programs, media services, rare book and special collections In library science, special collections (often abbreviated to Spec. Coll. or S.C.) is the name applied to a specific repository within a library which stores materials of a "special" nature. , and the like. It will be interesting to see whether libraries or higher education begin to use portions of the ISO (1) See ISO speed.

(2) (International Organization for Standardization, Geneva, Switzerland, An organization that sets international standards, founded in 1946. The U.S. member body is ANSI.
 9000 standard recently adopted for quality improvement programs (see, for example, Arnold, 1994).

The use of library standards as guides for assessing effectiveness is documented by Kania (1988), Kaser (1982), and Lynch (1982). Most standards seem to have moved away, though with some hesitation, from reliance on absolute quantities, and the focus is more on recommended structures, policies, and processes. Kania is particularly valuable for her derivation derivation, in grammar: see inflection.  of a series of further performance standards for use in self studies and accreditation, though she avoids recommending specific measures. Collections of norms and ratios have taken on the role of benchmarks for comparative assessment, for example, those listed in the annual compilation of ARL statistics (now on the World Wide Web in user-definable format), or the massive compilations edited by Minter (1993a, 1993b) using the IPEDS data. As these data are accompanied by little information about institutional characteristics and success factors, they are, at best, a starting point Noun 1. starting point - earliest limiting point
terminus a quo

commencement, get-go, offset, outset, showtime, starting time, beginning, start, kickoff, first - the time at which something is supposed to begin; "they got an early start"; "she knew from the
 or a very rough indicator; unfortunately, a ratio from an institution that is subjectively perceived as good, or one which is a competitive peer, will be assumed to have great validity and meaning where there is no justification for that in fact.

Within the substructure substructure /sub·struc·ture/ (-struk-chur) the underlying or supporting portion of an organ or appliance; that portion of an implant denture embedded in the tissues of the jaw.

 of standards or accreditation, institutions will generally use peer group analysis rather than absolute definitions of quality. The principal data series rarely go into enough detail to know for sure whether one is comparing apples and apples. Two libraries may have quite different ways of operating - i.e., responding to local needs and not necessarily implying better or worse management. It is possible that the peer institutions used by the administration for strategic planning will not each have a library that functions comparably. It may be further complicated if an institution wishes to look at an "aspiration aspiration /as·pi·ra·tion/ (as?pi-ra´shun)
1. the drawing of a foreign substance, such as the gastric contents, into the respiratory tract during inhalation.

" group and not a literal In programming, any data typed in by the programmer that remains unchanged when translated into machine language. Examples are a constant value used for calculation purposes as well as text messages displayed on screen. In the following lines of code, the literals are 1 and VALUE IS ONE.  peer group. The ability to make unambiguous and meaningful comparisons is an important issue in assessment whether through TQM or more traditional evaluation, and it is why many libraries have come to rely so heavily on regional and national compilations, whatever their flaws.

The two newest attempts at creating models of effectiveness show great promise for both local and comparative library assessment. Cullen and Calvert (1995), building on methods formulated by Childers and Van House, identify key performance indicators Key Performance Indicators (KPI) are financial and non-financial metrics used to quantify objectives to reflect strategic performance of an organization. KPIs are used in Business Intelligence to assess the present state of the business and to prescribe a course of action.  in university libraries as perceived by six separate stakeholder stakeholder n. a person having in his/her possession (holding) money or property in which he/she has no interest, right or title, awaiting the outcome of a dispute between two or more claimants to the money or property.  groups. Performance factors included those related to staff, collections, facilities, speed of services, use policies, and other areas; the researchers asked users which they thought were important and narrowed ninety-nine possibilities to a core group of twenty. The differences in rankings among the six groups reveal different models of organizational effectiveness; in the second stage of the study they will apply the core measures in those actual libraries.

McDonald and Micikas (1994) use the methodology established by Cameron (1978) and the taxonomies outlined in Du Mont and Du Mont (1979) to develop an integrative multidimensional approach that looks at inputs, processes, and outputs from the levels of the individual, the subunit sub·u·nit  
A subdivision of a larger unit.

Noun 1. subunit - a monetary unit that is valued at a fraction (usually one hundredth) of the basic monetary unit
fractional monetary unit
, and the whole organization. Their resulting model groups sixteen dimensions of effectiveness (for example, collection adequacy, staff size, college support, staff development, use of the collections) in four major domains (resources, services, library/ stakeholder interaction, and access). From individual measures for each dimension, a score is derived for each major domain. Cluster analysis Cluster analysis

A statistical technique that identifies clusters of stocks whose returns are highly correlated within each cluster and relatively uncorrelated across clusters. Cluster analysis has identified groupings such as growth, cyclical, stable, and energy stocks.
 showed that libraries vary in their effectiveness across the domains and group together into certain patterns of effectiveness (McDonald & Micikas, 1994, p. 74). Many of the questionnaire items still require a subjective interpretation, and there are no single ratings or patterns that can be held up as ideal, but the model presents criteria and measures with the potential for wide applicability and comparability.

Academic Libraries and Higher Education Effectiveness

The academic library is not a static free-standing unit. Ultimately, its quality must be judged by the quality of outcomes of the institution, however those are defined. In a more immediate sense, library success is realistically confirmed by feedback and support from stakeholders (faculty, administration, students, alumnae), and validation by accreditation and other external bodies. It seems a long way from the concrete measurement of library process and service data to this larger view of library impact and educational outcomes. Have the associations, institutions, and leaders within higher education viewed the library as a key component of these outcomes? Librarians have struggled to have their issues acknowledged by scholars, administrators, and policymakers, yet the only way to guarantee understanding is also to do the reverse - to use measures of library performance and effectiveness to demonstrate the success of processes and goals within higher education itself.

In skimming Skimming

An electronic method of capturing a victim's personal information used by identity thieves. The skimmer is a small device that scans a credit card and stores the information contained in the magnetic strip.
 the vast literature on higher education effectiveness, there are two things for librarians to note: (1) what current models and criteria are being promoted, and (2) in what ways is the library mentioned, if at all? There is no clear consensus on defining academic success, but almost all writers agree that higher education too must focus on definable outputs and outcomes as measured, for example, by indicators of job success, completion of advanced degrees, research productivity, student test scores, satisfaction surveys, and the like. Evaluative models, such as the continuing influential work of Cameron (1978), pursue a multidimensional model of organizational effectiveness that would accommodate different dimensions of satisfaction, performance, and resources, moving away from a one-size-fits-all ranking system. Cameron's work is now being fruitfully fruit·ful  
a. Producing fruit.

b. Conducive to productivity; causing to bear in abundance: fruitful soil.

 applied by library researchers (McDonald & Micikas, 1994), but his original dimensions did not address specific services pertaining per·tain  
intr.v. per·tained, per·tain·ing, per·tains
1. To have reference; relate: evidence that pertains to the accident.

 to libraries.

In recent months in the letters and opinion columns of the Chronicle of Higher Education, this old question was revived by Rothkopf (1995), with responses from Barrett (1995), Lindahl (1995), and others. Sparked by reports about the ways that colleges and universities manipulate their statistics in order to achieve better rankings in such influential lists as that published by U.S. News and World Report, the discussion quickly moved from admonishments about how to ensure compliance with data definitions to the underlying problem of finding better ways to compare the quality of colleges. The concerns and even the terms of rhetoric mirror many of those expressed by academic librarians, who will not take heart from the basic pessimism pessimism, philosophical opinion or doctrine that evil predominates over good; the opposite of optimism. Systematic forms of pessimism may be found in philosophy and religion.  expressed by Barrett about the validity of any current criteria. Cameron (1978) underscored the difficulty in establishing measurable criteria for success and observed that one of the reasons for the lack of progress in studies of organizational effectiveness is the tendency of researchers to do a fine-grained analysis of causes but a coarse-grained analysis of effects (p. 625).

Higher education, too, has been bitten bit·ten  
A past participle of bite.


the past participle of bite
 by the TQM bug. Library online databases list easily fifty or a hundred monographs with correspondingly larger numbers of articles on TQM from every conceivable con·ceive  
v. con·ceived, con·ceiv·ing, con·ceives
1. To become pregnant with (offspring).

 angle. Bogue and Saunders Saun´ders

n. 1. See Sandress.
 (1992) enumerate To count or list one by one. For example, an enumerated data type defines a list of all possible values for a variable, and no other value can then be placed into it. See device enumeration and ENUM.  six "tests of quality" for colleges and universities, including accreditation, rankings, follow-up surveys, licensure licensure
, academic program reviews, and outcomes as evidenced by student test scores. None is exactly a revolutionary concept, but this is useful insofar in·so·far  
To such an extent.

Adv. 1. insofar - to the degree or extent that; "insofar as it can be ascertained, the horse lung is comparable to that of man"; "so far as it is reasonably practical he should practice
 as it opens up specific possibilities for strengthening the scattered Scattered

Used for listed equity securities. Unconcentrated buy or sell interest.
 efforts libraries have already made within these six methods. In anthologies by Teeter and Lozier (1993) and Sherr and Teeter (1991), there is more focus on implementing TQM processes as part of campus administrative culture. Most of the works in this field include surprisingly little discussion of applying TQM to classroom teaching, faculty departments, or research support facilities.

A special case in recent higher education measurement projects is the benchmarking survey coordinated by the National Association of College and University Business Officers (1992) gathering data on forty administrative areas including the library. By most academic librarians' judgments, one would not term these new measures; the variables and resulting ratios are similar to those published by ARL or by Minter (1993a, 1993b). The risk is that these are taken very much out of context, and that they paint a reductionist re·duc·tion·ism  
An attempt or tendency to explain a complex set of facts, entities, phenomena, or structures by another, simpler set: "For the last 400 years science has advanced by reductionism ...
 and oversimplified picture of a complex organization. The introduction to the survey implies that comparing ratios of cost efficiency automatically identifies the institutions with the best practices, and it conflates definitions of performance, outputs, and quality. Claims that "benchmarking can potentially move the industry ahead at a pace more rapid than that of TQM alone" (p. 18) seem to elevate el·e·vate  
tr.v. ele·vat·ed, ele·vat·ing, ele·vates
1. To move (something) to a higher place or position from a lower one; lift.

2. To increase the amplitude, intensity, or volume of.

 the measurement process above the crucial conceptual and management frameworks outlined earlier. This survey cannot be ignored given the powerful role played by business offices and institutional research tools on campus. The appearance of these flawed flaw 1  
1. An imperfection, often concealed, that impairs soundness: a flaw in the crystal that caused it to shatter. See Synonyms at blemish.

, yet influential, documents is another motivation to press ahead with developing improved library instruments that can be segmented to fit into larger models.

Few analyses of educational evaluation Educational evaluation is the evaluation process of characterizing and appraising some aspect/s of an educational process.

There are two common purposes in educational evaluation which are, at times, in conflict with one another.
 and assessment mention the library. It may be one of die NACUBO sections, or occasionally the topic of an essay or case study in one of the TQM works, or one of the factors used by the National Research Council in its ratings of graduate programs. But in each of these it is incorporated as background, not targeted as a programmatic pro·gram·mat·ic  
1. Of, relating to, or having a program.

2. Following an overall plan or schedule: a step-by-step, programmatic approach to problem solving.

 center or linked to institutional performance overall. In part, this is because librarians themselves have not come up with a handy measurement to offer upon request. Yerbury (1992) is one of the rare administrators who has stated clear expectations for library performance and how it should be assessed. In the area of accreditation, however, there has always been extensive consideration of the library, and there is growing interest in developing better ways to assess its changing role.

Accreditation documents vary with each regional association, but in general they all attempt to define the library's role in support of academic programs. Adams (1992), Sacks and Whildin (1993), Garten (1994), and Williams (1993) have written excellent treatments of this topic, including numerous examples of measures and models that fit current practices in accreditation. Williams reports on one or two alternative measures of library effectiveness to link performance to institutional outcomes, and asserts the broader evolving perspective that a library be evaluated on the degree to which it takes responsibility for the support of all of the institution's programs, wherever offered and in whatever format. Wolff (1995) criticizes the resource-input bias of most accrediting standards and proposes organizing principles around which to focus the mission of the university: resources, research, students, and learning. For each he lists possible indicators of institutional and library quality. Coleman and Jarred (1994) demonstrate the prominent, yet ambiguous, role of the ACRL standards in accreditation review; while acknowledging the need for output and performance, the authors still see a complementary and continuing role for input criteria.

Troutt (1979) rather carefully debunks the assumption implicit in Adj. 1. implicit in - in the nature of something though not readily apparent; "shortcomings inherent in our approach"; "an underlying meaning"
underlying, inherent
 many accreditation documents that certain inputs and resources assure quality. His research on the correlates of educational quality and college impact specifically mentions the library as one of five major accreditation criteria: "Available research finds no relationship between differences in library resources and student achievement" (pp. 207-08). Williams (1994) and Wolff (1994) go on to urge developing more measures that demonstrate library impact on educational outcomes. Wolff notes that the library studies prepared for accrediting teams lack evidence of how the library is part of the institutional mission, for example, usage data broken down by discipline, evaluation of bibliographic instruction programs, role of the library in curricular development, and relationship of the library to campus information systems development. Sacks and Whildin (1993) also recommend an array of practical ways to demonstrate impact - for example, performance on library tests, analysis of term papers, detailed comparisons of the collection with syllabi syl·la·bi  
A plural of syllabus.
, interlibrary loan Interlibrary loan (abbreviated ILL, and sometimes called interloan, document delivery, or document supply etc.) is a service whereby a user of one library can borrow books, videos, DVDs, sound recordings, microfilms, or receive photocopies of  and circulation statistics, and so forth (pp. 54-55).

The barrier to more widespread adoption of these measures is that so far they are only defined in a sporadic sporadic /spo·rad·ic/ (spo-rad´ic) occurring singly; widely scattered; not epidemic or endemic.

spo·rad·ic or spo·rad·i·cal
1. Occurring at irregular intervals.

 and local context. They have not been adopted by national organizations, nor have they even been refined and boiled boiled  
adj. Slang
Intoxicated; drunk.

Adj. 1. boiled - cooked in hot water
poached, stewed

cooked - having been prepared for eating by the application of heat
 down into a few definable and replicable ratios that libraries can incorporate into regular routine. Library researchers have focused heavily on evaluating processes and service performance, with only a small number helping the profession collaborate with educational researchers to study outcomes and impacts. Mech (1990) itemizes a series of skills and competencies for students in general and then considers objectives and assessment strategies for library and information literacy Several conceptions and definitions of information literacy have become prevalent. For example, one conception defines information literacy in terms of a set of competencies that an informed citizen of an information society ought to possess to participate intelligently and . Powell (1988, 1992) summarizes earlier work on performance measures and moves directly into a study of possible methodologies for user satisfaction and impact evaluation, concluding that a lot more research is needed. The most interesting new models are those noted earlier being developed by Cullen and Calvert (1995) based on stakeholder perceptions of effectiveness, also being applied by Crawford (1995) and by McDonald and Micikas (1994). It is worth observing that both of these are derived from earlier research in evaluation (e.g., several works by Van House, Childers, and Cameron); it simply takes many years and many studies and many models to arrive gradually at new assessment frameworks.

One form of impact is almost neglected in evaluation models, yet it drives the management challenges of our greatest libraries. How do librarians and educators assess effectiveness if the goal is to ensure the availability of information resources (1) The data and information assets of an organization, department or unit. See data administration.

(2) Another name for the Information Systems (IS) or Information Technology (IT) department. See IT.
 for future users? That is the quintessential quin·tes·sen·tial  
Of, relating to, or having the nature of a quintessence; being the most typical: "Liszt was the quintessential romantic" Musical Heritage Review.
 function of the large research library - to acquire, describe, and preserve resources that are within the broad interests of the institution but for which the acquisition may not depend solely on the needs of the current users. Are universities willing to describe their mission in such a way that the measurable criteria would not be phrased in terms of the students and faculty on campus at a given moment? This is a provocative challenge and, despite lip service lip service
Verbal expression of agreement or allegiance, unsupported by real conviction or action; hypocritical respect:
 to the need for nationally focused research libraries committed to a long-term good which is broader than their own short-term program, there are few statements of effectiveness that show how this might be articulated within frameworks that rely on the efficient satisfaction of current constituencies. Recent prognostications on the future of the research library - e.g., Cummings (1992), Dougherty and Dougherty (1993), and Stevens (1993) - do not face this issue straight on. It is implied that mere volume counts can no longer be used, of course, but how the growth of research resources and access mechanisms will be documented is not discussed in future-oriented terms. These articles do make clear that the academic research library is still first and foremost an academic library; that implies the long-term research role will still have to be cast in terms of outcomes for higher education. These may differ from the outcomes by which one might assess the impact of government or public research libraries.

Directions for Future Research and Policy

There is no lack of advice and guidance on how to define quality and effectiveness in an immediate sense, and on how to begin establishing an evaluative framework relevant to the context of an individual academic library. It is still difficult, however, for library administrators to find readily workable ways to use existing data and to develop new information as services and situations change. Internet discussion lists that cater to library directors are frequently the site of urgent messages among colleagues to the effect of, "Help! My provost has asked me to find out how many other libraries have (or do) such-and-such and how well it works (or what it costs)."

It is frequently said that new measures need to be developed, for example, to be able to refine and measure "access." While this is doubtless the case, at the same time, more analyses are needed that simply improve the use of routine data, that show more ways to define and compare even conventional outputs, and that put forward processes for gathering and comparing data that might become foundations for better national assessment. Despite years of experience with statistics and surveys and management fads A management fad is a derisive term use to characterize a change in philosophy or operations that sweeps through businesses and institutions, and then disappears when enthusiasm for it wanes. , there is not a reliable and consistent way of assessing services, comparing alternative models of information delivery, and demonstrating comparative quality and effectiveness. What are some of the major drawbacks of the existing data, statistical series, benchmarks, and standards?

In general, consistency is lacking. Data are drawn from an immense patchwork of sources with many gaps where data are lacking for certain institutions, variables, and time periods. The comprehensive data sets are not disaggregated Broken up into parts.  adequately (e.g., by subject or country of materials, by institutional characteristics, branch library operations, or source of funds) to be able to make targeted analyses of resources and services. The local context is absent or skewed skewed

curve of a usually unimodal distribution with one tail drawn out more than the other and the median will lie above or below the mean.

skewed Epidemiology adjective Referring to an asymmetrical distribution of a population or of data
 from easily available data; it requires a great deal of redundant effort, however, to tailor measures to one's own users and mission and, once that is done, peer group analyses are harder. The over-reliance on national sources is problematic yet understandable, thus suggesting a strong motivation for collaborative efforts to develop broad new tools.

Available data do not support a functional approach to decision-making on service approaches - e.g., to explore access versus ownership or the trade-offs involved in performing a function in-house or by outsourcing (1) Contracting with outside consultants, software houses or service bureaus to perform systems analysis, programming and datacenter operations. Contrast with insourcing. See netsourcing, ASP, SSP and facilities management. . Library budgets and statistics are not categorized cat·e·go·rize  
tr.v. cat·e·go·rized, cat·e·go·riz·ing, cat·e·go·riz·es
To put into a category or categories; classify.

 to reflect shifting models of spending and of using value-added vendor services. Typical data are limited by department lines; current reporting mechanisms to which academic libraries contribute include variables for the library alone, and parallel measures do not always exist for other campus units. Budget and staffing data do not reflect campuswide shifts in provision of information support and in the way new information resources are acquired, documented, and stored; there is no way to make statements, for example, about the total information support to a given disciplinary area. Library managers and researchers need to collaborate with economists, information technologists, and others to devise cost models for new forms of service; fortunately, some projects of this sort have recently been initiated by the Council on Library Resources and the Coalition for Networked Information, and others.

New models, definitions, and measures are clearly needed in addition to a more refined way of interpreting customary data. Quality and performance of academic libraries is not addressed except in narrow contexts in individual studies. Measures are needed that work both locally and in broader comparisons; turnaround and productivity standards for tasks are few, and there are only abstruse studies of the value of information services See Information Systems.  to academic library users. To implement quality models in any meaningful way, managers will also need to know how to link individual performance to departmental performance.

One of the urgently needed tools is a replicable and straightfoward instrument to assess user satisfaction, not exactly a new concept but one for which there are no widely accepted models for academic libraries. The research underway by Cullen and Calvert may be invaluable to developing such a survey that could be used across a range of academic institutions. Another tool that might be derived from existing models would be a way to "score" progress on the ACRL standards - i.e., a series of scales that would show how close one is to having complete compliance with, or attainment of, a given standard. This would actually measure degrees of process (not an absolute quantity) and would fit well in TQM schemes.

It is already a problem for some library directors to report their annual expenditures and tallies TALLIES, evidence. The parts of a piece of wood out in two, which persons use to denote the quantity of goods supplied by one to the other. Poth. Obl. pt. 4, c. 1, art. 2, Sec. 7.  of user services when library and computing computing - computer  centers have been administratively merged. This trend will probably continue in many different variations. Libraries are increasingly linked to other information providers and educational services both on and off campus, such as computing centers, museums, research centers, media services, and student academic development programs. Models are needed that show the library contribution to, and the total shape of, campuswide (and consortia-wide in some cases) provision of information resources. Specifically, the challenges of the networked environment are significant: who are the users and the providers, what is the unit of service, what is a networked resource, what is the center for counting the costs, how do these forms of information mesh with more traditional academic resources? Even researchers pursuing this area actively, like Lopata and McClure (1995), are not sure whether they will be able to answer these questions, yet accreditation bodies will continue to press for ways to assess information support in this context.

Finally, more research is needed that will lead to agreed-upon measures of library- and information-related outcomes in higher education. As noted, such outcomes might include information literacy, success in graduate school, success in job seeking, faculty research productivity (as shown by grants and publications), and the library's success as a department in attracting gifts and external funding to the campus. With targeted research initiatives at institutions where subjective opinion indicates that the library and the university are vital and effective and the other tools suggested above, it might be possible gradually to establish progressive correlations among measures of inputs, processes, outputs, and performance or satisfaction. There might emerge several multidimensional models of effectiveness or an expanded index like that of the ARL, reflecting this more complete view of mission attainment.


Academic libraries will continue for some time to be obligated ob·li·gate  
tr.v. ob·li·gat·ed, ob·li·gat·ing, ob·li·gates
1. To bind, compel, or constrain by a social, legal, or moral tie. See Synonyms at force.

2. To cause to be grateful or indebted; oblige.
 to provide traditional acquisitions and public services Public services is a term usually used to mean services provided by government to its citizens, either directly (through the public sector) or by financing private provision of services. , yet they are already shifting their approaches in response to the explosion of networked information resources, third-party providers, self-publishing, and many other variations of the traditional mechanisms for producing, organizing, and delivering scholarly information. Some administrators have publicly questioned the need for conventional libraries, especially in new or technologically oriented o·ri·ent  
1. Orient The countries of Asia, especially of eastern Asia.

a. The luster characteristic of a pearl of high quality.

b. A pearl having exceptional luster.

 academic programs. The future vitality of libraries in academia will be dependent on whether they can dynamically and continually prove their value to the overall educational endeavor. This value must be documented at a level that transcends specific formats of information, locations of collections and locations of users, and that clearly links the investment in campuswide information resources to the effectiveness of particular disciplinary programs.

The measurement of quality will come back to the questions of who are the users, what are the inputs, what are the outputs, do we produce the outputs in a way that meets the needs of the users, and what do those outputs contribute to the productivity and accomplishments of those users? The questions are not new, but the object we are measuring has changed in many dimensions. Librarians may have to give up looking for Looking for

In the context of general equities, this describing a buy interest in which a dealer is asked to offer stock, often involving a capital commitment. Antithesis of in touch with.
 a single national instrument of performance or quality; however, we can move ahead by revisiting the fundamental questions in this new environment, by cleaning up existing practices, and by doing the large-scale coordinated research to identify truly pertinent indicators. It is not easy, but we are building on a long and still valuable base of theory and practice.


Adams, M. (1992). The role of academic libraries in teaching and learning. College & Research Libraries News, 53(7), 442-449. Allred, J. (1979). The measurement of library services: An appraisal of current problems and possibilities (Library Management Series, Vol. 1, No. 2). Bradford, England: MCB (Memory Control Block) An identifier (16 bytes) that DOS places in front of each block of memory it allocates.  Publications. Arnold, K. L. (1994). The manager's guide to ISO 9000. New York New York, state, United States
New York, Middle Atlantic state of the United States. It is bordered by Vermont, Massachusetts, Connecticut, and the Atlantic Ocean (E), New Jersey and Pennsylvania (S), Lakes Erie and Ontario and the Canadian province of
: Free Press. Association of College and Research Libraries (ACRL). (1989). Standards for university libraries: Evaluation of performance. College & Research Libraries News, 50(8), 679-691. Association of College and Research Libraries (ACRL). (1995). Standards for college libraries, 1995 edition. College & Research Libraries News, 56(4), 245-257. Baker, S. L., & Lancaster, F. W. (1991). The measurement and evaluation of library services (2d ed.). Arlington, VA: Information Resources Press. Barrett, W. O. (1995). [Letter to the editor]. Chronicle of Higher Education, 41(47), B3. Blagden, J. (1980). Do we really need libraries? An assessment of approaches to the Evaluation of the performance of libraries. New York: K. G. Saur. Bogue, E. G., & Saunders, R. L. (1992). The evidence for quality: Strengthening the tests of academic and administrative effectiveness. San Francisco San Francisco (săn frănsĭs`kō), city (1990 pop. 723,959), coextensive with San Francisco co., W Calif., on the tip of a peninsula between the Pacific Ocean and San Francisco Bay, which are connected by the strait known as the Golden , CA: Jossey-Bass. Cameron. K. (1978). Measuring organizational effectiveness in institutions of higher education. Administrative Science Quarterly Administrative Science Quarterly, founded in 1956, is one of the most eminent academic journals in the field of organizational studies. It is published by Cornell University.

People claimed to have been involved as founders include James D.
, 23(4), 604-632. Childers, T. (1989). Evaluative research in the library and information field. Library Trends, 38(2), 250-267. Clack, M. E. (1993). Organizational development and TQM: The Harvard College Harvard College is the undergraduate section and oldest school of Harvard University, founded in 1636 by the Massachusetts Legislature. The College is instructed by the Faculty of Arts and Sciences, which also instructs the Harvard Graduate School of Arts and Sciences.  Library's experience. In S. Jurow & S. B. Barnard (Eds.), Integrating total quality management in a library setting (pp. 29-43). New York: Haworth Press. (Simultaneously published in the Journal of Library Administration The Journal of Library Administration is a quarterly scholarly journal that provides information on how to manage a library. It is published by Haworth Information Press, and was launched in 1980. , 18[1-2].) Coleman, P., & Jarred, A. D. (1994). Regional association criteria and the Standards for College Libraries: The informal role of quantitative input measures for libraries in accreditation. Journal of Academic Librarianship, 20(5/6), 273-284. Crawford, J. C. (1995). The stakeholder approach to the construction of performance measures. Unpublished paper presented at the First Northumbria Conference on Performance Measurement in Libraries and Information Services, Northumberland, England, August 31 to September 4, 1995. Cullen, R., & Calvert, P.J. (In press). Stakeholder perceptions of academic library effectiveness. Journal of Academic Librarianship, 21. Cummings, A. M. (1992). University libraries and scholarly communication Scholarly Communication is an umbrella term used to describe the process of academics, scholars and researchers sharing and publishing their research findings so that they are available to the wider academic community (such as university academics) and beyond. : A study prepared for the Andrew W. Mellon Foundation The Andrew W. Mellon Foundation is a foundation endowed with wealth accumulated by the late Andrew W. Mellon. It is the product of the 1969 merger of the Avalon Foundation and the Old Dominion Foundation. . Washington, DC: Association of Research Libraries for the Andrew Mellon Foundation Mellon Foundation, officially the Andrew W. Mellon Foundation, philanthropic trust formed (1969) through the merger of the Avalon Foundation (est. 1940 by Ailsa Mellon Bruce) and the Old Dominion Foundation (est. 1941 by Paul Mellon). . Dougherty, R. M. (1972). The human side of library effectiveness. In A. Hershfield & M. Boone (Eds.), Approaches to measuring library effectiveness: A symposium (pp. 40-47). Syracuse, NY: School of Library Science, Syracuse University Syracuse University, main campus at Syracuse, N.Y.; coeducational; chartered 1870, opened 1871. Syracuse is noted for its research programs in government and industry; facilities include the Center for Science and Technology, the Newhouse Communications Center, and . Dougherty, R. M., & Dougherty, A. P. (1993). The academic library: A time of crisis, change and opportunity. Journal of Academic Librarianship, 18(6), 342-346. Du Mont, R. R., & Du Mont, P. F. (1979). Measuring library effectiveness: A review and an assessment. In M. H. Harris (Ed.), Advances in librarianship, (vol. 9, pp. 103-141). New York: Academic Press. Garten, E. D. (Ed.). (1994). The challenge and practice of academic accreditation: A sourcebook for library administrators. Westport, CT Greenwood Greenwood.

1 City (1990 pop. 26,265), Johnson co., central Ind.; settled 1822, inc. as a city 1960. A residential suburb of Indianapolis, Greenwood is in a retail shopping area. Manufactures include motor vehicle parts and metal products.
 Press. Goodall, D. L. (1988). Performance measurement: A historical perspective. Journal of Librarianship, 20(2), 128-144. Hernon, P., & McClure, C. R. (1990). Evaluation and library decision making. Norwood, NJ: Ablex. Jurow, S., & Barnard, S. B. (Eds.). (1993). Integrating total quality management in a library setting (simultaneously published in the Journal of Library Administration, 18[1-2]). New York: Haworth Press. Kania, A. M. (1988). Academic library standards and performance measures. College & Research Libraries, 49(1), 16-23. Kaser, D. (1982). Standards for college libraries. Library Trends, 31(1), 7-19. King, D., & Griffiths, J. M. (1991). Five years' research: New pieces of the information puzzle. Bulletin of the American Society for Information Science, 1 7(2), 11-15. Knightly, J. J. (1979). Overcoming the criterion problem in the evaluation of library performance. Special Libraries, 70(4), 173-178. Lewis, D. W. (1986). An organizational paradigm for effective academic libraries. College & Research Libraries, 4 7(4), 337-353. Landahl, C. W. (1995). [Letter to the editor]. Chronicle of Higher Education, 41(48), B3-B4. Lopata, C. L., & McClure, C. R. (1995). Assessing the impacts of internet networking on the academic institution (study abstract). Syracuse, NY: Syracuse University School of Information Studies. Lynch, B. P. (1982). University library standards. Library Trends, 31(1), 33-47. Martell, C. (1983a). The client-centered academic library: An organizational model. Westport, CT: Greenwood Press. Martell, C. (1983b). QWL QWL Quality of Work Life
QWL Quality of Working Life
QWL Quantum Well Laser
 strategies: Trust, subtlety sub·tle·ty  
n. pl. sub·tle·ties
1. The quality or state of being subtle.

2. Something subtle, especially a nicety of thought or a fine distinction.
, intimacy. Journal of Academic Librarianship, 9 (3), 164-165. Martell, C. (1985). QWL strategies: People are the castle, people are the walls, people are the moat. Journal of Academic Librarianship, 10(6), 350-354. Martell, C., & Tyson, J. (1983a). QWL strategies: Quality circles. Journal of Academic Librarianship, 9(5), 285-287. Martell, C., & Untawale, M. (1983b). Work enrichment enrichment Food industry The addition of vitamins or minerals to a food–eg, wheat, which may have been lost during processing. See White flour; Cf Whole grains.  for academic libraries. Journal of Academic Librarianship, 8(6), 339-343. McDonald, J. A., & Micikas, L. B. (1994). Academic libraries: The dimensions of their effectiveness. Westport, CT: Greenwood Press. Mech, T. F. (1990). Working with faculty in an outcomes-oriented curriculum: Observations from the library. In H. P. Hall & C. Byrd (Eds.), The Librarian in the university: Essays on membership in the academic community (pp. 72-91). Metuchen, NJ: Scarecrow Scarecrow

goes to Wizard of Oz to get brains. [Am. Lit.: The Wonderful Wizard of Oz]

See : Ignorance


can’t live up to his name. [Am. Lit.: The Wonderful Wizard of Oz; Am.
 Press. Meyer, R. (1995). Focusing library vision on educational outcomes. College & Research Libraries News, 56(5), 335-337. Millson-Martula, C., & Menon, V. (1995). Customer expectations: Concepts and reality for academic library services. College & Research Libraries, 56(1), 33-47. Minter, J. (Ed.). (1993a). Management ratios for colleges and universities. Boulder, CO: John Minter Associates. Minter, J. (Ed.). (1993b). Statistical norms for college and university libraries. Boulder, CO: John Minter Associates. National Association of College and University Business Officers & Coopers and Lybrand. (1992). Benchmark 91: Operational benchmarking for quality improvement and cost management in higher education. Washington, DC: NACUBO. Orr, R. H. (1973). Progress in documentation: Measuring the goodness of library services: A general framework for considering quantitative measures. Journal of Documentation, 29(3), 315-332. Pirsig, R. M. (1974). Zen and the art of motorcycle maintenance: An inquiry into values. New York: Morrow mor·row  
1. The following day: resolved to set out on the morrow.

2. The time immediately subsequent to a particular event.

3. Archaic The morning.
. Powell, R. R. (1988). The relationship of library user studies to performance measures: A review of the literature (Occasional Papers, no. 181). Urbana-Champaign, IL: University of Illinois University of Illinois may refer to:
  • University of Illinois at Urbana-Champaign (flagship campus)
  • University of Illinois at Chicago
  • University of Illinois at Springfield
  • University of Illinois system
It can also refer to:
, Graduate School of Library and Information Science A School of Library and Information Science (SLIS) is a university-based institution that provides a Master's degree or other advanced degrees associated with Library science, Information Science, or a combination of the two. . Powell, R. R. (1992). Impact assessment of university libraries: A consideration of issues and research methodologies. Library and Information Science Research, 14(3), 245-257. Riggs, D. E. (1992a). Strategic quality management in libraries. In I. P. Godden (Ed.), Advances in Librarianship, (vol. 16, pp. 93-105). San Diego San Diego (săn dēā`gō), city (1990 pop. 1,110,549), seat of San Diego co., S Calif., on San Diego Bay; inc. 1850. San Diego includes the unincorporated communities of La Jolla and Spring Valley. Coronado is across the bay. , CA: Academic Press. Riggs, D. E. (1992b). TQM: Quality improvement in new clothes. College & Research Libraries, 53(6), 481-483. Riggs, D. E. (1993). Managing quality: TQM total quality management in libraries. Library Administration and Management, 7(2), 73-78. Rothkopf, A. J. (1995). Devising better ways to measure the quality of colleges and universities. Chronicle of Higher Education, 41(44), B3. Sacks, P. A., & Whildin, S. L. (1993). Preparing for accreditation: A handbook for academic librarians. Chicago, IL: American Library Association. Shapiro, B. J. (1991). Access and performance measures in research libraries in the 1990's. Journal of Library Administration, 15(3-4), 49-66. Shaughnessy, T. W. (1990). Future directions for the ARL statistics. In Technology and the future of scholarly exchange (Minutes of the 114th meeting, May 10-12,1989, Providence, Rhode Island

“Providence” redirects here. For other uses, see Providence (disambiguation).
Providence is the capital and the most populous city of the U.S.
) (pp. 83-101). Washington, DC: Association of Research Libraries. Shaughnessy, T. W. (1993). Benchmarking, total quality management, and libraries. Library Administration and Management, 7(1), 7-12. Sherr, L. A., & Teeter, D. J. (1991). Total quality management in higher education (New Directions for Institutional Research, No. 71). San Francisco, CA: Jossey-Bass. Siggins, J. A., & Sullivan, M. (Eds.). (1993). Quality improvement programs in ARL libraries (SPEC (1) See specs and specification.

(2) (SPEC) (Standard Performance Evaluation Corporation, Warrenton, VA, An organization founded in 1988 to establish standard benchmarks for computers.
 Kit No. 196). Washington, DC: Association of Research Libraries, Office of Management Services. St. Clair, G. (1993). Customer service in the information environment. London, England: Bowker-Saur. Stephen, R., & Buschman, J. (1993). The "compleat" library organization: A case study of the academic library. Library Administration and Management, 7(2), 79-88. Stevens, N. D. (1993). Research libraries: Past, present, and future. In I. P. Godden (Ed.), Advances in Librarianship (vol. 17, pp. 79-109). San Diego, CA: Academic Press. Sweeney, R. T (1994). Leadership in the post-hierarchical library. Library Trends, 43(1), 62-94. Taylor, R. S. (1972). Measuring the immeasurable: Or can we get there from here? In A. Hershfield & M. Boone (Eds.), Approaches to measuring library effectiveness: A symposium (pp. 31-39). Syracuse, NY: Syracuse University School of Library Science. Teeter, D. J., & Lozier, G. G. (1993). Pursuit of quality in higher education: Case studies in total quality management (New Directions in Institutional Research, No. 78). San Francisco, CA: Jossey-Bass. Troutt, W. E. (1979). Regional accreditation evaluative criteria and quality assurance. Journal of Higher Education, 50(2),199-210. Van House, N. A. (1989). Output measures in libraries. Library Trends, 38(2), 268-279. Van House, N. A.; Weil, B. T.; & McClure, C. R. (1990). Measuring academic library performance: A practical approach. Chicago, IL: American Library Association. Whitehall, T. (1992). Quality in library and information service: A review. Library Management, 13(5), 26-35. Williams, D. E. (1993). Accreditation and the academic library. Library Administration and Management, 7(1), 31-37. Williams, D. E. (1994). Challenges to accreditation from the new academic library environment. In E. D. Garten, The challenge and practice of academic accreditation: A sourcebook for library administrators (pp. 23-31). Westport, CT: Greenwood Press. Wolff, R. A. (1994). Rethinking library self-studies and accreditation visits. In E. D. Garten (Ed.), The challenge and practice of academic accreditation: A sourcebook for library administrators (pp. 125-137). Westport, CT: Greenwood Press. Wolff, R. A. (1995). Using the accreditation process to transform the mission of the library. In Information technology and the university (New Directions in Higher Education Series No. 90). San Francisco, CA: Jossey-Bass. Yerbury, D. (1992). The viewpoint of a university manager. In J. Thawley (Ed.), The value of library and information services (Papers presented at a seminar held in Melbourne, September 19, 1990) (pp. 13-23). East Melbourne, Australia: CSIRO CSIRO Commonwealth Scientific & Industrial Research Organization (Australia) , Information Services Branch, Library Network Services.
COPYRIGHT 1996 University of Illinois at Urbana-Champaign
No portion of this article can be reproduced without the express written permission from the copyright holder.
Copyright 1996, Gale Group. All rights reserved. Gale Group is a Thomson Corporation Company.

 Reader Opinion




Article Details
Printer friendly Cite/link Email Feedback
Title Annotation:Perspectives on Quality in Libraries
Author:Pritchard, Sarah M.
Publication:Library Trends
Date:Jan 1, 1996
Previous Article:On becoming essential: An agenda for quality in twenty-first century public libraries.
Next Article:Where is the Xerox Corporation of the LIS sector?

Related Articles
Library user education: examining its past, projecting its future.
Quality in community college libraries.
The quality movement in Australian university libraries.
Services to Remote Users: Marketing the Library's Role.
Electronic Resources and Academic Libraries, 1980-2000: A Historical Perspective.
Assessing User Needs, Satisfaction, and Library Performance at the University of Washington Libraries.
Usage of Academic Libraries: The Role of Service Quality, Resources, and User Characteristics.
Perspectives on User Satisfaction Surveys.
Innovative United Kingdom Approaches to Measuring Service Quality.
International Variations in Measuring Customer Expectations.

Terms of use | Copyright © 2014 Farlex, Inc. | Feedback | For webmasters