Accounting for outcomes: demonstrating the impact of public libraries.
At a time when so much emphasis is being placed on quality, standards, and accountability, it should not come as a surprise that the academic world is spending time devising new ways of assessing the value and impact of organisations. At the same time, those responsible for managing and marketing library services are having to demonstrate their value to the public, policy makers and other stakeholders. However serious difficulties can arise for researchers and practitioners alike if they try to deal with these issues using inappropriate tools. For example tools used to demonstrate outputs may not be the same as those used to demonstrate outcomes.
For a number of years we had been considering social audit techniques as a way of evaluating library services. We were influenced by the approach to social auditing developed by the New Economics Foundation, and based on Zadek's idea that the stakeholders in an organisation have the right to be heard. This, says David Boyle (1) in his splendid book The tyranny of numbers, `gives social audits a kind of objectivity which simple market research doesn't have'. We discussed these ideas with a number of practitioners and thanks to the support of what was then the British Library Research and Innovation Centre, we were able to undertake social audits of library services in Somerset, a rural county in the south, and Newcastle, a large city in the north, of England. In doing this the aims were to determine how far ideas about libraries are reflected in practice, and to ascertain the degree to which social aspirations are translated into positive achievements. The intention was to provide a framework to enable professionals and policy makers to come to an informed judgement about the value and impact of public library organisations.
Value and impact studies are much more than mere measurement. Statistics are just a small part of the reality of the library, and any meaningful demonstration of its value has to go beyond simplistic quantification. Even the quality guru Deming, who is often associated with the use, if not over use, of statistics, has admitted that the most important things in life cannot be measured. How does one measure love, kindness, and generosity? The measures that are set out in the managerialist literature are often substitutes for qualities that are intangible or indirect.
Colleagues in academic libraries have observed that `relatively little attention has been paid to qualitative measures, or to output measures, but indicators which fail to take such factors into account will be inadequate and misleading'. (2) At the same time commentators on the public sector have argued that `performance can seldom be expressed in a meaningful way by quantitative data only. To a great extent, analysis of performance has to be based on qualitative descriptions and statements'. (3) Thus we need to use sociological and psychological research skills to demonstrate the value of library services in terms of their impact on individuals, and society. In addition our approach should be concerned with soft as well as hard data. The management expert Henry Mintzberg (3) reminds us that `hard information is often limited in scope, lacking in richness and often failing to encompass important noneconomic and nonquantitative factors' Managers, and certainly those involved in marketing, also need to know about `The expression on a customer's face, the mood in the factory, the tone of voice of a government official'. As Stewart and Ranson (4) have observed `Performance monitoring in the public domain is not merely concerned with effectiveness in achieving stated values, but with unexpected impact, and of values denied'.
The role of the researcher
This paper is concerned with outcomes, and it is presented from the point of view of a researcher. It needs to be made clear that the role of a researcher is different from that of somebody involved in marketing or public relations, and it is important that we do not transgress the boundaries between the different activities. That having been said, the activities can be complementary and the results of such studies can be used to promote and persuade. In the words of the American journal Marketing library services which reviewed our work
... if libraries ... carry out a social audit then ... they could well end up with the kind of images that marketing people could use to communicate ideas to fund holders, members of the public, the media and so on ... It's a very powerful technique (5)
As a result of the initial social audit of libraries in Newcastle and Somerset (6) we accumulated a large amount of rich evidence about the role of the public library and its impact on communities. Evidence that can, and has been, used to demonstrate the value of the public library.
The social role of the library
The intangible but significant role of the library in supporting the social cohesion of communities was a recurrent theme in both of the studies. It emerged in the urban inner city communities of Newcastle, and in the physically isolated rural communities in Somerset, served either by small static libraries or by mobile services. In both circumstances it appears that the library can help people overcome social isolation and individual loneliness.
In Newcastle the symbolic value of the library was seen in all the communities visited. The provision, and indeed the withdrawal, of library services was perceived to convey messages about the local authority's attitude to what one respondent called `communities under stress'. The same was true in rural Somerset where respondents observed that
[Closing the library] would take the heart out of the community. We have few enough facilities as it is ... there just wouldn't be anything for the children to do here if the library closed.
However the data suggest differences between the impact of the library in different areas. It appears that these are related to factors such as the levels of deprivation and the availability of other resources. It may, for example, be significant that in the most deprived ward the library was seen as the heart of the community, at least by library users.
There was evidence that this social dimension of library use also encourages a sense of community ownership of the library service. In both Newcastle and Somerset the public library was considered to be a warm and welcoming local institution. A similar situation was found by the Comedia research team in its Cleveland study where
Library staff had created a warm, almost domestic ambience, a `home from home,' and this made the library appear to be symbolically owned by--and belonging to--the community, unlike other forms of public provision such as job centres, youth clubs ... and community centres on the same estates. (7)
Our social audit study suggests that this is also true of rural areas. In Somerset the library was perceived to be the community notice board, and a source of community pride. One respondent observed
... just recently I walked past a lady, I suppose she must have been talking to somebody who didn't live in Taunton, and I heard her say, `have you seen our new library? You must go and see our new library! Elected member
There were also some very specific indications of ownership in Newcastle. A good example of this was a library which was free from vandalism, although the adjacent school suffered quite badly in this respect.
The educational role of the library
There was also strong evidence for the value of the local library's educational role. Our data mirror other research in the UK and support the view that it is not only the well resourced central and district libraries that have a key educational role. However until recently the library's educational role has tended to be been in terms of its value to the individual. This can be seen in the special provision being made for independent learners, for example through the government funded Open for Learning scheme, and in the professional debate over the undervaluing of the library's role as a homework centre. What has been less well covered is the potential for the local library, through its educational role, to support community development.
There may be an unexplored relationship between the local library's role as a catalyst for personal development and its role in community development itself. In the words of one of the respondents
... the library has the potential to assist the whole process of community development in trying to help people who are disadvantaged, to make the most of what's available to them Elected member
The availability of local resources for personal development is one of the essential ingredients in the creation of a self confident community with the skills and capacity to exert influence on the political and economic environment around it.
The economic impact of the library
In the Somerset case study the library appeared to have had a discernible economic impact, for example in supporting local shopping centres that were coming under pressure from the shops in the large town of Taunton. This confirms the relationship between visits to local shops and library use found in some other research. (8) In Somerset the library was also seen to be supporting the local tourist infrastructure. The library's economic impact was more difficult to evaluate in Newcastle. This is mainly because, in the wards studied, the emphasis is on the social and community role of the library. However local politicians and library staff did refer to the value of the business information services.
Reading and literacy
The role of the library in promoting reading emerged particularly strongly in Newcastle, where elected members and community coordinators were concerned about low levels of literacy and numeracy in the city. The personal experiences recounted by parents and carers attending focus groups suggested that libraries help develop the reading ability of young children, including English language skills in children of parents' whose first language was not English.
Every Saturday and Friday I have taken [my daughter] to the library and read some books there for her ... and I borrow a lot of tapes and a video to get her watching and listening ... I think without the libraries my daughter's English can't be improved so fast Focus group
The data from the interviews and focus groups in Newcastle suggested that the library could be an appropriate base, providing a nonstigmatised environment for specific initiatives on literacy.
Developing community confidence
As we began analysing the data, it became clear that once services are taken up, libraries can be important in developing confidence in individuals and local communities. Evidence for this is provided by a focus group respondent who said
I know the library helped me a lot when I went to college ... I never went to school when I was a kid, but I went to college in 1980 and I just learned a hell of a lot at the library. At one time I wouldn't talk to you like this because I couldn't, but with using the library as much as ... I did it just brought me out you know and that's a good thing as well.
The development of the self esteem of individuals can lead to greater community confidence and there are, of course, links here with the points already made about ownership, and the perception of the local library as a positive, warm and welcoming institution, and its contribution to social cohesion.
Public libraries are seen as community landmarks that reinforce community identity. In particular that local studies services can play a significant part in sustaining community identity. In Newcastle this record of community life was especially important given the decline in the industries that have traditionally help shape the local regional identity. For many people public libraries are the heart of the community. Our respondents, including local politicians saw the local library as having a symbolic value that was regarded by nonusers as well as users. One told us
I think that people see [the library] as a really important local landmark whether they use it or not, and I think if it was withdrawn it would have a terrible effect on people's morale ... The impact of it is that it's ... the only thing that people can point to and say that's the ... Council which isn't a school ... and in that sense I think it's an important presence of the Council. Elected member Newcastle
It has been argued that `One test of a democracy is whether it grants equal access to the tools that make knowledge possible' (9) The need to ensure equity in the distribution of services is one of the factors that distinguish public sector organisations from those in the commercial world. In such circumstances equity is a dimension of the service which needs to be properly demonstrated. The data from the social audit suggest that library services are usually administered fairly in terms of administrative justice.
In terms of equality between groups and communities, libraries were perceived as providing equity for most older people, people with disabilities and those from the ethnic minorities. The equity dimension was felt less by lone parents and unemployed young people. In addition some groups, especially lone parents, tended not to welcome being identified as a priority group. Work in the UK by Roach and Morrison (10) paints a less positive picture with regard to the minority ethnic communities. This does not necessarily mean that we, or they, are right or wrong. It may reflect the position in different authorities and different approaches to management.
The demonstration of equity is still in its infancy and there is a tendency to rely to a great extent on numbers. For example by counting the number of books in a foreign language, or the number of people from a particular community, or background, who use, or do not use a service. Such data are, of course, important. They are, if nothing else, an indicator of an authority's commitment. However such figures tend only to deal with inputs and outputs. Here, as elsewhere, there is a danger of only measuring what is measurable and missing what is important about the library service. In assessing equality, people's experience of using the service, and their perceptions of it, can be used to indicate outcomes. Such qualitative outcome indicators are often a more meaningful way of demonstrating a service and its achievements.
The extent to which the public library service fulfils its social objectives depends, in some degree, on how the service and the local authority are managed, and also on other factors outside the immediate control of library staff. Factors which have been identified as helping or hindering the attainment of social objectives include resources, marketing and awareness of the service, library rules and culture, structure and staff attitudes. Factors less in the control of staff are the location of the library and the fear of crime in the community.
We do not, and cannot, claim that the public library by itself is responsible for all the life changes that have been reported to us. There are, of course, many other agencies involved and in our research report we make a distinction between intermediate and final outcomes of library use. (11) However on the basis of the data from our original study, it is reasonable to claim that public libraries help individuals and communities `get started' and `keep going' on a wide range of activities. In addition, sometimes with the help of other agencies, public libraries help develop and maintain individual and community development. In short, libraries enrich the lives of many people. The social audit technique makes that enriching process visible.
The same can also be true when we borrow research methodologies from other fields. In a recent study on the value and impact of imaginative literature we combined the methodology developed in our previous social audit work, with the `uses and gratifications' techniques used by researchers in the mass media eg Brown, (12) McQuail, (13) and the reader response approach which is seen as integral to the understanding of literature. As McKearney (14) has stated the library profession has `not developed ways of demonstrating the impact of its work with readers [its] impact on individual lives and, on community life and on national priorities'. In other words librarians describe library activities but fail to demonstrate their impact in a way that will persuade politicians and policy makers.
Our research on public library book reading specifically sought to discover the kind of evidence that policy makers require if they, and their political colleagues, are to be convinced of the importance of reading and the value of the public provision of imaginative literature. Local politicians with responsibility for library services were asked
... when planning your local budget for the provision of services, how would you persuade your political colleagues of the importance of the library's provision of imaginative literature to your community?
Most respondents maintained that they did not need persuading. However it was acknowledged that `when someone suggested cutting the book fund ... it was very quickly knocked back. But that won't always be the case. It can't always continue and I can see a day will come when we will have to close libraries and it is a day I do not want to see.'
Indeed recent evidence (16) suggests that libraries have been closed and it was not so long ago that a British politician asked `what purpose does reading serve and why should it be provided free of charge to the user?' (17)
The politicians interviewed for the reading study placed a heavy reliance on quantitative data. Typically this was provided from a computer system.
[The] Quarterly operations report of the library service gives a snapshot of users and activities at any one particular moment in time. We can get an accurate picture of the proportion of the population who are users, the issue type, number of issues and what type of stock--we can really have our finger on the pulse. We need this to support our investment decisions. (18)
A minority made reference to qualitative techniques that provided feedback from the electorate. These included the Plus survey, other locally managed surveys, and their personal observation of the service
We look at the Plus surveys, comments and requests. Members use the libraries and can see for themselves. We like to see the library being innovative and employing varying forms to encourage the library user. In response to user surveys we have altered the loan period from 4 weeks to 3 weeks, that way they can borrow more books. (19)
However, that having been said, the evidence clearly suggests that qualitative data do not carry as much influence as quantitative data. Most politicians have not gone beyond book issues and still like what they regard as hard facts. By this they mean quantitative measures. At the same time, and confirming McKearney's observation, there is still an uncertainty amongst some professionals about how to demonstrate the value of the library's work with adult readers. One stock manager admitted
I don't know, we never seem to emphasise reading. Politicians are interested in IT. They like hard facts ... we also stress the library as a safe environment as it is a corporate objective--safety in the community. But we never focus on reading as such. At the launch of Bookstart, I spoke to politicians and parents on the value of getting children hooked on books from an early age and how it helps them in education. But we've never mentioned it at other times.
It has been recommended (20) that we should make arguments about the public library `political'. This idea was developed at the workshop organised as an integral part of our reading research. The delegates who attended suggested that it was imperative to capitalise on some current government strategies such as lifelong learning and social inclusion.
Links between the public provision of reading material and the government's concern for literacy were also thought to be important. For example some recent members of the government have mentioned the importance of reported links between low literacy levels and crime and other antisocial activity.
The Moser report (21) identified that literacy has an impact on a number of levels; those of the individual, the family, the community and society, and the economy. Participants reflected on the individual and social impact of the public library reading experience. Typically their responses included concerns about literacy, the role of fiction and the library's social responsibility.
As part of the research into public library book reading we asked library users and other stakeholders how they would explain the value of the public library and book reading to their local councillor. Their answers reflected the broad themes identified by Comedia in its work on the social impact of the arts. (22)
These topics, which are closely related to the findings of our social audit, are
* personal development
* social cohesion
* community empowerment
* local culture and identity
* imagination and creativity
* health and well being
Another Sheffield study The impact of library closures and reductions in opening hours (23) involved before and after studies of public library users in communities where closures and reductions were taking place. The data show that when a local library closes, up to a third of adults may be deprived of access to the public library. Worst affected are young children and elderly people. Parents and teachers told the researchers that the impact of library closure was devastating. Although some children were reading as much as before, their choice and quality of reading suffered dramatically.
People felt that only a local library helped them to feel a part of their own community and play an active part in it. It helped them to be part of a local information `network', reduced loneliness, and encouraged friendships.
The Sheffield strike research (24) provided further information on the value of the public library. In the summer of 1995 Sheffield Libraries closed for eight weeks. This gave researchers the chance to find out how people responded to the lack of a library service.
Over 500 library users were interviewed after the libraries reopened. Other local libraries and bookshops were surveyed to see whether people had transferred their use to them. A survey of telephone callers using the information services was also carded out.
The research showed that the social value of the local public library has been underestimated. People enjoy the experience of going to the public library, whether or not they need to borrow books or seek information. It appears to make a significant contribution to their quality of life. This seems especially true in communities with a high incidence of economic and educational deprivation. The local library can be an important resource for personal development, particularly when users have had a poor experience of formal education. The data also demonstrated the extent to which reading is perceived to be an essential and critical factor in the lives of library users. Many people cannot afford to buy enough books to replace those provided through the library service.
Perceptions play a significant part in value and impact studies, and research from other areas points to the dangers of making management, or indeed political, decisions on the basis of users' perceptions alone. Simply asking users is often not enough. Users know what they like, they sometimes know what information they need, but they often do not know what is possible for the library to provide. Some explanation of the different perceptions between librarians and users can be found in the literature on the psychology of prediction. For instance Slovic et al (25) and Tversky and Kahneman (26) suggest that people judge events as likely if instances of it are easy to imagine or recall.
What people can recall will depend on their knowledge and experience of the area. Library users by definition will have personal experience of using libraries. They will therefore have some relevant knowledge but as Stewart and Walsh (27) suggest, from a public service perspective, in a number of areas this knowledge will be incomplete.
Demonstrating value and impact
That having been said, qualitative assessments of outcomes are often a more meaningful way of demonstrating the value and impact of a service and its achievements. As we have seen, this more qualitative approach has been used in a number of recent studies of social impact. In Beyond book issues Matarasso (28) reviewed library based projects entered for the Library Association Community Initiative Award, and assessed the extent to which they have produced social benefits and sought to identify factors which lead to success.
The original Sheffield Social Audit study used a technique that had much in common with quality audits, as defined by Percy-Smith, (29) and implemented by a number of local authorities with regard to recreation, transport and information technology services. It has also been used by a variety of voluntary organisations, and companies such as Traidcraft and The Body Shop. To quote Gray, social accounting has experienced `a (long overdue) resurgence as academics ... look for new ways of providing accounts of organisational life'. (30)
The technique can be used to examine and demonstrate the success or failure of a portfolio of activities and services offered by a particular kind of library eg inner city, urban or rural branch libraries or those serving designated areas of poverty.
The importance of research
It is important that key professional staff and policy makers recognise the value of research as a contributor to, and demonstrator of, library performance. In the UK we have argued the need to establish a centre for the effective dissemination of public library research. We suggested that this should not be confined to work carried out in academic institutions and other research centres but also include the hidden research that is undertaken as part of the day to day management of libraries.
We have made some suggestions regarding the dissemination of research of interest to the library community. These include establishing a clearinghouse for the dissemination of reports and surveys prepared by different library organisations; developing a database of these activities and establishing a website which would keep practitioners, academics, policy makers and others up to date with research being carried in library authorities, in universities, and other organisations with an interest in the field. It is a pattern which you might wish to consider for Australia. *
The implementation of research findings involves more than simply applying a recipe. Research is often about innovation and all innovations require some organisational change if they are to be implemented effectively. For research to be implemented, someone needs to know about it, they must influence the organisation and persuade policy makers that change along the indicated lines will be valuable. `Thus whilst knowledge of the results of research may be important, it is rarely by itself sufficient to change practice, and other factors need to be taken into account'. (31) Senior library managers need to persuade and cajole others in their organisations to try out new things, and to invest in people and other resources to provide the training that changes will require. In other words, implementing research findings is a major organisational task and not something simply to be bolted on.
This paper has concentrated on the positive findings from our research but the approaches described also enable us to identify some of the matches, mismatches and differences between social objectives of local authorities, and the intermediate and final outcomes of the library services they provide. The process has also enabled us to identify positive differences between objectives and outcomes. That is to say, we have identified beneficial outcomes which may not have been previously identified as library, or even local authority, objectives. These matters have been discussed with the politicians and professionals responsible for the service and the masons for the matches, mismatches and differences analysed in terms of the various components of the audit. In particular that some management issues, especially staff attitudes, can help and/or hinder public libraries in their pursuit of social objectives. The point is that managers, armed with this information, can then take appropriate actions to maximise the matches and minimise the mismatches. They can take action `to accentuate the positive and eliminate the negative' aspects of the service.
The methods described can be used by staff in public libraries, and indeed other public and voluntary sector information organisations, to help demonstrate the impact of the services they provide, and to enable them to identify the reasons for their success or failure. Currently we are engaged in a research project to investigate the value and impact of cross-domain activities between library, museum, and archives services. An integral part of this project has been to train staff in social audit and other techniques to enable them to monitor and demonstrate the outcomes of their services.
Much of the work described in this contribution seeks to illuminate the activities of library users. As such it can help professionals and policy makers assess the impact of the services they provide, and enable them to identify the reasons for their success or failure. Such information should, of course, be of use to those who have the responsibility for marketing the library message. In addition such research can help managers guide and monitor the service, and improve the way outcomes of the service are reported to policy makers. It should enable stakeholders to make better judgements about the service, and affect organisational behaviour. In short, it has a practical worth. Not the least, it can provide professionals, and marketing experts with data to demonstrate the value of the public library.
* Editor's note: This role has been partly met for the past twenty years by Achlis, the Australian Clearing House for Library and Information Science in the University of South Australia Library. Achlis collects and abstracts for online access all Australian publications, articles, theses, conference papers, general and inhouse reports, research and surveys. Send your items to Achlis University of SA Library St Bernards Road Magill SA 5072 email firstname.lastname@example.org. Achlis maintained a register of in progress funded, and inlibrary, research and investigation for several years. Little data was received despite considerable effort
(1) Boyle, D The tyranny of numbers; why counting can't make us happy Harper Collins 2000
(2) Joint Funding Councils' Libraries Review Group report Libraries review group, Education Funding Council Bristol, HFCE. [Group chaired by Sir Brian Follett] 1993
(3) Mintzberg, H, Ahlstrand and Lampel, J Strategy safari London, Prentice Hall 1998
(4) Stewart, J and Ranson Management in the public domain Public money and management Spring Summer 1988 p13-19
(5) Lyon, J Measuring the unmeasurable value Marketing library services 12(8) 1998 p4-5
(6) Linley, R and Usherwood, B New measures for the new library. A social audit of public libraries Centre for the Public Library in the Information Society Department of Information Studies, The University of Sheffield 1998 [British Library Research & Innovation Centre Report 89]
(7) Greenhalgh, L The public library as a place Stroud, Comedia 1993 (The future of public library services: working paper 2)
(8) Sobczyk, G, Proctor, R and Usherwood, B What do people do when their public library service closes down? an investigation into the impact of the Sheffield Libraries strike British Library Research and Development Department 1996 (British Library R&D Report 6224)
(9) Editorial New York times 16 November 1998
(10) Roach, P and Morrison, M Public libraries ethnic diversity and citizenship Centre for Research in Ethnic Relations and Centre for Educational Development, Appraisal and Research. University of Warwick 1998 (British Library Research and Innovation Report 76)
(11) Linley and Usherwood op cit
(12) Brown, R Children and television Collier Macmillan 1976
(13) McQuail, D Mass communications theory: an introduction Sage 1994
(14) McKearney, M Spreading the word Public library journal 14 (4) 1999 p 106-107
(15) Linley and Usherwood op cit p3
(16) Proctor, R, Lee, H and Reilly Access to public libraries. The impact of opening hours reductions and closures 1986-1997 Centre for the Public Library in the Information Society (British Library Research & Innovation Centre Report 90)
(17) Sproat, I paper delivered to the Public Libraries Authorities conference Torquay UK 1993
(18) Linley and Usherwood op cit p9
(19) ibid p2
(20) Nicholson, V The political dimension in Coleman, P Libraries and the arts in action or inaction? Proceedings of the Sheffield conference November 1985 Sheffield City Libraries 1987 p90-95
(21) Moser, C A fresh start: improving literacy numeracy Sudbury, DfEE 1999
(22) Matarasso, F Beyond book issues: The social potential of library projects Comedia 1998 piv, v (BLRIC Report 87)
(23) Proctor, Lee and Reilly op cit
(24) Sobczyk, Proctor and Usherwood op cit
(25) Slovic, P et al Facts and fears: understanding perceived risk in Schwing, R and Albers, Weds Societal risk assessment. How safe is safe enough? New York, Plenum 1982
(26) Tversky, A and Kahneman, B Judgement under uncertainty: heuristic and biases Science 1982 p185
(27) Stewart, J and Walsh, K The search for quality Luton, Local Government Training Board 1989
(28) Matarasso op cit
(29) Percy-Smith, J Auditing social needs Policy and politics 20(1) 1992 p29-34
(30) Gray, R Social and environmental accounting research. Briefings from the GEC Programme GEC Programme Office, University of Sussex 1995
(31) Watt, I The dissemination of R & D information IFMH INform 7 (1) Spring 1996 p1-4
Arvidsson, R Performance evaluation in Kaufman, F X et al eds Guidance, control and evaluation in the public domain Berlin, de Gruyter 1986
Usherwood, B and Linley, R New library--new measures; a social audit of public libraries IFLA journal 25 (2) p90-96
Bob Usherwood is Professor of Librarianship at Sheffield University. Before joining Sheffield he was chief librarian in the London Borough of Lambeth. In 1978 he received the Senior Librarians Award which enabled him to investigate library public relations in the US and was invited back to America to attend the 1991 White House Conference on Library and Information Services. He was president of the Library Association in 1998 and obtained his doctorate as a result of research into the role of elected members in the operation of public library services. Address: Department of Information Studies The University of Sheffield, Sheffield S10 2TN UK tel 01114 2222635 fax 0114 2780300 email@example.com
|Printer friendly Cite/link Email Feedback|
|Publication:||Australasian Public Libraries and Information Services|
|Date:||Mar 1, 2002|
|Previous Article:||Children are our future ... right?|
|Next Article:||Solving the mystery: children's librarianship and how to nurture it.|