The Gulliver online database evaluation tool.The Gulliver consortium is a Victorian public library based group formed in 2001. It purchases electronic databases for members and selects them after evaluation on criteria of content, statistical and technical features and vendor viability. After months of evaluating databases using an Excel spreadsheet, an online version was developed to improve the administration of the tool and the transparency of the evaluation process. The online tool is also flexible and can easily grow to accommodate a larger number of databases. Additional features will be added to the tool in upgrades. Edited version of a paper presented at the Vala conference Melbourne February 2004 ********** In the late 1990s the Victorian public library community took the plunge into the new world of electronic databases, initially using seed funding Seed funding is investing capital to begin a new project, so that it has enough funds to sustain itself for a period of development until it reaches either a state where it is able to continue funding itself, or has created something in value so that it is worthy of future rounds from the Libraries Online project funded by Multimedia Victoria. A range of online information databases, selected by a group of inspired public librarians, was made available free to all Victorian public libraries. By the time this seed funding was expended ex·pend tr.v. ex·pend·ed, ex·pend·ing, ex·pends 1. To lay out; spend: expending tax revenues on government operations. See Synonyms at spend. 2. , the benefits of consortial purchasing arrangements for electronic databases were being recognised. Such arrangements were becoming increasingly common, internationally and within Australia. In 2000 there was no consortium covering the Victorian public library sector, so Gulliver was born. It began operating as a self funding public library consortium on 1 June 2001, with nearly every Victorian public library service committing to it. Tauranga District Libraries, New Zealand New Zealand (zē`lənd), island country (2005 est. pop. 4,035,000), 104,454 sq mi (270,534 sq km), in the S Pacific Ocean, over 1,000 mi (1,600 km) SE of Australia. The capital is Wellington; the largest city and leading port is Auckland. also joined. The Gulliver group is now a subcommittee sub·com·mit·tee n. A subordinate committee composed of members appointed from a main committee. subcommittee Noun of Viclink, the peak association for public libraries in Victoria. Gulliver grows Three databases from two vendors were chosen as the core products. These were Thomson Gale (Thomson Gale, a Thomson Learning business, Farmington Hills, MI, www.gale.com). A leading information publishing company for libraries, schools and businesses. Thomson Gale was formed in 1998 as a merger of Gale Research, Information Access Company and Primary Source Media, three Thomson Corporation's Infotrac Onefile database and the Health and Wellness Centre. Ebsco's ANZ ANZ Australia and New Zealand ANZ Australia and New Zealand Banking Group Limited ANZ Air New Zealand (NZ national airline) Reference Centre was chosen to complement these databases to provide access to Australian newspapers and other publications. Gulliver is managed by a voluntary steering committee steer·ing committee n. A committee that sets agendas and schedules of business, as for a legislative body or other assemblage. steering committee Noun which meets monthly. A business plan and a memorandum of understanding A Memorandum of Understanding (MoU) is a legal document describing a bilateral or multilateral agreement between parties. It expresses a convergence of will between the parties, indicating an intended common line of action and may not imply a legal commitment. were developed for the membership. Presentations are regularly given at conferences and Gulliver is becoming widely recognized as a highly successful and well organised consortium. Requests for further information about it were made by many libraries from various geographical areas and sectors, and it now has three New Zealand members. The concept of offering additional databases began to take shape, and Bruce was born. Bruce (Bibliographic bib·li·og·ra·phy n. pl. bib·li·og·ra·phies 1. A list of the works of a specific author or publisher. 2. a. Resources under a Consortia Environment) was developed in April 2002. The Bruce databases are offered in addition to the core databases, and are all optional, unlike the core databases which are automatically included with Gulliver membership. This approach is designed to enable libraries to be able to choose the extra databases they want, and a wide range is now available. Most of the Bruce databases are not restricted to Gulliver members. At present, as well as Gulliver members, South Australian, Australian Capital Territory Australian Capital Territory (1991 pop. 276,468), 939 sq mi (2,432 sq km), SE Australia, an enclave within New South Wales, containing Canberra, capital of Australia. It was called the Federal Capital Territory until 1938. and Tafe (Technical and Further Education) libraries subscribe. Evaluation: the core of Gulliver The selection of databases has been viewed by the Gulliver steering committee as crucial to integrity and consistency. From the outset, extreme care was taken in the evaluation process. Initially an Excel spreadsheet was developed to formally evaluate databases in four areas: content, statistics, vendor longterm viability and technical functionality, to give a final ranking or score. The overriding (programming) overriding - Redefining in a child class a method or function member defined in a parent class. Not to be confused with "overloading". criterion for selection of the databases is that they are web based Coming from a Web server. See Web application. with the aim of consisting of 70 per cent full text, a goal which remains elusive. Trials are conducted by a group of libraries that have indicated that they would be willing to trial and evaluate the databases. All databases are evaluated against the following four areas. The overall ranking and who is responsible for the evaluation, are Content * includes titles covered, interface and technical considerations for reference staff * weighting for content is 40 per cent * responsibility for evaluating content lies with the individual libraries carrying out trials on the databases Statistics * form of output and ease of retrieval and interpretation by libraries and whether this can be set by individual libraries * weighting for statistics is 10 per cent * responsibility for evaluating statistics lies with the Gulliver steering committee Vendor resources * resources provided by the vendor such as support and training, as well as value added Value Added The enhancement a company gives its product or service before offering the product to customers. Notes: This can either increase the products price or value. * services such as predefined searches, longterm viability of company and product development * weighting for vendor resources is 20 per cent * responsibility for evaluating vendor resources lies with the Gulliver steering committee Technical attributes * authentication (1) Verifying the integrity of a transmitted message. See message integrity, e-mail authentication and MAC. (2) Verifying the identity of a user logging into a network. , Z39.50 compliance, modification of databases/interface, outputs, speed, private links * weighting for technical attributes is 30 per cent * responsibility for evaluation of technical attributes lies with the Gulliver steering committee The spreadsheet In the beginning the Excel spreadsheet was adequate because the number of databases evaluated and the number of evaluators were small. As both grew, the limitations of this approach became apparent. In addition, the fact that the spreadsheet was the responsibility of the one member of the steering committee who handled the evaluation process, made the master version vulnerable to loss of data or data integrity resulting from the possibility of multiple versions of the spreadsheet existing at any one time. Updating of the spreadsheet was a laborious la·bo·ri·ous adj. 1. Marked by or requiring long, hard work: spent many laborious hours on the project. 2. Hard-working; industrious. task which only one person could do. The time taken for the evaluation process was quite lengthy and not transparent--other librarians were not able to see the results of the evaluation or the answers to specific questions, and had to accept the endorsement or non endorsement of a database virtually on faith. Clearly a different approach was required. Online database evaluation tool In 2002 a solution to the dilemma was found by a connection between one of the steering committee members with the Vicnet (a division of the State Library of Victoria) staff responsible for the Victorian Virtual Library website. This site was undergoing a redesign re·de·sign tr.v. re·de·signed, re·de·sign·ing, re·de·signs To make a revision in the appearance or function of. re , involving moving from a static page structure to a more dynamic structure and transferring the information on the site to storage in a structured database format which would enable more efficient searching. Establishing an online database evaluation tool for Gulliver, to make its evaluation process more robust and easier for evaluators to use, dovetailed nicely into this redevelopment. There already existed a strong link between Gulliver and the Victorian Virtual Library due to their parallel development and staff cooperation. The Victorian Virtual Library site maintained a central listing of links to databases for all Gulliver members, and a staff member from Vicnet sits on the Gulliver steering committee. Online database evaluation tool: specification It was determined that an online database evaluation tool would be hosted on a server within the State Library of Victoria, and a specification was developed. This allowed for the development of the tool which catered for the entering of data relating to relating to relate prep → concernant relating to relate prep → bezüglich +gen, mit Bezug auf +acc online information databases being evaluated, scoring of databases and automatic calculation of the score of a database. The online database evaluation tool was intended to have a number of audiences * general public, which could include public librarians, library managers and online database vendors * online database evaluators, who would enter their evaluations online * the administrator, who would oversee the entering, score selection and deletion deletion /de·le·tion/ (de-le´shun) in genetics, loss of genetic material from a chromosome. de·le·tion n. Loss, as from mutation, of one or more nucleotides from a chromosome. of online information databases A formal specification for the tool was developed, as follows. Purpose of tool * to display evaluation information for databases assessed by Gulliver members * to enable new evaluations for additional databases to be added by authenticated au·then·ti·cate tr.v. au·then·ti·cat·ed, au·then·ti·cat·ing, au·then·ti·cates To establish the authenticity of; prove genuine: a specialist who authenticated the antique samovar. administrators * to enable new sets of information to be added to the evaluation database by authenticated editors Features The tool must be * searchable * browsable by all users The tool must also allow restricted ie password access to an * administration section * editing section Tables required * online database table * content table * statistic statistic, n a value or number that describes a series of quantitative observations or measures; a value calculated from a sample. statistic a numerical value calculated from a number of observations in order to summarize them. evaluation table * vendor evaluation table * technical evaluation The study and investigations by a developing agency to determine the technical suitability of material, equipment, or a system for use in the Military Services. See also operational evaluation. table * total ranking table Unpassworded pages The following information must be viewable * name of database * vendor name * result for each aspect of evaluation * calculated result of evaluation (numerical result) * date evaluated * name of evaluating library and contact person * comments Administration pages The following features must be available * add, delete To remove an item of data from a file or to remove a file from the disk. See file wipe, trash and undelete. 1. (operating system) delete - (Or "erase") To make a file inaccessible. , edit databases * suspend function Editor's pages The following features must be available * add, delete, edit databases * suspend function * add new tables Scripts required The following scripts are required * add/edit/delete for each of the tables * scripts to calculate the statistics * visual aid for the calculations * search for the databases * format as HTML HTML in full HyperText Markup Language Markup language derived from SGML that is used to prepare hypertext documents. Relatively easy for nonprogrammers to master, HTML is the language used for documents on the World Wide Web. A quotation for the work was provided and accepted. Several meetings were held with the Vicnet programmer (1) A hardware device used to customize a programmable logic chip such as a PAL, GAL, EPROM, etc. See PROM programmer. (2) A person who designs the logic for and writes the lines of codes of a computer program. while the tool was being developed, to refine the layout, features and help text. The Gulliver evaluation database was written using the Perl programming language and was converted from the existing Excel spreadsheet. The Perl program was designed to increase efficiency by modularising certain aspects of the program, for example the automated process of working out ranking scores based on predefined formulas. The backend database which holds all the evaluation information is stored in a PostgreSQL relational database relational database Database in which all data are represented in tabular form. The description of a particular entity is provided by the set of its attribute values, stored as one row or record of the table, called a tuple. . The server on which the program is stored, a Debian Linux See Debian. , is hosted by Vicnet. Total time taken from inception to completion of the tool was less than six months. Online database evaluation tool: so what does it look like? The tool has a different interface, depending on whether you are the administrator (in other words Adv. 1. in other words - otherwise stated; "in other words, we are broke" put differently , you can do anything), an evaluator (who can edit a database and complete an evaluation online) or a viewer (who can look but not touch). Administrator and evaluator views have adding and editing links, while the public view is purely a listing of the databases evaluated and their scores. What is in an evaluation? The administrator adds vendors and data relating to the online information databases to be evaluated and assigns a subject to a database. Evaluators then select a database for evaluation and complete their evaluation online. The administrator reviews the evaluations and combines the findings into one master evaluation, which is then selected and appears as the final scored evaluation. The final score appears on the first screen of all views as a clear indication of the quality of a database. The evaluation can then be drilled down to view scores and answers to individual questions within particular sections of the evaluation. The content section is the single most important section of the evaluation. To engender en·gen·der v. en·gen·dered, en·gen·der·ing, en·gen·ders v.tr. 1. To bring into existence; give rise to: "Every cloud engenders not a storm" confidence, the evaluator and evaluating library is clearly stated so other libraries can be confident that the evaluation has been carded out by experienced colleagues. A scope statement enables a concise description of the breadth of the database and any specific limitations or other relevant observations. Online database evaluation tool: what does it tell you? The evaluation tool in all of its views provides complete information about every aspect of an evaluation. The answers to each question and the corresponding calculated score is shown for each question in each section. Online database evaluation tool: so why is it better? The change from a single, static, Excel spreadsheet as an evaluation tool for databases to the online version has many advantages * administration of the evaluation process is much easier, with evaluators being able to complete evaluations directly themselves and have the results calculated and made available immediately * data integrity is now ensured, with the evaluations all stored on a server in a secure environment with regular backups done * searching across subject areas, database names and vendors is now possible * the improved layout makes it easier to see individual answers and scores, as well as the overall ranking, both total and within sections * however, most importantly Adv. 1. most importantly - above and beyond all other consideration; "above all, you must be independent" above all, most especially , the online database has improved the transparency of the evaluation process by clearly detailing the performance of a database in specific areas. Library staff interested in a particular database can obtain comprehensive information about that database to assist in subscription decision making. In addition, evaluations are now available to a far wider number of library staff via the public view, which also increases confidence in the evaluation process. Online database evaluation tool: reaction The online database evaluation tool was unveiled at the October 2002 meeting of Viclink. This was the first time it had been demonstrated to the broad library community, although quite a few librarians and vendors had been aware of it, and of the whole Gulliver project and the importance of the evaluation process. Below are examples of comments from observers * From a vendor's perspective, I was extremely impressed with the professional nature of the database evaluation process created by the Gulliver committee ... From the outset, vendors can determine that there are no hidden agendas and if the database is rejected, there is justified reasoning behind the committee's decision. Natalie Blanchard Manager Electronic Sales, Thomson Gale (now with Swets Blackwell) * The Gulliver evaluation database is a great tool to gain a summary of the different databases on offer through the Gulliver consortium. The information provided gives a quick overview of the strengths and weaknesses of each database and can be used for comparisons between databases when contemplating a subscription. The evaluations have been conducted by other public librarians who share the same outlook and have the same needs and can be trusted. Gail White Information Services See Information Systems. Librarian, Eastern Regional Libraries The Gulliver evaluation database has made the process of both submitting an evaluation and viewing completed evaluations much easier for members. Being available online, rather than emailed to members, means that past evaluation forms can be viewed, and evaluations of databases can be updated when required. Lynette Lewis Regional Information Services Librarian, Yarra Plenty Regional Library Service Thank God for the evaluation database! It allows us to make more informed decisions about what databases we choose. Jill Watson Jill Watson (born 1963) is an American figure skater and coach. With her partner Peter Oppegard, she won three U.S. championships and a bronze medal at the 1988 Winter Olympics. Information and Corporate Reference Services Librarian, Bayside bay·side adj. Situated very close to or on the shore of a bay: bayside cottages. Library Service Future developments The online database evaluation tool currently contains scores for over 25 different databases. Due to its structure it can continue to expand limitlessly. However it is not intended that the evaluation tool remains static in its features. As the nature of online databases changes so too must the evaluation tool. Different issues arise, and increase in importance when assessing databases. These must be incorporated into the tool. A modification which has already been made is to allow for a weighting for directories so that they do not automatically fail evaluations because directories only have the one title included, do not provide links to journal articles etc. Thus the Gulliver steering committee has agreed to annually assess the evaluation tool and submit a list of modifications to the programmer to incorporate. At the end of 2003 some of the features on the list were * does the database conform to Verb 1. conform to - satisfy a condition or restriction; "Does this paper meet the requirements for the degree?" fit, meet coordinate - be co-ordinated; "These activities coordinate well" the American Disability Act accessibility standards accessibility standards (akses´abil´itē), n.pl the requirements designed by the Americans with Disabilities Act (ADA), by which public places must provide disabled individuals with barrier-free access to ? * can the database be viewed in languages other than English LOTE or Languages Other Than English is the name given to language subjects at Australian schools. LOTEs have often historically been related to the policy of multiculturalism, and tend to reflect the predominant non-English languages spoken in a school's local area, the ? What about ebooks? In 2002, when the online database evaluation tool was born, ebooks were relatively new. The concept was new, the hardware and software required to view them were new, and so we evaluated our very first ebook product in the same way as all of the other online databases, using the same criteria and scoring methodology. The result was probably not entirely fair to the product, but at first the issue stayed in the too hard basket Too Hard Basket subtitled B Sides and Bastards is a compilation of B Sides and unreleased tracks by New Zealand band the Mutton Birds. It was given an extremely limited release via the band's website and gig sales. . However, ebooks matured. Proprietary software required for different products was relinquished re·lin·quish tr.v. re·lin·quished, re·lin·quish·ing, re·lin·quish·es 1. To retire from; give up or abandon. 2. To put aside or desist from (something practiced, professed, or intended). 3. for more uniform standards, with the web and http becoming the predominant pre·dom·i·nant adj. 1. Having greatest ascendancy, importance, influence, authority, or force. See Synonyms at dominant. 2. delivery mechanism. Ebooks were also becoming more common in public libraries. From a few trials, they became just another format offered by public libraries to their users. Clearly, Gulliver had to respond. Accordingly, in July 2003, the steering committee made a decision to develop another evaluation tool specifically for ebooks. Whilst using the original Excel spreadsheet as a starting point Noun 1. starting point - earliest limiting point terminus a quo commencement, get-go, offset, outset, showtime, starting time, beginning, start, kickoff, first - the time at which something is supposed to begin; "they got an early start"; "she knew from the , obviously different scoring and ranking criteria will be used in the ebook tool. However this time it will be written in Perl from the beginning, thus incorporating all the advantages offered by a tool in this format. Marketing the evaluation tool The growth of, and interest in, consortial purchasing has been very high in the library sector in the last few years. Consortia now exist in all sectors and states in Australia, from Casl (Council of Australian State Noun 1. Australian state - one of the several states constituting Australia province, state - the territory occupied by one of the constituent administrative districts of a nation; "his state is in the deep south" Libraries) at a national level, to NSWNet (Network of NSW NSW New South Wales Noun 1. NSW - the agency that provides units to conduct unconventional and counter-guerilla warfare Naval Special Warfare public libraries)at a state level. However the success of Gulliver has proven to be phenomenal. Due to the enthusiasm and energy of a voluntary group of dedicated public librarians, the consortium is well organised and financially viable. This has attracted much attention in the library community and many approaches from other consortia that wish to learn from Gulliver's experience and/or hitch hitch to fasten by a knot, usually used to describe tying a horse to a post. their own wagon wagon: see carriage. wagon Four-wheeled vehicle designed to be drawn by draft animals. Wagons have been used from the 1st century BC; early examples used spoked wheels with metal rims, pivoted front axles, and linchpins to secure the wheels. to Gulliver's. This is true in relation to the online database evaluation tool as well as other Gulliver documentation such as the memorandum of understanding, business plan and marketing plan. The Gulliver steering committee is more than willing to share its expertise and experience and believes that the online database evaluation tool has the potential to be used by other consortia. The library sector has always benefited from cooperation and collaboration, and the adoption and/or elaboration of the evaluation tool has obvious advantages. This concept is one which the steering committee is more than willing to explore. Bibliography bibliography. The listing of books is of ancient origin. Lists of clay tablets have been found at Nineveh and elsewhere; the library at Alexandria had subject lists of its books. McGlinchy, S What is the selection criteria for Gulliver databases? http://www.libraries.vic.gov.au/gulliver/about.htm#criteria accessed 26 August 2003 Rae, J Gulliver report 2001/2002 http://www.libraries.vic.gov.au/gulliver/members/reports/GullRepAGM2002.doc accessed 27 August 2003 Rasiah, T Evaluation tool http://troyr.vicnet.net.au/2001/libsonline/gulliver/gulliver.cgi accessed 5 September 2003 Teresa Wight wight 1 n. Obsolete A living being; a creature. [Middle English, from Old English wiht; see wekti- in Indo-European roots. is the Information Technology and Information Services Manager at Whitehorse Manningham Regional Library Corporation. She has been involved in library IT and systems for nearly 15 years, is a member of the Gulliver steering committee and is actively involved with evaluation of databases for member libraries. Address: PO Box 65 Box Hill Vic 3128 tel(03)98901005 fax(03)98990076 teresaw@wev.vic.gov.au Teresa Wight Information Technology and Information Service Manager Whitehorse Manningham Regional Library Corporation Victoria |
|
||||||||||||||||||
Printer friendly
Cite/link
Email
Feedback
Reader Opinion