Evaluating Digital Libraries for Teaching and Learning in Undergraduate Education: A Case Study of the Alexandria Digital Earth ProtoType (ADEPT).ABSTRACT
THIS IS A DISCUSSION ON THE RESEARCH DESIGN FOR AN educational evaluation Educational evaluation is the evaluation process of characterizing and appraising some aspect/s of an educational process.
There are two common purposes in educational evaluation which are, at times, in conflict with one another. of the Alexandria Digital Earth ProtoType (ADEPT ADEPT Psychology A study–Adolescent Depression Prevention & Treatment–funded by the Natl Inst Mental Health, examining the cost-effectiveness of a 'service' model for treating children of depressed adults enrolled in HMOs. See Depression. ), a digital library of geo-referenced information resources (1) The data and information assets of an organization, department or unit. See data administration.
(2) Another name for the Information Systems (IS) or Information Technology (IT) department. See IT. . ADEPT is being studied in undergraduate classrooms at the University of California, Los Angeles UCLA comprises the College of Letters and Science (the primary undergraduate college), seven professional schools, and five professional Health Science schools. Since 2001, UCLA has enrolled over 33,000 total students, and that number is steadily rising. , and the University of California, Santa Barbara History
The predecessor to UCSB, Santa Barbara State College, focused on teacher training, industrial arts, home economics, and foreign languages. Intense lobbying by an interest group in the City of Santa Barbara led by Thomas Storke and Pearl Chase persuaded the State . The article provides a brief review of the deployment of digital libraries in educational settings, the role of information technology in developing students' scientific thinking, and the evaluation of digital libraries. We outline the overall research design, report on progress to date, and describe plans for the remainder of the five-year project. The article concludes with initial observations about classroom environments for using ADEPT and about the initial deployment of ADEPT prototypes.
Digital libraries offer a wealth of opportunities to improve access to information resources in support of both "traditional" on-campus instruction and distance-independent learning (Borgman, in press). We are still at the early stages of realizing the potential of digital libraries in educational contexts, however. Few of the technological, logistical lo·gis·tic also lo·gis·ti·cal
1. Of or relating to symbolic logic.
2. Of or relating to logistics.
[Medieval Latin logisticus, of calculation , and economic aspects of integrating digital libraries into university education have yet been assessed, much less the curricular and pedagogical ped·a·gog·ic also ped·a·gog·i·cal
1. Of, relating to, or characteristic of pedagogy.
2. Characterized by pedantic formality: a haughty, pedagogic manner. challenges (National Research Council, Center for Science, Mathematics, and Engineering Education, 1998). Relatively little work has been done on evaluating the usability of digital libraries in any context, and minimal work has been done on assessing learning outcomes associated with the implementation of digital libraries in instruction. Many complex research design questions remain to be addressed, such as what to evaluate, by what methods, and how to determine if learning is occurring.
We report here on the research questions, research design, and preliminary observations from the first year of a five-year project (1999-2004) to develop and deploy a digital library of geo-referenced information resources ("geolibrary") in undergraduate courses at the University of California, Los Angeles, and the University of California, Santa Barbara. This study is part of the Alexandria Digital Earth ProtoType (ADEPT) project funded by the U.S. Digital Libraries Initiative, Phase 2 (National Science Foundation, 1999). ADEPT is an emerging digital library that will provide instructors and students with the means to discover, manipulate, and display dynamic geographical processes. The ADEPT system provides an interesting case study to observe the deployment of a digital library in instructional settings. Our thesis is that digital library services will contribute positively to undergraduate instruction and to student learning of scientific processes. To examine this thesis, we employ a variety of qualitative and quantitative methods to investigate the impact of ADEPT in undergraduate instruction. This article extends our initial reports on the education and evaluation component of ADEPT (Leazer, Gilliland-Swetland, & Borgman, 2000; Leazer, Gilliland-Swetland, Borgman, & Mayer, in press). Continuing reports will be provided on the ADEPT Web sites at UCLA UCLA University of California at Los Angeles
UCLA University Center for Learning Assistance (Illinois State University)
UCLA University of Carrollton, TX and Lower Addison, TX (http://dlis.gseis.ucla.edu/adept/) and UCSB UCSB University of California at Santa Barbara
UCSB University of Casual Sex and Beer (http://www.alexandria. ucsb.edu/adept/).
DIGITAL LIBRARIES AND UNDERGRADUATE EDUCATION undergraduate education Medtalk In the US, a 4+ yr college or university education leading to a baccalaureate degree, the minimum education level required for medical school admission; undergraduate medical education refers to the 4 yrs of medical school. Cf CME.
Educational applications of digital libraries range from primary school through graduate school and across all disciplines. One of our chief interests is how the use of digital libraries can promote thinking processes associated with problem domains (e.g., science, social sciences, humanities) at the undergraduate level. The first stage of the ADEPT educational evaluation focuses on scientific thinking and is being conducted in physical geography physical geography: see geography. courses. This section provides a brief literature review of the role of digital libraries in education and in scientific thinking to set the context for the case study of ADEPT. The literature is reviewed relating to relating to relate prep → concernant
relating to relate prep → bezüglich +gen, mit Bezug auf +acc the evaluation of digital libraries in general.
Educational Applications of Digital Libraries
Faculty and librarians alike are concerned about ways to implement digital libraries in education. The Council on Library and Information Resources (1999) held a meeting "to consider changes in the process of scholarship and instruction that will result from the use of digital technology and to make recommendations to ensure that libraries continue to serve the research needs of scholars." Among their recommendations was that institutions of higher education higher education
Study beyond the level of secondary education. Institutions of higher education include not only colleges and universities but also professional schools in such fields as law, theology, medicine, business, music, and art. should "place more emphasis on training and support for faculty use of information and instructional technologies There are two types of instructional technology: those with a systems approach, and those focusing on sensory technologies.
The definition of instructional technology prepared by the Association for Educational Communications and Technology (AECT) Definitions and Terminology ."
The University of Michigan Digital Library Project (project) University of Michigan Digital Library Project - (UMDL) The University of Michigan's part of the Digital Library Initiative. (Wallace, Krajcik, & Soloway, 1996) posits that the main benefit of digital libraries in the classroom is improved means and opportunity for inquiry-based learning Inquiry based learning describes a range of philosophical, curricular and pedagogical approaches to teaching. Its core premises include the requirement that learning should be based around student questions. . A component of this research, the Middle Years Digital Library project (Soloway et al., 2000), allowed science students in grades six through nine to learn and explore topics in a less-regimented manner than traditional textbook learning. The few outcome-based studies that have been conducted have suggested a positive correlation Noun 1. positive correlation - a correlation in which large values of one variable are associated with large values of the other and small with small; the correlation coefficient is between 0 and +1
direct correlation between integrating electronic information sources into the classroom and increased scholastic success. Newnham, Mather, Grattan, Holmes, and Gardner (1998), for example, gave geography students access to Internet source material downloaded onto a local network file server. Students were encouraged to make use of the material and communicate among themselves via electronic mail. The study found that access to electronic geo-information sources enhanced student learning.
In a distance-learning study, Mose and Maney (1993) found that geology students with access to a combination of televised instruction and computer-based communications software (communications, software) communications software - Application programs, operating system components, and probably firmware, forming part of a communication system. These different software components might be classified according to the functions within the Open Systems demonstrated higher levels of learning than did non-computer-equipped students. Data were collected via software questionnaires, case study interviews, and course grades. While this study focused on the student-student and student-instructor communications opportunities provided by educational technology, the researchers concluded that the use of this technology in undergraduate geology courses facilitated learning and increased student engagement.
At the other end of the age spectrum, digital libraries also have helped very young children understand complex scientific concepts. Many complex concepts become understandable when taught in a contextualized and incremental Additional or increased growth, bulk, quantity, number, or value; enlarged.
Incremental cost is additional or increased cost of an item or service apart from its actual cost. manner (Metz, 1995). Kafai and Gilliland-Swetland (in press) built on Metz's work in a study where young science students recreated the process of generating and describing digital scientific documentation by emulating the activities of an early naturalist. The researchers noted that the students found visual materials more intellectually accessible, even when the source materials Noun 1. source materials - publications from which information is obtained
source - a document (or organization) from which information is obtained; "the reporter had two sources for the story" were meant for more advanced students.
Digital Libraries and Scientific Thinking
One method of determining the success of digital libraries in improving student learning is to examine whether they are helping to achieve pedagogical objectives. One such overarching o·ver·arch·ing
1. Forming an arch overhead or above: overarching branches.
2. Extending over or throughout: "I am not sure whether the missing ingredient . . . objective in geography instruction, for example, is the development of scientific thinking in students. Consensus exists that students need to learn five skill sets in order to engage in scientific thinking in geography: (1) asking geographic questions, (2) acquiring geographic information, (3) organizing geographic information, (4) analyzing geographic information, and (5) answering geographic questions (Geography Education Standards Project, 1994, pp. 42-44).
The first skill set--asking geographic questions--involves being able to pose questions that can be addressed in the field of geography. When faced with an issue, students need to be able to formulate geographic questions, such as How did that get there? or What are the consequences of that being there? Digital libraries have a role to play at the question-asking phase of scientific thinking, because students can get ideas for questions by browsing some of the available information. In this way, the browsing capabilities of digital libraries can aid the user's question-asking process.
The second skill set--acquiring geographic information--includes locating and collecting relevant information, such as reading maps and other visual representations of space. The information-gathering phase represents perhaps the most obvious venue for digital libraries, because they can greatly increase the efficiency of obtaining relevant information and may even allow access to information that would not otherwise be available. In this way, the information retrieval information retrieval
Recovery of information, especially in a database stored in a computer. Two main approaches are matching words in the query against the database index (keyword searching) and traversing the database using hypertext or hypermedia links. capabilities of digital libraries can aid the user's information-gathering process.
The third skill set--organizing geographic information--includes systematically arranging and displaying geographic information, such as maps and graphs. For example, in some cases, a user may create a map based on collected information. Digital libraries can help users in their information-organizing process if the libraries include tools for manipulating information--especially tools for creating visual representations of information.
The fourth skill set is analyzing geographic information--a process that includes finding patterns, trends, relationships, and connections in the information one has gathered and organized. In some cases, the analysis may involve scrutinizing patterns in maps or charts, and in other cases the analysis may involve statistical analyses of quantitative data. Digital libraries can aid in this information-analysis process by providing tools for aggregating and analyzing data.
The fifth skill set is answering geographic questions and often takes the form of a written or oral generalization gen·er·al·i·za·tion
1. The act or an instance of generalizing.
2. A principle, a statement, or an idea having general application. or conclusion. The Standards emphasize that "students should also understand that there are alternative ways to reach generalizations and conclusions" (Geography Education Standards Project, 1994, p. 44). Digital libraries can assist in this question-answering process by allowing users easily to check the predictions of the explanatory models they construct.
By understanding how students will use a digital library in the context of scientific thinking, it is possible to construct the digital library in a way that will support the underlying processes. In short, digital libraries are more than storehouses of information; they should be aids to the question-asking, information-gathering, information-organizing, information-analyzing, and question-answering processes of users. In this way, digital libraries can also support the broader call in the National Science Education Standards The National Science Education Standards (NSES) are a set of guidelines for the science education in primary and secondary schools in the United States, as established by the National Research Council in 1996. (National Research Council, 1996) for allowing "inquiry into authentic questions generated from student experiences" (p. 31). Although we focus on promoting geographic thinking in our project, the same kinds of skills that support geographic thinking also apply to other scientific disciplines. The five skills can be used in inductive inductive
1. eliciting a reaction within an organism.
a form of radiofrequency hyperthermia that selectively heats muscle, blood and proteinaceous tissue, sparing fat and air-containing tissues. (or data-driven) reasoning, such as looking for Looking for
In the context of general equities, this describing a buy interest in which a dealer is asked to offer stock, often involving a capital commitment. Antithesis of in touch with. trends in data that lead to a theory, or in deductive de·duc·tive
1. Of or based on deduction.
2. Involving or using deduction in reasoning.
de·duc (or theory-driven) reasoning, such as testing two competing theories through dynamic modeling.
Evaluating Digital Libraries
The goals of the ADEPT project, from an educational perspective, are to construct a digital library that will make geo-spatial and geo-referenced information resources useful in undergraduate instruction and whose use will lead to better learning outcomes than with traditional modes of instruction. We are conducting both formative evaluation Formative evaluation is a type of evaluation which has the purpose of improving programmes. It goes under other names such as developmental evaluation and implementation evaluation. that assists in formulating design requirements and summative Adj. 1. summative - of or relating to a summation or produced by summation
additive - characterized or produced by addition; "an additive process" evaluation that assesses learning outcomes from using the system in instructional settings.
Digital libraries are difficult to evaluate due to their richness, complexity, and variety of uses and users. Few proven methods are available. The need for evaluation methods and metrics metrics Managed care A popular term for standards by which the quality of a product, service, or outcome of a particular form of Pt management is evaluated. See TQM. was among the key findings of the Social Aspects of Digital Libraries Workshop (Borgman et al., 1996). Some progress is being made, as evidenced by this special issue and by a forthcoming book on the evaluation of digital libraries (Bishop, Buttenfield, & Van House, in press). Most evaluation studies of digital libraries address questions of usability (Borgman, in press-b). "Usability," however, like "user friendly," is an amorphous Unorganized or vague. A lack of structure. For example, the amorphous state of a spot on a rewritable optical disc means that the laser beam will not be reflected from it, which is in contrast to a crystalline state which will reflect light. See crystalline. term with a wide range of context-dependent interpretations.
Many general criteria for usability exist, such as those proposed for "every citizen interfaces to the nation's information infrastructure" (Toward an Every-Citizen Interface ..., 1997, p. 45). Criteria include "easy to understand, easy to learn, error tolerant, flexible and adaptable, appropriate and effective for the task, powerful and efficient, inexpensive, portable, compatible, intelligent, supportive of social and group interactions, trustworthy--secure, private, safe, and reliable, information centered, [and] pleasant to use" (p. 45). Other applicable criteria are the user interface design rules established by Shneiderman (1998) as adapted to information retrieval (Shneiderman, Byrd, & Croft CROFT, obsolete. A little close adjoining to a dwelling-house, and enclosed for pasture or arable, or any particular use. Jacob's Law Dict. , 1997): strive for consistency, provide shortcuts See Win Shortcuts. for skilled users, offer informative feedback, design for closure, simple error handling, permit easy reversal of actions, support user control, and reduce short-term memory short-term memory
Abbr. STM The phase of the memory process in which stimuli that have been recognized and registered are stored briefly. load. Nielsen (1993) identifies five general usability attributes for information systems as well as other applications: learnability, efficiency, memorability, errors, and satisfaction.
These principles offer general guidance for design but are far from a "cookbook (programming) cookbook - (From amateur electronics and radio) A book of small code segments that the reader can use to do various magic things in programs.
One current example is the "PostScript Language Tutorial and Cookbook" by Adobe Systems, Inc (Addison-Wesley, ISBN " for constructing any individual digital library. Principles such as "easy to learn" must be applied relative to the application and the user community. A system that supplies daily weather reports to the public must be much easier to learn than one that supplies geophysical ge·o·phys·ics
n. (used with a sing. verb)
The physics of the earth and its environment, including the physics of fields such as meteorology, oceanography, and seismology. data to researchers, for example. Setting appropriate benchmarks for any given system involves evaluation with members of the target audience and comparisons to similar applications.
Design guidelines guidelines,
n.pl a set of standards, criteria, or specifications to be used or followed in the performance of certain tasks. and evaluation criteria can be employed to build more usable systems but only to the extent that design goals are appropriate for the application. Determining appropriate design goals for digital libraries is itself a challenge given the early stages of research on uses, users, and usability and the rapid evolution of the underlying technologies. Formative evaluation is a particularly valuable approach in such situations, because user needs and requirements can be studied concurrently with initial stages of designing the system (Gilliland-Swetland, 1998; Marchionini & Crane, 1994; Fitz-Gibbon & Morris, 1987).
The Digital Library for Earth Sciences Education (DLESE DLESE Digital Library for Earth System Education ) project (http://www.dlese.org/SoftwareArchitecture/requirements/index.html), with which ADEPT is a cooperating partner, also is in the early stages of formative evaluation. DLESE has solicited scenarios from their user community of K-12 teachers as a basis for identifying requirements for functionalities and systems architecture. The scenarios are reviewed and refined with assistance from the user community, which is actively involved in the project. The DLESE project is finding that the requirements are a moving target because, as community level of sophistication so·phis·ti·cate
v. so·phis·ti·cat·ed, so·phis·ti·cat·ing, so·phis·ti·cates
1. To cause to become less natural, especially to make less naive and more worldly.
2. grows, so does the number and sophistication of their requests. One goal of the evaluation plan is to examine how well the final products meet the functionalities defined in the scenarios submitted by the community (Marlino, personal communication, August 14, 2000).
ADEPT UNDERGRADUATE IMPLEMENTATION AND EVALUATION
ADEPT is an extension and enhancement of the Alexandria Digital Library (ADL), which was developed at the University of California, Santa Barbara (UCSB) under the first Digital Libraries Initiative (1994-1998). ADL is an operational digital library that provides access to collections of maps, images, and other geo-referenced materials from a 1.5 terabyte One trillion bytes. Also TB, Tbyte and T-byte. See tera and space/time.
(unit) terabyte - 2^40 = 1,099,511,627,776 bytes = 1024 gigabytes or roughly 10^12 bytes.
(Note the spelling - one 'r'). See prefix. (and growing) collection of materials from UCSB's Map and Imagery Laboratory (http://www.alexandria.ucsb.edu). The operational version of ADL provides users with access to services that allow them to answer such questions as what information is available about a given phenomenon at a particular set of places. ADL also provides new types of library services based on gazetteers and other information access tools. ADL went online in Fall 1999 as part of the California Digital Library The California Digital Library, or CDL, is the University of California's 11th University Library. The CDL assists the ten University of California libraries in sharing their resources and holdings more effectively, in part through negotiating and acquiring consortial licenses on (http://www.cdlib.edu). Formative evaluation of ADL focused on multiple target user communities (earth scientists, information specialists, and educators) using a variety of methods, including online surveys, ethnographic studies ethnographic studies,
n.pl methods of qualitative research developed by anthropologists, in which the researcher attends to and inter-prets communication while participating in the research context. , a classroom study with a later version of the interface (Hill et al., 2000) and transaction logs (Buttenfield & Kumler, 1996).
The ADEPT project, also centered at UCSB, is developing a digital earth metaphor for organizing, using, and presenting information at all levels of spatial and temporal resolution Temporal resolution refers to the precision of a measurement with respect to time. Often there is a tradeoff between temporal resolution of a measurement and its spatial precision (spatial resolution). . A central aspect of ADEPT is the development of I-scapes (information landscapes), whose working definition is as follows:
I-scapes are a means of expressing and visualizing geo-spatial concepts and processes, for research, instruction, and learning. I-scapes also include a set of tools and resources for the use of geo-spatial and geo-referenced information resources in teaching undergraduate courses. Instructors will employ I-scapes to convey concepts; students will use I-scapes to learn concepts and to get experience manipulating information resources in the ways that domain experts use them.
Our working scenario for the use of ADEPT I-scapes is that the course instructor will define the scope and concepts of a topic to be taught. The instructor, with the aid of a graduate student researcher from the education and evaluation team, will assemble a small collection of information resources for teaching the topic and will apply ADEPT tools and services to create I-scapes. The instructor will use one or more I-scapes to present the topic in class lecture sessions. Teaching assistants also will use I-scapes to discuss and demonstrate the topic in laboratory sessions. Students will perform exercises in lab sessions and outside of class using I-scapes to test hypotheses in the pre-selected collection of resources.
In support of this proposed scenario, the ADEPT project is developing a range of analysis tools and modeling services that will enable users to construct their own personalized per·son·al·ize
tr.v. per·son·al·ized, per·son·al·iz·ing, per·son·al·iz·es
1. To take (a general remark or characterization) in a personal manner.
2. To attribute human or personal qualities to; personify. digital libraries and to use them in creative ways alone and in collaboration with other users.
At the core of effective digital library design is the relationship between the content to be provided and the user community to be served. Design goals can originate from either perspective. Best of all, design goals from both perspectives should converge. We are taking a convergence approach to design, with the education and evaluation team focusing on needs assessment, evaluating prototypes in active use, and identifying system requirements To be used efficiently, all computer software needs certain hardware components or other software resources to be present on a computer system. These pre-requisites are known as (computer) system requirements and are often used as a guideline as opposed to an absolute rule. . Concurrently, the ADEPT implementation team is focusing on evolving the ADL testbed architecture and services, such as interface specifications, service prototypes, interoperability The capability of two or more hardware devices or two or more software routines to work harmoniously together. For example, in an Ethernet network, display adapters, hubs, switches and routers from different vendors must conform to the Ethernet standard and interoperate with each other. , and collection growth and diversity. Ours is an iterative it·er·a·tive
1. Characterized by or involving repetition, recurrence, reiteration, or repetitiousness.
2. Grammar Frequentative.
Noun 1. and collaborative approach to development, with evaluation integrally embedded Inserted into. See embedded system. in design. Needs are identified from the user and collections perspective, prototypes are constructed and evaluated, and the results fed back into the design and development process.
We hypothesize hy·poth·e·size
v. hy·poth·e·sized, hy·poth·e·siz·ing, hy·poth·e·siz·es
To assert as a hypothesis.
To form a hypothesis. that digital library services that provide instructors and students with the means to discover, manipulate, and display dynamic geographical processes will contribute positively to undergraduate instruction and to the development of scientific and other discipline-specific reasoning skills. This hypothesis generates a number of research questions, only a few of which are addressed in this article. Here we focus on how the evaluation research design addresses the following questions:
* How can ADEPT modules support domain knowledge, work practices, and reasoning models of multiple disciplines that use geo-spatial resources? For example, can ADEPT modules and services be structured in such a way that they will help a student to think like a geographer or an environmentalist environmentalist
a person with an interest and knowledge about the interaction of humans and animals with the environment. ?
* How can ADEPT accommodate users with different skills, knowledge, cognitive styles Cognitive style is a term used in cognitive psychology to describe the way individuals think, perceive and remember information, or their preferred approach to using such information to solve problems. , and pedagogical styles? While it is difficult for any digital library to support such heterogeneity het·er·o·ge·ne·i·ty
The quality or state of being heterogeneous.
the state of being heterogeneous. in its users, are there ways in which ADEPT can facilitate moving between different domain knowledge and technological skill levels as users become more sophisticated? Are there also ways in which users who are less comfortable with the spatial metaphor can enhance their spatial processing capabilities?
* How can ADEPT help users view primary geographical evidence in new ways to answer scientific or geographical questions? ADEPT provides users with diverse primary research data (e.g., remotely sensed data) as well as published information. It also provides users with links to non-digital information held elsewhere and enables users to incorporate additional content that they have created or collected themselves. Using the tools and services provided by ADEPT, can users combine, manipulate, and visualize these resources in ways that will allow them to ask and answer questions in original and creative ways?
* How can ADEPT support the range of heterogeneous resources and their metadata necessary for learning applications? Even though ADEPT's holdings are vast, it is difficult to anticipate the specific resources that instructors or students may wish to incorporate into a class presentation or project. Frequently, they will wish to draw upon additional materials that they already have in their possession, materials that may not be in the preferred file formats or accompanied by the systematic metadata of existing ADEPT holdings.
We are addressing these research questions through several concurrent approaches, including establishing general design principles; analyzing cognitive processes Cognitive processes
Thought processes (i.e., reasoning, perception, judgment, memory).
Mentioned in: Psychosocial Disorders and information-seeking needs of instructors and learners; analyzing practices, behaviors, and knowledge-processing requirements of the disciplines within which ADEPT is being implemented; and developing case-based prototypes.
General Design Principles. We began the project with a top-down approach Top-down approach
A method of security selection that starts with asset allocation and works systematically through sector and industry allocation to individual security selection. to setting requirements, establishing general design principles such as:
* ADEPT should support real scientific problems that can be studied, learned, or solved with the use of geo-spatial and geo-referenced information resources.
* I-scapes should focus on dynamic (rather than static) processes for which "real data" can be visualized and manipulated by instructors and students.
* Research should concentrate initially on geography, as we have the most knowledge and information resources in this domain.
* Research should expand later to other disciplines that use geo-spatial and geo-referenced information resources (e.g., geology, earth and environmental sciences, sociology, urban planning urban planning: see city planning.
Programs pursued as a means of improving the urban environment and achieving certain social and economic objectives. , and even humanities fields that sometimes organize content by geographic location, such as art history and theology).
* ADEPT must be easy to learn and use Easy to learn refers to software that is well designed and capable of being used right away without having to wade through documentation. If you make the program work with little effort, it is easy to learn. by undergraduates with minimal domain knowledge and technical skills (e.g., freshmen in geography courses for non-majors).
* ADEPT must be easy for instructors to learn and use.
* ADEPT should improve teaching productivity.
* ADEPT should create minimal additional workload for instructors.
The latter three principles address the problem of incentives for faculty to use ADEPT in instruction. While faculty at research universities such as UCLA and UCSB are held to high standards of teaching, it is but one of several significant demands on their time. They are most likely to adopt new teaching tools and methods if the overhead in time and effort is minimized and if the resulting advantages are deemed worthy of the investment. The project includes a substantial amount of graduate student research assistance for working with faculty in developing and deploying I-scapes in instruction as a means to encourage participation.
User Groups and Tasks. Based on the usage scenario described earlier, we identified four user groups for study: (1) faculty (in their role as course instructors), (2) teaching assistants, (3) students in the courses where ADEPT is implemented, and (4) students continuing to study the discipline after the completion of the course. These groups differ in needs, activities, and levels of domain knowledge. Analysis of baseline data indicates that each of these user groups generates different requirements for what and how ADEPT resources and services are developed. We have identified the following list of candidate tasks that ADEPT should support:
* highly-directed uses such as lab exercises to reinforce a specific disciplinary concept;
* instructional modules that introduce concepts in an incremental manner and can be customized and extended by faculty for use in lectures;
* free-form exploration conducted by students preparing term papers or faculty putting together a lecture that might include personal manipulation of data sets, information visualization Representing data in 3D images in order to navigate through it more quickly and access it in a more natural manner. Although the term was coined at Xerox's Palo Alto Research Center, which has developed very advanced techniques, multidimensional cubes, or pivot tables, are a simpler form , and the integration of new information or data sets to augment existing content;
* collaborative applications that might be used by students doing team projects or faculty and teaching assistants who are team-teaching; and
* discipline or domain-specific methods of building knowledge that support specific information seeking Information seeking is the process or activity of attempting to obtain information in both human and technological contexts. Information seeking is related to, but yet different from, information retrieval (IR). and use processes.
Information systems and services designed to facilitate learning must accommodate a variety of pedagogical goals and styles and a variety of learning styles. The processes whereby information is identified, selected, retrieved, manipulated, annotated, and presented to others often are more important than the retrieval and use of the information itself. To examine these usage contexts and the interplay in·ter·play
Reciprocal action and reaction; interaction.
intr.v. in·ter·played, in·ter·play·ing, in·ter·plays
To act or react on each other; interact. among them, we are conducting three types of studies, employing qualitative and quantitative methods: classroom-based studies, laboratory studies, and system use studies. Some will be conducted longitudinally and others will occur at different points throughout the development of ADEPT.
Assessing Learning Outcomes. How can we assess students' scientific thinking in geography? What cognitive changes occur in students who receive experience in ADEPT that do not occur in non-ADEPT students? In short, what are the cognitive consequences of participating in an ADEPT environment? Our evaluation of cognitive learning outcomes seeks to answer these questions through performance assessments of scientific problem solving problem solving
Process involved in finding a solution to a problem. Many animals routinely solve problems of locomotion, food finding, and shelter through trial and error. . Performance assessment involves giving learners realistic tasks and carefully observing how they go about handling them; in a science domain, this means presenting a scientific problem and observing how students engage in scientific problem solving--that is, observing how students actually "do science" (Doran, Lawrenz, & Helgeson, 1994, p. 415). Performance assessment techniques are popular in science education because they allow for a richer assessment that goes beyond testing for students' remembering specific facts (Baxter, Shavelson, Goldman, & Pine, 1992; Persky et al., 1996). We plan to assess the cognitive consequences of participation in the ADEPT program by testing an ADEPT group and a comparison group on a series of performance tasks, each tapping one of the five target skills in geographical thinking.
1. Asking Geographic Questions. The first skill is to formulate a testable question. To assess this skill, we will present students with a geography scenario, expressed as a short video, and ask them to generate as many testable questions as possible. Scoring will be based on the number of acceptable questions that each student proposes.
2. Acquiring Geographic Information. The second skill is to gather relevant information. To assess this skill, we will present students with a geography question, expressed as a short video, and ask them to list the kinds of information they would need. As in the prior test, scoring will be based on the number of acceptable information requests that each student proposes.
3. Organizing Geographic Information. The third skill is to organize relevant information in a way that supports scientific thinking. To assess this skill, we will present various information sources to students in the form of text, video, or graphics files that can be accessed and ask them to create summary graphics for a future presentation about a target question. For this and the following two tests, scoring will be based on a scale of 0 to 5.
4. Analyzing Geographic Information. The fourth skill is to find patterns or relations in organized geographical material, such as graphics. To assess this skill, we will present various summary graphics--intended to address a geography question--and ask students to write a sentence to accompany each one.
5. Answering Geographic Questions. The final skill is to create a verbal conclusion or generalization to a target question based on organized and analyzed information. To assess this skill, students will be asked to write a one-paragraph answer to a target question based on a series of narrated graphics that organize and analyze the relevant information.
The design of a performance assessment program involves a number of issues. Our design decisions are guided by a conception of assessment in which the quality of a test depends on four characteristics:
1. reliability--the test gives a consistent score,
2. validity--the test measures what it is supposed to measure,
3. standardization--the test score allows for comparison among test-takers, and
4. objectivity--the test is scored and administered the same way for everyone.
First, should we focus intensively on one geography scenario or broadly on several? We plan to include several geography scenarios rather than focus on one because a broader set of scenarios is likely to increase the reliability of our assessment.
Second, should we test for :near transfer (i.e., problems like those used during instruction) or far transfer (i.e., problems that are not closely related to those used during instruction)? We opt for testing students on problems that are similar in format to the problems used during instruction so they require the same scientific thinking skills but which involve different geography content so students cannot simply remember specific answers. This approach increases the validity of our assessment.
Third, should our measurements be quantitative--i.e., in the form of numbers--or qualitative--i.e., in the form of a written summary of our observations? We opt for quantitative measurements that are tied to a clear scoring rubric RUBRIC, civil law. The title or inscription of any law or statute, because the copyists formerly drew and painted the title of laws and statutes rubro colore, in red letters. Ayl. Pand. B. 1, t. 8; Diet. do Juris. h.t. . This approach increases the standardization standardization
In industry, the development and application of standards that make it possible to manufacture a large volume of interchangeable parts. Standardization may focus on engineering standards, such as properties of materials, fits and tolerances, and drafting of the assessment.
Fourth, should we provide scripted or open-ended guidance to students as they seek to solve the problems we present? We intend to provide scripted guidance so that all students will receive the same kinds of interactions with teachers. This approach increases the objectivity of the assessment.
The final product will be a set of performance assessment instruments that allow for quantitative measurement of each of the five target skills in scientific thinking in geography.
Research Schedule and Strategy
First Year Progress (1999-2000). This first academic year of the implementation and evaluation component of the ADEPT project has been devoted to requirements analysis (project) requirements analysis - The process of reviewing a business's processes to determine the business needs and functional requirements that a system must meet. , evaluation design, and pilot testing that is concurrent with the development of the ADEPT architecture by the UCSB-based development team. Activities have included:
* establishing general design principles;
* developing the evaluation design and instruments;
* identifying and recruiting faculty and students to participate in implementation and follow-up;
* identifying pedagogical goals and styles in participating faculty through classroom observations and interviews;
* identifying canonical The standard or authoritative method. The term comes from "canon," which is the law or rules of the church. See canonical name and canonical synthesis.
canonical - (Historically, "according to religious law")
* developing prototype I-scapes and pilot testing our evaluation methods in courses at UCLA and UCSB;
* gathering baseline demographic and performance data on students who have taken the classes under study in the previous five years as well as students currently enrolled in the classes; and
* gathering preliminary feedback from students at the mid- and end-points of the classes in which prototype modules of ADEPT have been implemented.
Based on consultations with the chairs of the geography departments at UCSB and UCLA, we determined that the introductory courses in physical geography at both campuses were best suited for initial studies. This course is taught three times per year at UCLA (Fall, Winter, and Spring terms), which is a larger campus, and the equivalent course is taught once per year at UCSB. The course is taught at the lower division level (freshman-sophomore) and satisfies general education requirements for the bachelor's degree, so it draws students from all disciplines. The course also is a prerequisite for geography majors. All four instructors agreed to participate in the evaluation study, and all four sections of the course were observed on a regular basis by members of the ADEPT education and evaluation team. The instructors also provided copies of laboratory assignments and exams used to assess students.
During the Spring term (April to June 2000), prototype I-scapes were deployed in the physical geography courses at UCSB and UCLA. We consulted the instructors to identify a list of course topics that involved dynamic processes and that could be explained better through dynamic presentations rather than the current static presentations on overheads, slides, or chalkboards. Of the suggested list, the topics selected for the initial I-scapes were hydrology hydrology, study of water and its properties, including its distribution and movement in and through the land areas of the earth. The hydrologic cycle consists of the passage of water from the oceans into the atmosphere by evaporation and transpiration (or and fluvial flu·vi·al
1. Of, relating to, or inhabiting a river or stream.
2. Produced by the action of a river or stream.
[Middle English, from Latin processes, as a body of materials and instructor expertise were most readily available. At UCSB, four ADEPT lectures were guest lectures given by a faculty member who is part of the ADEPT team rather than being presented by the course instructor.
Plans for Year Two (2000-2001). Based on knowledge gained from the first year of the project, we will devote the second year to a case-based bottom-up approach to the education and evaluation project. We are constructing several exemplar ex·em·plar
1. One that is worthy of imitation; a model. See Synonyms at ideal.
2. One that is typical or representative; an example.
3. An ideal that serves as a pattern; an archetype.
4. cases that will help to generalize generalize /gen·er·al·ize/ (-iz)
1. to spread throughout the body, as when local disease becomes systemic.
2. to form a general principle; to reason inductively. design from specific instances. We have selected two cases from those previously agreed upon Adj. 1. agreed upon - constituted or contracted by stipulation or agreement; "stipulatory obligations"
noncontroversial, uncontroversial - not likely to arouse controversy by participating instructors of physical geography and one for a course on human geography Human geography, is a branch of geography that focuses on the study of patterns and processes that shape human interaction with the environment, with particular reference to the causes and consequences of the spatial distribution of human activity on the Earth's surface. to be taught in the 2000-2001 academic year. The physical geography cases are erosion, as a topic on fluvial processes, and subduction sub·duc·tion
A geologic process in which one edge of one crustal plate is forced below the edge of another.
[French, from Latin subductus, past participle of , as a topic on plate tectonics plate tectonics, theory that unifies many of the features and characteristics of continental drift and seafloor spreading into a coherent model and has revolutionized geologists' understanding of continents, ocean basins, mountains, and earth history. . For human geography, we have tentatively selected Von Thunen models as a topic on land usage.
We will collect appropriate content for these topics, working with participating instructors, and the implementation team will develop services and functional capabilities for ADEPT prototypes. To date, we have identified these requirements for the three cases:
* appropriate metadata and representation of content;
* appropriate searching capabilities to select content within I-scapes;
* ability to manipulate appropriate parameters to demonstrate processes and test hypotheses;
* visualization Using the computer to convert data into picture form. The most basic visualization is that of turning transaction data and summary information into charts and graphs. Visualization is used in computer-aided design (CAD) to render screen images into 3D models that can be viewed from all features to demonstrate processes and test hypotheses;
* ability for individual students and instructors to save their work for reuse reuse - Using code developed for one application program in another application. Traditionally achieved using program libraries. Object-oriented programming offers reusability of code via its techniques of inheritance and genericity. ; and
* instrumentation to capture user-system interactions.
Evaluation will be conducted in classrooms and in laboratory settings, as discussed earlier.
Plans for Years Three through Five (2001-2004). Years three through five of the project will continue the usability and evaluation studies with subsequent iterations of I-scapes in multiple classrooms in multiple disciplines. Lectures are only a starting point Noun 1. starting point - earliest limiting point
terminus a quo
commencement, get-go, offset, outset, showtime, starting time, beginning, start, kickoff, first - the time at which something is supposed to begin; "they got an early start"; "she knew from the , as noted earlier. We plan to expand ADEPT into laboratory applications, independent learning, and provide functionality for collaborative and distributed learning Distributed Learning means a method of instruction that relies primarily on indirect communication between students and teachers, including internet or other electronic-based delivery, teleconferencing or correspondence; (British Columbia, School Act, 2006). .
Our findings are preliminary, as we are still analyzing the first year's data at this writing, and ADEPT itself is in the early stages of development. We report here only on baseline classroom observations and on results from the prototype I-scapes deployment at UCLA and UCSB. The prototypes provided a first look at introducing computer-based technology into geography classrooms for teaching dynamic processes and an opportunity to test our instruments. We have no learning outcome data yet. Further data from faculty interviews, student interviews, and student demographics The attributes of people in a particular geographic area. Used for marketing purposes, population, ethnic origins, religion, spoken language, income and age range are examples of demographic data. will be reported later. Initial observations reported here are in two categories: classroom environment and implementation of ADEPT I-scapes prototypes.
We recognized early in the project that ADEPT modules would have to be flexible, adaptable, and relatively small in scope. Among the design questions we are exploring at this stage are the following: At what level of detail or granularity should I-scapes be created? How detailed can or should modules be if they are to be useful for multiple instructors? What are the implications for metadata to describe and represent the concepts? What are the collection requirements for ADEPT? These questions guided our observations of lectures and discussion sections of the physical geography courses at UCLA and UCSB. Faculty and graduate student researchers from the ADEPT education and evaluation team observed a sampling of class sessions before, during, and after the implementation of the I-scapes prototypes. We also videotaped one lecture by each of the UCLA instructors and videotaped the I-scapes lectures at UCSB. These observations form baseline data to determine what is in common and what varies across the multiple offerings of the same and similar courses, what aspects of the courses might be incorporated in I-scapes, and what aspects are independent of I-scapes. We are not attempting to reform undergraduate education as a whole; rather, our purpose is to identify ways in which digital libraries can be utilized effectively in instruction.
At UCLA, the same introductory physical geography course was taught by three different instructors in one academic year (Fall, Winter, and Spring terms on the quarter system), providing the opportunity to compare multiple approaches to teaching the same course. The comparable introductory course at UCSB, taught only once per year, has similar content and used one of the same textbooks as at UCLA. The three UCLA courses were taught with two lectures per week plus one required laboratory session taught by a graduate teaching assistant. The UCSB course was taught with three 50-minute lectures per week plus one required laboratory session taught by a graduate teaching assistant. Enrollment ranged from 60 to 120 students, and four to six laboratory sections were offered for each course. In total, we observed four courses (three at UCLA and one at UCSB) and five instructors (three at UCLA and two at UCSB: the regular course instructor and the guest who presented the four I-scapes lectures).
Course Content. Topic emphasis varied considerably due to differences in course texts and in instructors' interests and expertise. The five instructors were experts in climatology climatology
Branch of atmospheric science concerned with describing climate and analyzing the causes and practical consequences of climatic differences and changes. Climatology treats the same atmospheric processes as meteorology, but it also seeks to identify slower-acting , geomorphology geomorphology, study of the origin and evolution of the earth's landforms, both on the continents and within the ocean basins. It is concerned with the internal geologic processes of the earth's crust, such as tectonic activity and volcanism that constructs new , remote sensing Deriving digital models of an area on the earth. Using special cameras from airplanes or satellites, either the sun's reflections or the earth's temperature is turned into digital maps of the area. , river systems, pedology pedology
A branch of soil science focusing on the formation, morphology, and classification of soils as bodies within the natural landscape. Pedology seeks to understand how the properties and distribution patterns of soils worldwide (collectively called the pedosphere) have , and soil evolution. Although all the instructors covered all the requisite topics, the proportional amount of time devoted to each topic reflected their respective research areas. In interviews with the instructors and in observation of their lectures, all had a core teaching goal in common, which was to present geography as a system of interacting processes. In the areas of their greatest expertise, instructors were most likely to interject in·ter·ject
tr.v. in·ter·ject·ed, in·ter·ject·ing, in·ter·jects
To insert between other elements; interpose. See Synonyms at introduce. anecdotal anecdotal /an·ec·do·tal/ (an?ek-do´t'l) based on case histories rather than on controlled clinical trials.
anecdotal adjective Unsubstantiated; occurring as single or isolated event. stories, specific case studies, and state-of-the-art research into the classroom discussion. In doing so, instructors were able to personalize per·son·al·ize
tr.v. per·son·al·ized, per·son·al·iz·ing, per·son·al·iz·es
1. To take (a general remark or characterization) in a personal manner.
2. To attribute human or personal qualities to; personify. the scientific process and explain "how science is done" in the field or in the lab.
The three UCLA instructors chose three different textbooks and followed them to varying degrees. One instructor followed the text closely, lecturing chapter by chapter and testing students on textbook content. Another drew illustrations from the text and incorporated other materials into lectures, testing students on all materials covered in lectures and lab sessions. The third instructor relied on the text for background reading and structured his lectures much differently than the textbook organization. The UCSB instructor and guest lecturer occasionally referenced textbook chapters for the day's lecture but did not follow the text closely, nor did they follow the ordering of the chapters. From time to time, one of the UCSB instructors would indicate parts of the textbook, such as a table or graph, which summarized something he had just discussed, or he would display a table pulled from the text and elaborate upon it.
Teaching Styles. We also found considerable variation in teaching style, part of which we attribute to the amount of teaching experience. At UCLA, one of the instructors was an assistant professor who was teaching the course for the first time, another was a recently-promoted associate professor who had taught the course at least once before, and the third was a senior full professor who had taught this course many times over a period of several decades. At UCSB, the instructor and guest instructor were full professors with many years of teaching experience, though neither had taught this course recently. The five faculty varied in lecture style, use of instructional technology, and their approaches to engaging students in discussion.
Perhaps, as might be expected, the degree of reliance on the textbook varied with teaching experience. The assistant professor relied most heavily on the text and the full professors the least. While all the instructors answered student questions during lectures, the degree to which they directly elicited e·lic·it
tr.v. e·lic·it·ed, e·lic·it·ing, e·lic·its
a. To bring or draw out (something latent); educe.
b. To arrive at (a truth, for example) by logic.
2. student involvement varied. At UCLA, one instructor actively involved students in the lecture sessions, while the other two tended to defer student discussion to office hours office hours,
n.pl See business hours. and lab sections, devoting more classroom time to lecturing. At UCSB, the instructors would occasionally direct questions to the students or give them a scenario and ask them to make predictions, but they also tried to engage the students by drawing references between the topics being discussed and students' personal experiences. For example, in discussing types of land or river formations, the instructor asked how many students had taken a hike in a certain local area, asked them to describe what they saw, then related specific features of that area to a topic in the lecture. For the most part, however, any active student discussion took place in lab or during office hours.
The five faculty also had different approaches to the use of instructional technology in their classrooms. At UCLA, one instructor lectured almost entirely from notes on an overhead projector, another lectured almost entirely from the chalkboard, and the third used a mixture of chalkboard and overheads. The instructor who made the most use of the chalkboard wrote copious co·pi·ous
1. Yielding or containing plenty; affording ample supply: a copious harvest. See Synonyms at plentiful.
2. notes on the board with detailed diagrams. Dynamic processes were illustrated in multiple colors of chalk. He often came to the classroom about twenty minutes before class to prepare his diagrams on the board. He also brought in a large map of the world on which he would point out the location where specific processes occurred. Some lectures were augmented with slides and one with rock samples.
At UCSB, both instructors primarily used computer presentations which were displayed on one or both of the two screens in the front of the room and on several smaller television screens mounted throughout the lecture hall lecture hall n → sala de conferencias;
(UNIV) → aula
lecture hall lecture n → amphithéâtre m
. The primary course instructor used a hypermedia hypermedia: see hypertext.
The use of hyperlinks, regular text, graphics, audio and video to provide an interactive, multimedia presentation. All the various elements are linked, enabling the user to move from one to another. format for his computer presentation, which usually consisted of an outline of key terms and an accompanying image or animation. He occasionally used an overhead projector, displayed on the second screen, to write down additional information or to use one of his favorite transparencies. He did not read from prepared notes but instead used the outline of his computer presentation as a prompting tool. He would spend from two to twenty minutes on each screen image. The guest lecturer used two computers, presenting his lecture outline on one screen and images and animations on the other. His lecture was fairly fast paced, mostly following the outline but occasionally deleting or re-ordering topics.
The instructors also varied in the emphasis they placed on learning specific concepts and on learning processes. One instructor focused on approximately six processes per lecture, each illustrated with an overhead from the textbook. This format allowed students to follow the lecture in the book and to repeat concepts and clarify processes during the lecture period.
We also noted differences in the instructors' use of the physical space in the classroom. One of the UCLA instructors was very active, constantly moving as he drew detailed diagrams at the board and as he pointed out locations on the world map. Another instructor was moderately active, as she switched from writing notes on the board, to speaking from the overhead, to entertaining questions from the class. The latter two instructors were in a large bright lecture hall with many windows and a high ceiling. The third instructor was least active, standing at the podium podium
In architecture, a pedestal on a large scale. It may be any of various elements that form the base of a structure, such as the platform forming the floor and substructure of a Classical temple, a low wall supporting columns, or the structurally or decoratively for the duration of the lecture, showing overhead figures and writing on transparencies. His course was held in a low-ceilinged windowless room that he kept dark to maximize the legibility leg·i·ble
1. Possible to read or decipher: legible handwriting.
2. Plainly discernible; apparent: legible weaknesses in character and disposition. of his overheads.
The two UCSB faculty were physically constrained con·strain
tr.v. con·strained, con·strain·ing, con·strains
1. To compel by physical, moral, or circumstantial force; oblige: felt constrained to object. See Synonyms at force.
2. by their use of computing computing - computer equipment, although they used it differently. The regular professor used one computer terminal, which was located in the lectern. He would pace around the front of the lecture hall, point to relevant areas on the projection screen, and speak directly toward the students, returning to the lectern only when he needed to switch to the next screen. The guest professor occasionally walked up closer to the students and moved around but, due to the use of two computers, the faster pace of screen changes, and some technical difficulties with the microphone, was more restricted in physical movement. He wrote diagrams and notes on the computer screen using a computer pen rather than using an overhead projector.
Classroom Implementation of ADEPT 1-scapes Prototypes
Given the amount of development required to produce fully-operational I-scapes with real scientific data, we took a rapid-prototyping approach and constructed simple prototypes using MicroSoft PowerPoint. This approach enabled us to incorporate text, photographs, diagrams, images, and moving images that illustrated dynamic processes and allowed the instructor to add comments and annotations in real time. As noted in the prior section, the UCLA instructor developed one lecture on fluvial processes using a one-screen method that combined graphics, simulations, and the lecture outline, while the UCSB guest instructor developed four lectures on hydrology and fluvial processes using a two-screen projection method (one for graphics and simulations and one for the lecture outline). Both instructors were assisted in the I-scapes development by graduate student researchers employed by the ADEPT project.
Following are some of our initial findings on the classroom implementation in Spring 2000 and their implications for subsequent iterations of ADEPT.
Integrating Information Resources into ADEPT I-scapes. Despite the richness of the Alexandria Digital Library, we found it necessary to locate additional information resources in support of the hydrology and fluvial processes lectures for introductory physical geography courses. Instructors wished to integrate materials such as lecture outlines and notes, diagrams, or personal slide collections for which they presumably pre·sum·a·ble
That can be presumed or taken for granted; reasonable as a supposition: presumable causes of the disaster. held intellectual property rights. Other materials of interest were drawn from textbooks, CD-ROMs, online resources, or other sources for which they did not hold the rights. The need to integrate additional materials has implications for system design, management of intellectual property, and sharing of resources. Under fair use guidelines, instructors normally can present published materials in a classroom lecture and often do. When materials are incorporated in other products, posted online, or shared with other instructors, rights and permissions are much less clean The simple prototypes were developed for research purposes only and will not be shared until and unless we can resolve the intellectual property issues.
Presentation Capabilities. In gathering materials for the ADEPT I-scapes prototypes, we found that display, layout, and other presentation features are essential considerations. Instructors often selected illustrations based on graphical qualities over relevance and familiarity (e.g., an image of a river in Africa was visually more striking than an available image of a local river). Visual context must be provided by clear labeling, zooming, use of recognizable geographic features, and other means.
Instructors' Experiences with Information Technologies. All of the instructors studied at UCLA and UCSB are accomplished researchers who employ high-end technology in their scholarship. Geography is a technology-intensive field, particularly in areas covered by courses such as climatology and geomorphology. Even so, the UCLA instructors did not normally employ computer-based instructional technology in teaching this introductory course (other than that provided by the ADEPT project), instead relying upon chalkboards, overhead projectors, slide projectors, and paper maps for instruction. When asked their reasons, they said that too much advance planning was required for computer-based instruction, and that too much assistance would be required to install equipment, keep it running, and so on. They were interested in experimenting with new instructional methods, however, and were willing to participate in ADEPT because we supplied equipment, technical assistance, and graduate assistant support in developing lecture materials. Several of the instructors commented that they would prefer, at least initially, to have "canned" materials rather than live digital libraries or online connections in the classroom. Two faculty members commented that they did not wish to present a technology-based lecture in front of 100 or more students "without a net." The course section studied at UCSB did incorporate computer-based materials and had technical assistance at a level equal to, or greater than, that supplied by ADEPT.
The UCLA instructor who implemented the ADEPT I-scape prototype Spring term told us in interviews that he was willing to experiment with our technology in his lectures, acknowledging that he was somewhat apprehensive about the computer-driven nature of the presentation as it ran counter to his usual teaching style. He found the experience satisfactory, however, and sees ADEPT's primary benefit as an effective visual aid to communicate concepts in physical geography. While he was concerned initially that the extra effort and stress might be detrimental to his teaching, he felt that his students benefitted from the experience. He found the one-screen I-scapes module somewhat cumbersome, however. He had trouble moving back and forth through concepts as he does in regular lectures and felt constrained by the computer. He also felt the slide show sped up the lecture and did not give students time to digest the content and to take notes at the same time. He said that next time he would keep overheads nearby and use ADEPT as a supplement, not as the driver of a whole lecture.
The UCSB guest instructor used dual screens, one for the illustrations and one for the lecture outline. He was very positive about the experience (it "converted him"), even though he had not used computer-based tools in his own teaching in the past. He said that he "loves it" and "feels like he can get his points across more cleanly clean·ly
adj. clean·li·er, clean·li·est
Habitually and carefully neat and clean. See Synonyms at clean.
In a clean manner.
clean and effectively." He also thinks that the only way for students to comprehend complex phenomena is to see these demonstrated visually. He did find that ADEPT was something of an obstacle to communicating with students because the technology drew some of his focus, but thought that he would become more comfortable as he became more familiar with the setup. Also, the computer kept him physically in the same place during the lecture, which he sees as an improvement over the amount of pacing across the room he normally does during lectures.
Our goals for the ADEPT project are to construct a digital library that will make geo-spatial and geo-referenced information resources useful in undergraduate instruction, ultimately leading to better learning outcomes than with traditional modes of instruction. The first year of the education and evaluation component of the ADEPT project has been devoted to establishing general design principles; developing the research design; developing and pilot testing data collection instruments; gathering baseline data on how geography courses currently are taught and on the students who take these courses; developing, deploying, and evaluating the first ADEPT I-scapes prototypes; and planning subsequent stages of the project.
The research design, including the development of methods and instruments for gathering qualitative and quantitative data, has been a substantial undertaking in itself. To date, we have drawn our methods from educational evaluation, cognitive psychology cognitive psychology, school of psychology that examines internal mental processes such as problem solving, memory, and language. It had its foundations in the Gestalt psychology of Max Wertheimer, Wolfgang Köhler, and Kurt Koffka, and in the work of Jean , human factors, systems analysis, and user-centered design In broad terms, user-centered design (UCD) is a design philosophy and a process in which the needs, wants, and limitations of the end user of an interface or document are given extensive attention at each stage of the design process. . We expect the methods and research questions to evolve throughout the project as the technology, classroom environments, and user requirements are all moving targets.
Our initial observations suggest that matching the content and capabilities of I-scapes to the range of instructors' approaches to teaching the same topics will be a considerable challenge. The five instructors we have studied so far vary substantially in their topic emphases, choice of texts, use of instructional technology, and student assessments. We need to determine the appropriate granularity of I-scapes topics as well as their contents and their features, such as the abilities to manipulate data, test hypotheses, and visualize processes. The early data also suggest that the presentation quality of images or simulations may be as important as their source or location. These instructors traded clarity and labeling of images with familiarity to the students (e.g., local rivers versus rivers on other continents). Similarly, the metadata to describe content is an issue. While some information resources were sought by location (e.g., names of local rivers; latitude and longitude latitude and longitude
Coordinate system by which the position or location of any place on the Earth's surface can be determined and described. Latitude is a measurement of location north or south of the Equator. coordinates), others were sought by type and topic (e.g., simulations of river erosion). Pacing is yet another concern. Much more material can be presented with computer-based instructional tools than with chalkboards and overhead projectors, and students easily can be overwhelmed o·ver·whelm
tr.v. o·ver·whelmed, o·ver·whelm·ing, o·ver·whelms
1. To surge over and submerge; engulf: waves overwhelming the rocky shoreline.
a. . Conversely con·verse 1
intr.v. con·versed, con·vers·ing, con·vers·es
1. To engage in a spoken exchange of thoughts, ideas, or feelings; talk. See Synonyms at speak.
2. , students may be entertained by slick presentations without learning the scientific processes as well as they might through slower-paced chalkboard explanations. We hope to address these issues in more depth in the next year by concentrating on laboratory sections and student interaction for a few selected cases.
Campus infrastructure, instructional support, and technical support are essential concerns of the faculty studied. They are willing, if not always eager, to experiment with computer-based technologies in the classroom, provided sufficient support is available. They want support for developing instructional materials such as I-scapes. The ADEPT project provided this support by investing a considerable amount of graduate research assistant effort. Similarly, they need technical support so that precious minutes in the classroom are not wasted with set-up, debugging (programming) debugging - The process of attempting to determine the cause of the symptoms of malfunctions in a program or other system. These symptoms may be detected during testing or use by real users. , and take-down of equipment. The faculty we are studying all are sophisticated users (and some are developers) of information technologies. They know from experience that overhead projectors and chalk are more reliable instructional technologies than are computer systems, and these experiences are reflected in their teaching styles and their advice to the project.
Digital libraries hold great potential for teaching and learning at the undergraduate level. The ADEPT project is building on a rich source of geographic information resources in the Alexandria Digital Library, advanced technical infrastructures at two major research universities, and the participation of technically sophisticated faculty who teach undergraduate courses. We have a tremendous opportunity to understand more about the requirements for constructing digital libraries that will enhance scientific thinking and learning. We are making inroads inroads
make inroads into to start affecting or reducing: my gambling has made great inroads into my savings
inroads npl to make inroads into [+ at understanding the problem and hope to offer some workable solutions in later phases of the project.
This article is based on work supported by the National Science Foundation under grant no. IIS-9817432, Terence R. Smith, University of California, Santa Barbara, Principal Investigator Noun 1. principal investigator - the scientist in charge of an experiment or research project
scientist - a person with advanced knowledge of one or more sciences . Our thanks go to our colleagues at UCSB and to the faculty and students at UCLA and UCSB who are participating in the ADEPT project. The article benefitted greatly from comments on earlier drafts by Michael Goodchild, UCSB; Mary Marlino, Digital Library for Earth Sciences Education; and Anita Coleman, UCLA.
Baxter, G. P.; Shavelson, R.J.; Goldman, S. R.; & Pine, J. (1992). Evaluation of procedure-based scoring for hands-on science assessment. Journal of Educational Measurement, 29(1), 1-17.
Bishop, A. P.; Buttenfield, B. P.; & Van House, N. (Eds.). (in press). Digital library use: Social practice in design and evaluation. Cambridge, MA: MIT MIT - Massachusetts Institute of Technology Press.
Borgman, C. L. (in press-a). Digital libraries and virtual universities. In T. Della Senta & T. Tschang (Eds.), Access to knowledge: New information technologies and the emergence of the virtual university. Paris: International Association of Universities, and Oxford, England: Pergamon.
Borgman, C. L. (In press-b). Uses, users, and usability of digital libraries. In A. P. Bishop, B. P. Buttenfield, & N. Van House (Eds.), Digital library use: Social practice in design and evaluation. Cambridge, MA: MIT Press.
Borgman, C. L. (2000). From Gutenberg to the global information infrastructure: Access to information in the networked world. Cambridge, MA: MIT Press.
Borgman, C. L.; Bates Bates , Katherine Lee 1859-1929.
American educator and writer best known for her poem "America the Beautiful," written in 1893 and revised in 1904 and 1911. , M.J.; Cloonan, M. V.; Efthimiadis, E. N.; Gilliland-Swetland, A.; Kafai,Y.; Leazer, G. L.; & Maddox, A. (1996). Social aspects of digital libraries (Final Report to the National Science Foundation; Computer, Information Science, and Engineering Directorate; Division of Information, Robotics robotics, science and technology of general purpose, programmable machine systems. Contrary to the popular fiction image of robots as ambulatory machines of human appearance capable of performing almost any task, most robotic systems are anchored to fixed positions , and Intelligent Systems; Information Technology and Organizations Program, 1996. Award number 95-28808). Retrieved September 8, 2000 from the World Wide Web: http://dlis.gseis.ucla.edu/ DL/UCLA_DL_Report.html.
Buttenfield, B. P., & Kumler, M. P. (1996). Tools for browsing environmental data: The Alexandria Digital Library interface (Proceedings of the Third International Conference on Integrating Geographic Information Systems geographic information system (GIS)
Computerized system that relates and displays data collected from a geographic entity in the form of a map. The ability of GIS to overlay existing data with new information and display it in colour on a computer screen is used primarily to and Environmental Modeling, Santa Fe Santa Fe, city, Argentina
Santa Fe, city (1991 pop. 341,000), capital of Santa Fe prov., NE Argentina, a river port near the Paraná, with which it is connected by canal. , NM, January 21-25, 1996). Santa Barbara Santa Barbara (săn'tə bär`brə, –bərə), city (1990 pop. 85,571), seat of Santa Barbara co., S Calif., on the Pacific Ocean; inc. 1850. , CA: National Center for (geographic Information and Analysis. Retrieved September 9, 2000 from the World Wide Web: http://www.ncgia.ucsb.edu/conf/SANTA_FE_CD-ROM/sf_papers/buttenfield_babs/ babs_paper.html.
Council on Library and Information Resources. (1999). Executive summary. In Scholarship, instruction, and libraries at the turn of the century: Results from five task forces appointed by the American Council of Learned Societies The American Council of Learned Societies, founded in 1919, is a private non-profit federation of sixty-eight scholarly organizations.
ACLS is best known as a funder of humanities research through fellowships and grants awards. and the Council on Library and Information Resources. Washington, DC: CLIR CLIR Council on Library and Information Resources
CLIR cross-language information retrieval
CLIR Connected Line Identification Restriction
CLIR Calling Line Identity Restriction
CLIR cross-lingual information retrieval
CLIR Calling Line Identification Restriction . Retrieved September 9, 2000 from the World Wide Web: http://www.clir.org/pubs/reports/pub78/preface.html#executive.
Doran, R. L.; Lawrenz, E; & Helgeson, S. (1994). Research on assessment in science. In D. L. Gabel (Ed.), Handbook of research on science teaching and learning (pp. 388-442). New York New York, state, United States
New York, Middle Atlantic state of the United States. It is bordered by Vermont, Massachusetts, Connecticut, and the Atlantic Ocean (E), New Jersey and Pennsylvania (S), Lakes Erie and Ontario and the Canadian province of : Macmillan.
Fitz-Gibbon, C. T., & Morris, L. L. (1987). How to design a program evaluation Program evaluation is a formalized approach to studying and assessing projects, policies and program and determining if they 'work'. Program evaluation is used in government and the private sector and it's taught in numerous universities. , 2d ed. Newbury Park, CA: Sage.
Geography Education Standards Project. (1994). Geography for life: National geography standards. Washington, DC: National Geographic Research & Exploration.
Gilliland-Swetland, A.J. (1998). Evaluation design for large-scale, collaborative online archives: Interim report of the Online Archive of California Evaluation Project. Archives & Museum Informatics Same as information technology and information systems. The term is more widely used in Europe. , 12(3-4), 177-203.
Hill, L. L.; Carver carver /car·ver/ (kahr´ver) a tool for producing anatomic form in artificial teeth and dental restorations.
carver (carving instrument),
n , L.; Larsgaard, M.; Dolin, R.; Smith, T. R.; Frew, J.; & Rae, M. A. (2000). Alexandria Digital Library: User evaluation studies and system design. Journal of the American Society for Information Science, 51(3), 246-259.
Kafai, Y., & Gilliland-Swetland, A.J. (in press). Integrating historical source materials into elementary science classroom activities. Science Education.
Leazer, G. H.; Gilliland-Swetland, A.J.; & Borgman, C. L. (2000). Evaluating the use of a geographic digital library in undergraduate classrooms: The Alexandria Digital Earth Prototype (ADEPT). In Proceedings of the Fifth ACM (Association for Computing Machinery, New York, www.acm.org) A membership organization founded in 1947 dedicated to advancing the arts and sciences of information processing. In addition to awards and publications, ACM also maintains special interest groups (SIGs) in the computer field. Conference on Digital Libraries (San Antonio San Antonio (săn ăntō`nēō, əntōn`), city (1990 pop. 935,933), seat of Bexar co., S central Tex., at the source of the San Antonio River; inc. 1837. , TX, June 2-7, 2000) (pp. 248-249). New York: ACM Press.
Leazer, G. H.; Gilliland-Swetland, A.J.; Borgman, C. L.; & Mayer, R. (in press). Classroom evaluation of the Alexandria Digital Earth Prototype (ADEPT). Unpublished paper presented at the American Society for Information Science, 2000 Annual Conference, Chicago.
Marchionini, G., & Crane, G. (1994). Evaluating hypermedia and learning: Methods and results from the Perseus Project The Perseus Project is a digital library project of Tufts University that assembles digital collections of humanities resources. It is hosted by the Department of Classics. . ACM Transactions on Information Systems, 12(1), 5-34.
Metz, K. (1995). Reassessment Reassessment
The process of re-determining the value of property or land for tax purposes.
Property is usually reassessed on an annual basis. You may request a "reassessment" if you disagree with your assessment. of developmental constraints on children's science instruction. Review of Educational Research, 65(2), 93-127.
Mose, D., & Maney, T. (1993). An experiment in distance learning of geology. Journal of Computers in Mathematics and Science Teaching, 12(1), 5-18.
National Research Council, Center for Science, Mathematics, and Engineering Education. (1998). Developing a digital national library for undergraduate science, mathematics, engineering and technology education: Report of a workshop. Washington, DC: National Academy Press.
National Research Council. (1996). National science education standards: Observe, interact, change, learn. Washington, DC: National Academy Press.
National Science Foundation. (1999). Digital Libraries Initiative, Phase IL Retrieved September 9, 2000 from the World Wide Web: http://www.dli2.nsf.gov.
Newnham, R.; Mather, A.; Grattan, J.; Holmes, A.; & Gardner, A. (1998). An evaluation of the use of Internet sources as a basis for geography coursework coursework
work done by a student and assessed as part of an educational course
Noun 1. coursework - work assigned to and done by a student during a course of study; usually it is evaluated as part of the student's . Journal of Geography in Higher Education, 22(1), 19-34.
Nielsen, J. (1993). Usability engineering Usability engineering is a subset of human factors that is specific to computer science and is concerned with the question of how to design software that is easy to use. It is closely related to the field of human-computer interaction and industrial design. . Boston: Academic Press.
Persky, H. R.; Reese, C. M.; O'Sullivan, C. Y.; Lazer, S.; Moore, J.; & Shakrani, S. (1996). NAEP NAEP National Assessment of Educational Progress
NAEP National Association of Environmental Professionals
NAEP National Association of Educational Progress
NAEP National Agricultural Extension Policy
NAEP Native American Employment Program 1994 Geography Report Card: Findings from the National Assessment of Educational Progress The National Assessment of Educational Progress (NAEP), also known as "the Nation's Report Card," is the only nationally representative and continuing assessment of what America's students know and can do in various subject areas. (NCES NCES National Center for Education Statistics
NCES Net-Centric Enterprise Services (US DoD)
NCES Network Centric Enterprise Services
NCES Net Condition Event Systems 96-087). Washington, DC: Office of Educational Research and Improvement, U.S. Department of Education.
Shneiderman, B. (1998). Designing the user interface: Strategies for effective human-computer interaction Human-computer interaction
An interdisciplinary field focused on the interactions between human users and computer systems, including the user interface and the underlying processes which produce the interactions. , 3d ed. Reading, MA: Addison Wesley Longman.
Shneiderman, B.; Byrd, D.; & Croft, W. B. (1997). Clarifying search: A user-interface framework for text searches. D-Lib Magazine D-Lib Magazine is an on-line magazine dedicated to digital library research and development. Content of current and past issues are available free of charge. The publication is financially supported by the Defense Advanced Research Projects Agency (as part of the Digital , 3(1). Retrieved September 9, 2000 from the World WideWeb: http://www.dlib.org/dlib/january97/retrieval/01shneiderman.html.
Soloway, E.; Norris, C.; Blumenfeld, P.; Fishman, B.; Krajcik, J.; & Marx, R. (2000). K-12 and the Internet. Communications of the ACM (publication) Communications of the ACM - (CACM) A monthly publication by the Association for Computing Machinery sent to all members. CACM is an influential publication that keeps computer science professionals up to date on developments. , 43(1), 19-23.
Toward an Every-Citizen Interface to the Nation's Information Infrastructure Steering Committee steer·ing committee
A committee that sets agendas and schedules of business, as for a legislative body or other assemblage.
Noun . Computer Science and Telecommunications Board; Commission on Physical Sciences, Mathematics, and Applications; National Research Council. (1997). More than screen deep: Toward every-citizen interfaces to the nation's information infrastructure. Washington, DC: National Academy Press.
Wallace, R.; Krajcik, J.; & Soloway, E. (1996). Digital libraries in the science classroom: An opportunity for inquiry. D-Lib Magazine, September 1996. Retrieved September 9, 2000 from the World Wide Web: http://www.dlib.org/dlib/september96/umdl/09wallace.html.
CHRISTINE BORGMAN holds the Presidential Chair in Information Studies at UCLA, is Visiting Professor in the Department of Information Science at Loughborough University Loughborough University is located in the market town of Loughborough, Leicestershire in the East Midlands of England. The University offers degree programmes and research. , England (1996 through 2002), and was a Fulbright Professor in Budapest, Hungary. Her teaching and research interests include digital libraries, human-computer interaction, electronic publishing An umbrella term for non-paper publishing, which includes publishing online or on media such as CDs and DVDs. , information seeking behavior, and scholarly communication Scholarly Communication is an umbrella term used to describe the process of academics, scholars and researchers sharing and publishing their research findings so that they are available to the wider academic community (such as university academics) and beyond. and bibliometrics Bibliometrics is a set of methods used to study or measure texts and information. Citation analysis and content analysis are commonly used bibliometric methods. While bibliometric methods are most often used in the field of library and information science, bibliometrics have wide , as well as information technology policy in Central and Eastern Europe The term "Central and Eastern Europe" came into wide spread use, replacing "Eastern bloc", to describe former Communist countries in Europe, after the collapse of the Iron Curtain in 1989/90. . She is the author, most recently, of From Gutenberg to the Global Information Infrastructure: Access to Information in a Networked World (MIT Press, 2000). Her current research addresses the use of geo-spatial and geo-referenced digital libraries in undergraduate education. She has lectured or conducted research in more than twenty countries. She received her Ph.D. in communication from Stanford University Stanford University, at Stanford, Calif.; coeducational; chartered 1885, opened 1891 as Leland Stanford Junior Univ. (still the legal name). The original campus was designed by Frederick Law Olmsted. David Starr Jordan was its first president. and her M.L.S. from the University of Pittsburgh.
ANNE J. GILLILAND-SWETLAND is an Assistant Professor in the Department of Information Studies of the Graduate School of Education and Information Studies at UCLA. Her teaching and research interests relate to electronic records administration and the development and evaluation of information systems containing primary sources.
GREGORY H. LEAZER is an Assistant Professor in the Department of Information Studies at the University of California-Los Angeles. His research interests include the design and evaluation of information retrieval systems, bibliographic control, metadata, and bibliographic works.
RICHARD MAYER is a Professor of Psychology at the University of California, Santa Barbara. He received a Ph.D. in psychology from the University of Michigan (body, education) University of Michigan - A large cosmopolitan university in the Midwest USA. Over 50000 students are enrolled at the University of Michigan's three campuses. The students come from 50 states and over 100 foreign countries. in 1973. He is past president of the Division of Educational Psychology of the American Psychological Association The American Psychological Association (APA) is a professional organization representing psychology in the US. Description and history
The association has around 150,000 members and an annual budget of around $70m. , former editor of the Educational Psychologist and Instructional Science and the year 2000 recipient of the E.L. Thorndike Award for lifetime contributions to educational psychology. He is the author of 200 publications which include The Promise of Educational Psychology (1999) and Multimedia Learning (in press).
DAVID David, in the Bible
David, d. c.970 B.C., king of ancient Israel (c.1010–970 B.C.), successor of Saul. The Book of First Samuel introduces him as the youngest of eight sons who is anointed king by Samuel to replace Saul, who had been deemed a failure. GWYNN is in the Department of Psychology at the University of California, Santa Barbara.
RICH GAZAN is a Ph.D. student in the UCLA Department of Information Studies and has worked for several database publishing Using desktop publishing to produce reports of database contents. See desktop publishing. companies. His research interests include information systems, knowledge integration, and the information industry. He received his MLIS MLIS Master of Library and Information Science
MLIS Multilingual Information Society
MLIS Molecular Laser Isotope Separation
MLIS Masters of Library and Information Studies
MLIS Medical/Legal Information Services from the University of Hawaii (body, education) University of Hawaii - A University spread over 10 campuses on 4 islands throughout the state.
See also Aloha, Aloha Net. .
PATRICIA PATRICIA Practical Algorithm To Retrieve Information Coded In Alphanumeric
PATRICIA Proving and Testability for Reliability Improvement of Complex Integrated Architectures
PATRICIA PApilloma TRIal Cervical cancer In young Adults MAUTONE is a graduate student in Cognitive Psychology at the University of California, Santa Barbara. Her main focus of research is multimedia learning and instruction.
Christine L. Borgman Christine L. Borgman (b. 1951) is Professor and University of California Presidential Chair in Information Studies at the Graduate School of Education and Information Studies, University of California, Los Angeles. , Anne J. Gilliland-Swetland, Gregory H. Leazer, and Rich Gazan, Department of Information Studies, GSE GSE
general somatic efferent system. & IS Bldg, University of California, Los Angeles, CA 90095-1520
Richard Mayer and Patricia Mautone, Department of Psychology, University of California, Santa Barbara, CA 93106
David Gwynn, Department of Geography, Bunche Hall, University of California, Los Angeles, CA 90095