Quantity with quality? Teaching quantitative and qualitative methods in an LIS master's program.

INTRODUCTION

Many schools of library and information science (LIS) are faced with an acute problem. Both quantitative and qualitative research paradigms are represented in the LIS literature, yet only a single research methods course is offered at the master's level. This being so, the problem invites a number of questions: What ought such a course include? Should instructors, opting for breadth, attempt to teach both qualitative and quantitative methods? Alternatively, might it be desirable to opt for one or the other and achieve a greater degree of depth? How should such a course be taught? These questions are certainly important. The answers hinge on the answer to yet another question--can LIS programs afford to graduate information professionals who are literate in one methodological paradigm only?

Evaluating the research output in an appropriate domain is an essential component of professionalism. That is the case for all professions. In LIS, a profession that has adopted a user-centered approach to service provision, services and systems must constantly be reviewed in light of the most current research findings. Only by doing so can LIS practitioners express confidence that their services and systems are the best they can offer. When viewed in this way, it becomes obvious that the toolkit of the competent information professional must include an understanding of the methods of both paradigms. Without this understanding, master's level graduates will be inadequately prepared to do research or to be critical consumers of the research of others.

To acquire this element of the toolkit requires not only a theoretical grounding in methods but also a practical element--LIS students must experience research. The value of experiential learning is well documented in the education literature. Schall (1996) points out that professionals often deal with complex issues and argues for the inclusion of experiential learning in graduate and professional curricula. Schall believes that, through hands-on experience in dealing with such issues during course work, today's student will be able to address these issues in future professional practice. Research too is complex, and students cannot learn to do or to evaluate research from the literature alone. Practical hands-on experience in doing research is an essential ingredient of the learning process. Consequently, an LIS research methods course must embrace a substantial practical element that takes students through the process of design, data collection, data analysis, and reporting.

THE TWO PARADIGMS

As pointed out by Orlikowsky and Baroudi (1991), a quantitative research methodology is appropriate where quantifiable measures of variables of interest are possible, where hypotheses can be formulated and tested, and inferences drawn from samples to populations. Qualitative methods, on the other hand, are appropriate when the phenomena under study are complex, are social in nature, and do not lend themselves to quantification. Typically, qualitative methods are used when understanding the cultural context from which people derive meaning is an important element of a study. Such cultural context is usually not susceptible to quantification and aggregation and is, therefore, usually ignored in quantitative studies. Yet failure to understand cultural context may deprive the researcher of a real understanding of the problem at hand (Kaplan & Maxwell, 1994).

In LIS, researchers grapple with problems of both sorts--they study information systems as well as the interactions of people with those systems. In other words, they are concerned with both functionality and usability. The cultural context is vital for determining usability and, to some extent, for functionality. Researchers are beginning to recognize that resorting to a single methodological paradigm in LIS research does not provide the understanding needed to design and maintain effective services. It follows, therefore, that courses based on a single methodological paradigm are inadequate preparation for the information professional.

QUANTITATIVE METHODS

Because quantitative methods are well defined and easy to validate, it has not been a difficult problem, historically, to fit these methods into a single semester course. In quantitative research, observations are reported in aggregate quantitative form. Formal hypotheses are posed that are tested and either accepted or rejected. To conduct quantitative research implies the need for very precise identification and definition of variables and the ability to operationalize them in such a way that numbers can be attributed to them--e.g., age, GRE, opinions on satisfaction, liking, and so on. Admittedly, this is often difficult because many variables may be relevant. Fidel and Soergel (1983) identified more than 200 variables that affect just the online bibliographic search process. However, if the variables of interest have been identified and operationalized and a data gathering scheme has been devised, the researcher can design the study and rely on well established and accepted criteria for judging its validity. In other words, much can be said about the validity, reliability, and generalizability of the study from the design alone.

While identifying and operationalizing variables in complex research problems is difficult, once it has been done, the process of data gathering and analysis is well defined and relatively straightforward. Furthermore, unless the study is longitudinal, data collection is generally also relatively fast-i.e., once the study design is determined, the gathering and analysis of data proceed rapidly. In addition, although data for quantitative studies can be gathered in the field, more often data are gathered in-house (through laboratory experiments) or through survey instruments of various types. In any case, from a pedagogical standpoint, data for a variety of quantitative studies can be gathered "from within the classroom" so to speak.

Given the nature of quantitative research, it is entirely possible to teach a quantitative methods course with dummy data only. Although students invariably question the limited relevance of dummy data in the learning process, many courses are, nevertheless, taught in exactly this way. Of course, this approach lends itself rather well to the time constraints imposed by the academic semester. Pedagogically, the quantitative approach is also well suited to simple examples that can be tackled in a short time. "Dummy" studies (using small data sets and a limited set of variables) can be "conducted" and the results analyzed in class. Thus, students can be given considerable practice over the course of a semester in testing a variety of hypotheses.

QUALITATIVE METHODS

Qualitative research, on the other hand, is highly time consuming. The very language of qualitative research suggests an exhaustive process--prolonged engagement, intensive observation, thick description (Lincoln & Guba, 1985). Good design is, of course, a requisite for any type of study. However, it is fair to say that qualitative studies lack the same clear cut and objective standards for design as quantitative studies--i.e., the qualitative design emerges as the study unfolds. Indeed, Creswell (1994) points out that there is little agreement among researchers in terms of a set of precise procedures for data collection, analysis, and reporting of qualitative studies. Consequently, the evaluation of a qualitative study cannot be realized in the absence of data collection and analysis (since analysis proceeds hand in hand with observation). The quantitative researcher can be confident that the standards of rigor for design and analysis have been met before data collection commences. The qualitative researcher cannot do the same. As Lincoln and Guba (1985) point out, the pillars of qualitative research--credibility, transferability, dependability, and confirmability--cannot be demonstrated in the absence of data. Of course, this does not mean that the design process for a qualitative study is unimportant. Before collecting and analyzing data, the qualitative researcher must, according to Berg (1998), "sketch out the entire research project in an effort to foresee any possible glitches that might arise" (p. 27). But unlike a quantitative study, an evaluation of the design without data is much more difficult.

While the quantitative researcher can gather data without leaving her institution, the qualitative researcher most often gathers data through observations "in the field." Indeed, an immersion in the natural setting of the phenomena under observation is often an essential element of qualitative research. As Cavan (1966) pointed out, if you want to know how people socialize in bars, you have to visit bars to observe them. The principal emphasis in learning qualitative methods lies in learning how to observe, record, and analyze real interactions between people and between people and systems. Thus, using dummy data in teaching qualitative research is clearly absurd. It appears, therefore, that teaching qualitative methods in the classroom alone is untenable and that it may be difficult to teach through field observations in a single semester course.

However, if LIS professionals face research problems that call for both methodologies, teaching only one or the other is a disservice to graduates of LIS programs and to the profession. Given the arguments just presented, it may seem curious to suggest that both methodological paradigms can be taught in a single semester, indeed, that there may be sound pedagogical reasons for doing so.

SHOWED WE COMBINE?

Chenail (1992) pointed out that some qualitative research has sometimes been legitimized by juxtaposing it with quantitative studies. In this sense, qualitative research is seen as pre-quantitative, post-quantitative, or in combination as a triangulation of methods. Of course, qualitative research in LIS no longer needs to be legitimized by juxtaposing it with quantitative research. Instead, a growing number of researchers are employing such combinations of methods to gain greater insights. Because many phenomena in LIS are highly complex, it makes sense to employ multiple perspectives to expand understanding.

The nature of research in LIS is such that solutions to important problems are seldom found in one study and by using one methodology. Indeed, there is consensus within each of the methodological paradigms that there are advantages to using multiple methods to enhance understanding standing of phenomena--i.e., triangulation, as coined by Denzin (1978). Indeed, Lincoln and Guba (1985) suggest that triangulation is one important means of demonstrating the credibility of a qualitative study. It seems that, within each methodological paradigm, triangulation is accepted, in deed viewed, as highly desirable.

A growing number of researchers, interested in obtaining answers to thorny questions, are selecting methods from both paradigms. Jick (1979) showed how quantitative methods could augment the findings of a qualitative study, pointing out that weaknesses in one methodology can be compensated for by strengths in another. Morse (1991) used both qualitative and quantitative methods concurrently and in sequence and argued that these approaches result in deeper understanding of phenomena. Creswell (1994) refers to these approaches to research as "between methods" triangulation.

In addition to triangulation, Greene, Caracelli, and Graham (1989) identify several other factors that might lead researchers to opt for "between methods" studies. They suggest that this approach allows researchers to focus on different facets of a phenomenon, to look at phenomena sequentially to observe development, to discover paradoxes and new perspectives, and to add depth and breadth to a study.

It appears that "between methods" approaches to research are gaining credence within the social sciences. If that is so, then a strong case can be made for LIS research methods courses that integrate the two paradigms through their application to common research questions. Herein lies the key to the pedagogical problem. If, rather than teaching methods in isolation, instructors take an integrated approach to a research question, then the problem under investigation is the focus of the semester's study and not the particular methods being taught--truly a "between methods" approach.

All of us are aware that research methods can be learned uncritically--almost by rote. If they are learned in this way, they will almost certainly be applied in a like manner. The integrated "between methods" approach to learning research methods allows the instructor to juxtapose methods and expose both their weaknesses and their strengths. Learning in this environment is much more likely to result in critical appraisal of research designs and methodologies.

Careful sifting of potential research problems can ensure development of a set of questions that can be investigated concurrently using both methodological paradigms and, through "between methods" triangulation, gaining added perspectives as well as supporting evidence. This approach achieves two pedagogical objectives: the student gains a richer understanding of the phenomena under study, while at the same time critically appraising the strengths and weaknesses of the various methods employed on a single problem. In this scenario, the semester consists of problem identification, study design (using both quantitative and qualitative methods), data collection and analysis, and reporting results, all centered on a single set of research questions. Data collection for the quantitative approach is completed relatively quickly, while in the qualitative approach, the process continues for much of the semester.

Of course, there are many possible designs and procedures for qualitative research. Tesch (1990) developed a typology of twenty design types. Clearly not all can be taught in one course. From a pedagogical perspective, students should have as much exposure as possible to whatever common ground can be identified among the different methods. The question really is, What should LIS students know that will allow them to interpret all qualitative studies critically and intelligently? The essentials are:

* defining (and justifying) purposive samples,

* data collection through interviews and observation,

* data analysis simultaneous with data collection, and

* data analysis through reduction and interpretation.

Interview methods lend themselves particularly well to the time constraints of the single semester. A common interview method in some areas of qualitative research is the use of focus groups. When the purpose of a study is to gain insights into attitudes of participants, the focus group is an appropriate method. Catterall and Maclaran (1997) point out that, from a pedagogical standpoint, a central benefit is the convenience of interviewing several participants at once while simultaneously obtaining a wider range of responses, including forgotten details of experiences that are brought out through group interaction. Time saved is, of course, an important factor in a single semester course. However, the use of focus groups in a course suffers one significant disadvantage--since focus groups are not natural entities, observations are conducted in the artificial setting of the focus interview. The method is, therefore, unsuitable for participant observation of natural groups in natural settings (Berg, 1998).

Focus groups may reveal what participants think about a task while they are not actually performing it. They will not necessarily disclose what participants actually do and how they feel while they are performing a task. It is well to recognize, as did Mintzberg (1979, 1983), that to genuinely understand, to be able to answer the interesting questions in some domain, requires direct observations and many studies. Mintzberg devoted a career to answering the question: What do managers really do? His methods, and those of his students, were qualitative, and the insights they attained over several years were remarkable. So, students must have experience in direct observation. It is also instructive for students to realize that a study conducted over one semester is unlikely to add a great deal to our knowledge in any domain of LIS. The goal then must be to achieve an understanding of what each method can contribute to the answers sought and how methods can complement each other. In the classroom, the goal is to reveal research problems that have these qualities. While they are attractive, interviews and focus groups alone are insufficient to meet these goals. The course work must include, as a necessary component, direct observations of individuals while they are performing tasks of interest.

USE OF COMPUTERS AS ANALYSIS TOOLS

The use of computers for data manipulation and analysis is well established among researchers in both methodological paradigms. Consequently, hands-on use of appropriate computer software must augment a methods course. For quantitative researchers, the analysis of anything but the smallest data sets requires a statistical package. It is almost inconceivable that researchers should test statistical hypotheses by hand. Although not as pervasive, increasing use is also being made of analysis software such as Ethnograph and NU'DIST in qualitative research. As well as using software for analysis, students must be made aware of some of the pitfalls of its uncritical use. A particular danger is that the existence of the software will drive research design and data collection. Some researchers are warning against the homogenizing effects of this software on data collection and analysis. For example, Coffey, Holbrook, and Atkinson (1996) argue that developmental trends for such software tend to incorporate grounded theory strategies, and that use of this software may lead to a single rigid qualitative methodology based on the grounded theory approach. However, Coffey, Holbrook, and Atkinson also point out that, although coding of textual data and its analysis with the aid of computer software is important, computer software can be used for more than coding and retrieving textual data. In any case, computer software is widely used in qualitative research and must form a component of a research methods course so that students can be given the opportunity to use it critically.

A RESEARCH PROBLEM

What sort of research problem might lend itself to investigation with both qualitative and quantitative methodologies in a single semester course? Problems of this kind should draw on theory or a body of empirical evidence to formulate and test hypotheses, yet be enriched by taking a holistic approach--one in which context and linguistic description are vital. Hypothesis testing may determine that a treatment, or some characteristic, or some condition is instrumental in bringing about an effect. To stop the research at this point might be premature not to say frustrating. A number of other questions may arise from this finding. Why did a particular condition bring about the observed effect? Why is the observed effect much stronger in some participants than in others? To answer these questions requires more than an analysis of aggregate data. It may require intensive probing of individuals' attitudes as well as a sustained interaction with participants in their natural settings.

How can intense probing and sustained interaction with study participants be achieved? Any number of scenarios might be fruitful. What is required is that instructors are aware of ongoing research opportunities available within easy geographic reach. Local public libraries have diverse research needs and may welcome assistance from the academic sector. The librarian may already have identified one or more research problems but may not have the resources or expertise for a study. Other opportunities may be quite serendipitous--e.g., an instructor's research pursuit in one area may open an unexpected opportunity in another. The real point is that appropriate research opportunities abound if instructors are open to their local environment. An example of the often serendipitous nature of research problem identification is presented here.

While conducting research on linkages between public libraries and social service agencies for the provision of information and referral services (I & R), the instructor looked at some initiatives that had, over an extended period of time, provided I & R services to senior citizens. Through that avenue, he discovered that a large manufacturer of electronic telecommunications equipment had recently conducted a pilot marketing project in which a new kind of telephone receiver that incorporates an LCD screen and a qwerty keyboard was distributed to residents of a local community. The new device could be used as an ordinary telephone but also as an information access device. Using the device in this way, residents could dial into a number of online services such as banking, a community activities database, and e-mail.

Here was an obvious opportunity for a study that could involve students and employ both quantitative and qualitative methods. The phone device was distributed to all residents in the community but, because of the instructor's research focus on services to the senior population, this group's inclusion was of particular interest. A local organization for seniors, involved in providing support services to senior citizens, had invited them to a series of meetings to explain the technology. A mailing list of 300 senior citizens who lived in the area was available. This list could be, and was, used to draw a sample of participants for a study. But what kind of a study?

Since the device was distributed to the entire community free of charge, a number of interesting questions could be asked. Do all residents share the same experience in using the device? It was hypothesized that senior citizens' experiences with this technology are quite different from that of other residents. Although they might gain substantial benefits from this information technology, seniors, being in general less adept with digital communications technologies, may be less likely to adopt the device. Those who do adopt the device may face special difficulties in using it. Again, it was hypothesized that the small, rather dim, screen and the small keyboard present special problems for at least some seniors. These hypotheses were certainly testable with an appropriate quantitative methodology. A random sample of 30 seniors was drawn from the list of 300, and data were collected through mail questionnaires. Given this scenario, students were able to conduct all phases of a quantitative study--design, hypothesis formulation, data collection, and analysis. They learned and applied sampling techniques, questionnaire construction, how to maximize return rates, how to code and analyze data, and how to report and interpret results.

Ideally, students should gain hands-on experience in the entire "between methods" research process within a study. However, due to the academic semester time constraints, this was not feasible. Integrating several methods within a single semester course required that a number of activities occur in parallel. Assigning students to research teams allowed that to take place as each team conducted a separate facet of the study. So that all students gained insights into the process, teams reported progress and problems to the class on a weekly basis.

It is clear that, other than for the literature review, the quantitative phase of the study can be conducted "in the classroom" and can easily be completed within one semester. Consequently, one student research team executed this facet of the study and reported its progress, including difficulties encountered. However, the quantitative team was concerned with reporting and analyzing aggregate results--e.g., what proportion of seniors have trouble reading the screen or, in testing hypotheses, do seniors have greater difficulty in reading the screen than do younger users? The quantitative team simply did not analyze data on individuals--e.g., why did this particular senior have problems with the screen and what, in particular, was causing her these problems? While quantitative studies are valuable tools for obtaining answers to specific questions, they do not, by their very nature, deal well with surprising unanticipated results. That is because formulating testable hypotheses as well as determining the data-gathering instruments force researchers to specify an anticipated result before they conduct the study.

How was this study enlarged so that a deeper, more holistic, understanding of the phenomena involved could be attained? Two other student research teams addressed these facets of the problem. Both teams focused on methods of data collection and analysis that were far less constrained than the mail survey. One team conducted loosely structured individual interviews with four seniors using open ended questions. However, this method, while getting at individual opinions, only established what seniors remembered about their experience or what they chose to reveal. They took no account of peoples' inability to remember past experiences accurately or at all. One way to enhance memory of experiences is to have people recall them in a group setting where members of the group share somewhat similar experiences. Therefore, in addition to individual interviews, this research team conducted a focus group interview composed of selected seniors. Eight senior citizens volunteered to participate and were invited to attend a focus group session lasting approximately two hours. The purpose of the meetings was to have participants recount past experiences with the device in a setting where comments by some participants might cause recall of similar experiences in others. Individual interviews and focus group sessions were held in a meeting room of the local public library, and the proceedings were recorded for later transcription and analysis as well as for sharing with the other student research teams. This approach gave students valuable experience in recording and analyzing large amounts of textual data.

The course instructor facilitated the focus group proceedings. The purpose of the facilitator was not to control the agenda but to encourage discussion among the participants. However, participants were asked to focus the discussion on information services--their value, their variety, and their utility--as well as on the hardware itself, its functionality, and usability.

The third research team observed what seniors actually do when using the device, rather than expressions of feelings and opinions when they are removed from it. Invitations were sought from selected seniors to allow students to visit their homes and observe a session during which seniors operate the device as they would normally. Three seniors volunteered to participate in this phase of the study. Since observations were in a natural setting, session times were flexible, depending on the length of the task undertaken by the senior. Student researchers took detailed field notes of their observations. As was the case for the second research team, students in this team also gained valuable experience in recording and analyzing large amounts of textual data.

What we see here is a study that triangulated between the two methodological paradigms and, in the case of research teams 2 and 3, also triangulated within a methodological paradigm. Clearly, because of the time constraints, each student did not have the opportunity to apply every method used in the study. However, through weekly team reports and class discussions of methods and problems encountered, every student developed an understanding of how the two paradigms can complement one another and how triangulation, both within and between methods, can extend what is learned about a research problem.

As is often the case, the opportunity for this study came quite unexpectedly. It underscores the importance for instructors of methods courses to be aware of circumstances in their communities that offer prospects for research. This study offers opportunities for further research. Because only a small number of the 300 seniors in the population pool were sampled, the study can be replicated one or more times. This will give students in later methods courses another research perspective--i.e., what can be learned from studies that essentially replicate earlier studies.

Because of the time constraint, not all data gathered were analyzed thoroughly. Being a first time effort, this study also suffered some problems of planning and execution. Consequently, few substantive conclusions could be drawn. However, the principal purpose of the study was to provide a platform for learning and evaluating methods rather than reaching substantive conclusions, and it was able to achieve that goal.

CONCLUSION

The scenario outlined here is but one of many relevant to LIS studies. It allows students to investigate methods that can be applied to the exploration of relationships between people and information systems--to focus on peoples' interactions with both hardware and the system interface. As libraries' tools and information resources become increasingly digital, library professionals can ill afford to ignore or to misinterpret research in those areas. LIS educators have an obligation to ensure that their graduates are competent consumers of research. Yet, the master's degree from the majority of schools of library and information science is, and will likely remain, a thirty-six credit hour degree. Were it to be expanded

to a forty-two or even a forty-eight credit hour degree, it is unlikely that an additional research methods course would be added to the core curriculum although additional courses would be beneficial. Integrating the teaching of both quantitative and qualitative methods in one course can have substantial benefits. It allows students to explore research problems from multiple perspectives and to evaluate critically the strengths and weaknesses of each methodology.

It is obvious that the scenario outlined here excludes many methods currently used by quantitative and qualitative researchers. Although some LIS graduates will initiate research projects immediately in their professional lives, for many of our graduates, research is principally a product to be evaluated rather than a process they themselves will undertake. While there is great value in learning by doing, the doing need not be overly broad. Consequently, instructors can be quite modest in the variety of methods taught, concentrating instead on critical evaluation of the underlying principles that govern qualitative and quantitative research.

REFERENCES

Berg, B. L. (1998). Qualitative research methods for the social sciences, 3d ed. Boston, MA: Allyn & Bacon

Catterall, M., & Maclaran, P. (1997). Focus group data and qualitative analysis programs: Coding the moving picture as well as the snapshots. Sociological Research Online., 2(1). Retrieved April 20, 1998 from the World Wide Web: http://www.socresonline.org.uk/socresonline/2/1/6.html

Cavan, S. (1966). Liquor license: An ethnography of bar behavior: Chicago, IL: Aldine Publishing Co.

Chenail, R. J. (1992). Qualitative research: Central tendencies and ranges. Qualitative Report, 1(4). Retrieved June 29,1998 from the World Wide Web: http://www.nova.edu/ssss/QR/QRI-4/clinqual.html

Coffey, A.; Holbrook, B.; & Atkinson, P. (1996). Qualitative data analysis: Technologies and representations. Sociological Research Online, 1(1). Retrieved April 20, 1998 from the World Wide Web: http://www.socresonline.org.uk/socresonline/1/1/4.html

Creswell, J. W. (1994). Research design: Qualitative & quantitative approaches. Thousand Oaks, CA: Sage.

Denzin, N. K. (1978). The research act: A theoretical introduction to sociological methods, 2d ed. New York: McGraw-Hill.

Fidel, R., & Soergel, D. (1983). Factors affecting online bibliographic retrieval: A conceptual framework for research. Journal of the American Society for Information Science., 34(3), 163-180.

Greene, J. C.; Caracelli, V.J.; & Graham, W. E. (1989). Toward a conceptual framework for mixed-method evaluation designs. Educational Evaluation and Policy Analysis, 11(3), 255-274.

Jick, T. D. (1979). Mixing qualitative and quantitative methods: Triangulation in action. Administration Science Quarterly, 24(4), 602-611.

Kaplan, B., & Maxwell, J. A. (1994). Qualitative research methods for evaluating computer information systems. In J. G. Anderson, C. E. Aydin, & S. J. Jay (Eds.), Evaluating health care information systems: Methods and applications (pp. 45-68). Thousand Oaks, CA: Sage Publications.

Lincoln, Y. S., & Guba, E. G. (1985). Naturalistic inquiry. Beverly Hills, CA: Sage.

Mintzberg, H. (1979). The structuring of organizations: A synthesis of the research. Englewood Cliffs, NJ: Prentice-Hall.

Mintzberg, H. (1983). Power in and around organizations. Englewood Cliffs, NJ: Prentice Hall.

Morse, J. M. (1991). Approaches to, qualitative-quantitative methodological triangulation. Nursing Research, 40(2), 120-123.

Orlikowski, W. I., & Baroudi, J. J. (1991) Studying Information technology ill organizations: Research approaches and assumptions. information Systems Research, 2, 1-28.

Schall, E. (1996). Learning to love the swamp: Reshaping education for public service. Metropolitan l Universities: An International Forum 7(3), 73-84.

Tesch, R. (1990). Qualitative research: Analysis types and software tools. New York: Falmer Press.