Printer Friendly

Faculty opinions on the use of master's degree end of program assessments.

Introduction

The majority of Library and Information Studies (LIS) master's degree programs require students to successfully complete an end-of-program assessment (EPA) to receive a master's degree. Other LIS master degree programs do not use EPAs to determine whether a student earns a master's degree. Some EPAs may assess students' knowledge and abilities more directly than others and there is a wide range of opinions among LIS faculty on the use and value of these assessments. This study examines faculty opinions of the advantages and disadvantages of different EPAs. It also reports preferences for the use of EPAs in LIS education.

Background

Final assessments are standard in many disciplines. Comprehensive exams and theses or dissertations have been part of graduate education in the United States for at least 150 years (Storr, 1953). Portfolios have become popular more recently.

Master's degree requirements are variable both across disciplines and across universities (Berelson, 1960). While U.S. doctoral degree components became standardized in the 1930s to generally include coursework, comprehensive exams, two foreign languages, dissertation, and oral exam (Schafer & Giblin, 2008), a similar process has not standardized Master's degrees. While some Master's programs focus on research and discipline-specific knowledge acquisition, the Council of Graduate Schools reported that the majority are practice-oriented professional programs (Council of Graduate Schools, 1994). Whether an EPA is a required component of master's education may be dictated by university requirements or accreditation standards (Oehrtman, Smolen, Hoblet, & Phillips, 2010).

While few studies on EPAs exist in the LIS literature, there are several such studies in the literature of other fields. Many of the papers reviewed pointed to Bloom's Taxonomy of Educational Objectives as a rubric for designing effective EPAs, particularly the cognitive aspects of the taxonomy (Fitch, Reed, Peet, & Tolman, 2008; Loughead, 1997; Ponder, Beatty, & Foxx, 2004; Schafer & Giblin, 2008). For LIS education, Latrobe and Lester (2000) reported that there was a variety of EPA options including theses, capstone courses, various kinds of projects or papers, and portfolios, with about 40% of programs requiring comprehensive exams. Following, are brief lists of advantages and disadvantages of EPAs as gathered from the literature.

Portfolios

Portfolios are relative newcomers to graduate education and are hailed as a means to spur self-reflective analysis by students. In Latrobe and Lester's (2000) piece on the adoption of portfolios at University of Oklahoma, they reported that, by 1999, there were four LIS programs using portfolios. By 2010, there were 15 LIS programs doing so (Wallace & Nai doo, 2010).

Not all portfolios are the same and different types result in different products and learning experiences. Fitch, Reed, Peet, and Tolman (2008) identify the following portfolio types: Assessment or evaluative, reflective, integrative, structured, process or learning, and showcase or professional. Additionally, they can be paper or electronic.

Portfolios have a number of advantages. Unlike other EPAs they offer a longitudinal assessment that represents the educational process (Fitch, Reed, Peet, & Tolman, 2008; McNamara & Bailey, 2006; Prus & Johnson, 1994; Ryan, 2011). They also illustrate multiple components like writing, critical thinking, and ability to integrate knowledge (Fitch, Reed, Peet, & Tolman, 2008; McNamara & Bailey, 2006; Prus & Johnson, 1994; Thyer, 2003) and demonstrate student achievement of program objectives and competencies (Latrobe & Lester, 2000; McNamara & Bailey, 2006; Ryan, 2011; Thyer, 2003). They teach self-reflection (Fitch, Reed, Peet, & Tolman, 2008; Latrobe & Lester, 2000; Ryan, 2011; Scott, 2010). Reputed to be more related to professional expectations than other EPAs (Fitch, Reed, Peet, & Tolman, 2008; Mc Namara & Bailey, 2006; Prus & Johnson, 1994; Ryan, 2011; Thyer, 2003), they give a more accurate picture of abilities and potential success in a career (McNamara & Bailey, 2006). They can also be useful as a product to show future employers (Ryan, 2011). Offering a degree of flexibility (Latrobe & Lester, 2000; Thyer, 2003), they minimize test anxiety and other one-shot measurement problems (Prus & Johnson, 1994; Thyer, 2003). Lastly, portfolios help faculty get a broad picture of the program (Fitch, Reed, Peet, & Tolman, 2008; McNamara & Bailey, 2006; Prus & Johnson, 1994).

Several disadvantages to portfolios were also given. Reliability and validity might be issues in grading (Ryan, 2011) and there may be subjective, uneven standards across committees because of qualitative evaluation (McNamara & Bailey, 2006; Thyer, 2003). Students may feel uncertain about what the portfolio entails (Ryan, 2011) and the process is time consuming and laborious (McNamara & Bailey, 2006; Neumann, 2009; Prus & Johnson, 1994; Ryan, 2011). Thyer (2003) reported when his graduate program switched to portfolios, each successive class felt it needed to exceed the efforts of previous cohorts which increased expectations of students and put a heavier grading load on faculty.

Comprehensive Exams

Issues with comprehensive exams in graduate education were discussed in the academic literature of the 1930s and 1940s and again in the last decade. Concern with appropriate design and grading of exams was central to this discussion. An early study found that grading of exams did not demonstrate inter-rater reliability and thus the validity of the exam as an assessment tool was questioned (Pressey, Pressey, & Barnes, 1932). Beck and Becker (1969) also raised doubt about the reliability and the usefulness of comprehensive and qualifying exams for graduate students. Schafer and Giblin (2008) emphasized that in order to design effective exams it is important to clearly define the objectives for the exams to address. They stated, "The extent to which coherent educational objectives, as opposed to ritual and tradition, govern comprehensive examination practices remains unclear" (p. 276).

The advantages of comprehensive exams listed in the literature fell into several categories. Most of the advantages concerned student knowledge and skills that the exam responses illustrated. For example: Demonstrates mastery of knowledge, literature, and research in a discipline (Lindquist, Wortman, & Francis, 2011; Loughead, 1997; Mawn & Goldberg, 2012; Oehrtman, Smolen, Hoblet, & Phillips, 2010; Ponder, Beatty, & Foxx, 2004; Shafer & Giblin, 2008; Thyer, 2003); requires students to integrate, analyze, and synthesize knowledge (Anderson, Krauskopf, Rogers, & Neal, 1984; Loughead, 1997; Oehrtman, Smolen, Hoblet, & Phillips, 2010; Ponder, Beatty, & Foxx, 2004; Shafer & Giblin, 2008); fosters creative, critical thinking (Ponder, Beatty, & Foxx, 2004.); hones problem solving skills (Anderson, Krauskopf, Rogers, & Neal, 1984; Loughead, 1997; Ponder, Beatty, & Foxx, 2004); assesses students' basic skills and abilities (Shafer & Giblin, 2008). Other advantages concerned what students gain by experiencing the process: Imparts a sense of mastery (Anderson, Krauskopf, Rogers, & Neal, 1984); prepares students for scholarly, academic life (Ponder, Beatty, & Foxx, 2004.); rite of passage (Anderson, Krauskopf, Rogers, & Neal, 1984; Loughead, 1997; Ponder, Beatty, & Foxx, 2004; Shafer & Giblin, 2008). An additional advantage for both students and faculty is that exams are less time consuming than a thesis or professional paper (Oehrtman, Smolen, Hoblet, & Phillips, 2010). Exams also function for the department to weed out weak students (Anderson, Krauskopf, Rogers, & Neal, 1984; Shafer & Giblin, 2008) and give feedback about effectiveness of the program (Anderson, Krauskopf, Rogers, & Neal, 1984).

Disadvantages of comprehensive exams listed in the literature concern problems with the exams themselves such as potential issues with reliability and validity (Beck & Becker, 1969; McNamara & Bailey, 2006; Pressey, Pressey, & Barnes, 1932) and a vagueness of purpose (Anderson, Krauskopf, Rogers, & Neal, 1984). Disadvantages for students include the considerable study time and effort (Ponder, Beatty, & Foxx, 2004), vague guidance on how to effectively prepare (Anderson, Krauskopf, Rogers, & Neal, 1984), and the high level of stress and anxiety generated by the process (Anderson, Krauskopf, Rogers, & Neal, 1984; Bartle & Brodwin, 2006; Ponder, Beatty, & Foxx, 2004).

Theses and Dissertations

The traditional academic purpose of theses and dissertations is to provide guided research practice to graduate students and to contribute to knowledge in the field (Thomas & Brubaker, 2000). Regardless of the long-standing use of theses and dissertations in graduate education, few articles were found examining their pedagogical value as EPAs.

A major research and writing project such as a thesis or dissertation is a traditional rite of passage in graduate education. It gives students practice with many of the expectations common in academia such as choosing a topic, managing a self-directed project, reviewing the literature of the field, conducting research, and writing research reports (Benor & Phillips, 2009). It also functions as the final identifiable milestone for a degree (Hamilton, Johnson, & Poudrier, 2010).

Disadvantages of theses and dissertations are that their subjective grading may compromise reliability and validity (Hamilton, Johnson, & Poudrier, 2010) and they are labor intensive for faculty and students (Oehrtman, Smolen, Hoblet, & Phillips, 2010). They also require faculty to have graduate standing (Oehrtman, Smolen, Hoblet, & Phillips, 2010) and may not reflect skills learned in the program (Hamilton, Johnson, & Poudrier, 2010).

This Study

Faculties continually review and revise master's degree programs in an attempt to keep the educational experience relevant and up to date with current needs and expectations. Such decisions are generally based on the professional experiences and opinions of the faculty members. While each department can draw on the expertise of its own faculty, the value of this study is that it aimed to collect professional opinions about EPAs from the faculty base of the ALA-accredited schools of LIS in North America, thus presenting a broader opinion and experience base than a single faculty for considering EPAs. Such a collection of professional opinions may provide useful data for informing decision-making in LIS master's programs. This study queried respondents about their experiences with comprehensive exams, theses, portfolios, internships/practicums, design or research projects (other than thesis), scholarly papers (other than thesis), capstone courses, and other types of EPA to be defined by the respondent.

In 2009-2010, the Graduate Studies Committee at the University of Oklahoma SLIS discussed changing the format of the master's comprehensive exam. Discussion turned to questions about the use of EPAs in LIS education which became the impetus for this study. The purpose of the study was to collect information on LIS faculty members' professional experiences and views concerning types of end of program assessments. Base study questions included:

1. What do LIS faculty members see as the advantages and disadvantages of different types of end of program assessments?

2. Do characteristics of faculty such as rank and area of PhD impact their opinions on EPAs?

Methodology

The survey for this study was developed by combing through numerous administrative documents from different disciplines and different universities for assertions about the advantages and disadvantages of different EPAs. These statements were synthesized to create typologies that were used to create closed-ended answer categories. Acknowledging that these categories were not exhaustive, each question also had an open-ended "other" response for respondents to contribute additional thoughts.

Names and email addresses for faculty were obtained from the websites of ALA accredited LIS programs excluding Puerto Rico and Quebec for language reasons. In February 2011, all faculty members listed on these programs' websites were emailed an invitation to participate in an electronic survey posted to Survey Monkey. Two weeks later a follow-up email was sent to generate additional responses. Of the total number of invitations sent, 37 either bounced or we received a response to remove them from the list as they were no longer involved in LIS education. This resulted in 961 invitations. A total of 125 faculty members responded to the survey, a 13.0% response rate. One respondent filled out the survey, but indicated in the initial question that they did not give consent for their responses to be used. Therefore, that respondent was deleted from the dataset. Respondents were asked to complete sections of the survey that related to their experience. Consequently, not all respondents filled out all sections.

Responses were downloaded into SPSS for analysis and are described below using descriptive statistics. Since the population was surveyed rather than a random sample, inferential statistics are not appropriate for this study and were not used. Limitations of the study concern the lack of generalizability of the results. While the entire population of the LIS faculty was surveyed, the response rate was low. No assumption can be made that the opinions of those who did respond is similar to the opinions of those who did not.

Findings

Respondents had been teaching from 1 to 48 years with a mean of 14 and a median of 11 years. The majority had doctorates in LIS/IS (63.8%) while 27.6% had doctorates in other fields. An additional eight did not have doctorates. As to rank, 30.0% were full professosr, 26.0% were associates, and 36.0% assistants, with four Professor Emeriti, two administrators, two instructors, and two adjuncts.

A key question of this study was, "If you had to choose one type of master's degree end of program assessment as your top preference, which would it be?" One-third of respondents chose portfolio (36.7%) with most of the other EPA choices receiving 10 to 12.2% of the votes. Preference varied by faculty rank with half of full professors compared to 29% of assistant professors choosing portfolio. Assistant professors were somewhat more likely to choose research (thesis, scholarly paper, research projects) and fieldwork (internship, practicum) EPAs than associate and full professors.

The field of the respondent's doctorate was also related to EPA top choice with LIS/IS PhDs being somewhat more likely to choose research and experience EPAs while PhDs in other fields were much more likely to choose comprehensive exams. Portfolio was still the most popular choice for both groups.

For those respondents who were unable to choose a top preference for an EPA, most said that they support having options so students can choose the EPA that best fits their own goals and abilities. Some thought that completion of coursework should be enough for students to receive the master's degree if the state, college, or university does not require an EPA. Others support the use of multiple EPAs such as exams and a paper, a portfolio and an internship, or a capstone course that includes projects and scholarly papers.

Portfolios

Although the closed-ended answer categories were gleaned from professional documents, respondents did not agree with most of the statements. They were most likely to agree that the finished portfolio can be used in job seeking, and the portfolio fosters self-reflection, thinking in terms of goals and objectives, and synthesis of learning experiences. Fewer than half of respondents agreed with the majority of advantages and disadvantages listed.

Advantages listed by respondents in open-ended responses were both program-related and student-related. Several respondents thought that portfolios help faculty assess instruction, diagnose problems with the program or courses and evaluate whether program competencies are being met. For students, it taps into skills such as creativity, ability to choose among products to include, and technology skills. One respondent stated, "I don't see any negatives with this approach."

Concerning disadvantages, several respondents thought that the resulting product was "so what" and held no real value, nor was the experience valuable as it did not teach skills or require critique. Other disadvantages mentioned included issues with technical skills, difficulty keeping students on track, grading difficulties, time-consuming for faculty and students and some faculty were not engaged in the process. One person noted, "No matter how clear the instructions, some students will misinterpret them."

Quite a number of respondents made general comments about portfolios. For many, the portfolio was a new option in their program or their process was under revision. There was clear disagreement about the potential for students using their portfolios as a job seeking tool. Many commented that the portfolios are useful in job seeking, but others stated "prospective employers are not interested in seeing them." One declared, "Although the argument is usually made that a portfolio can be used for job seeking, as one who has been active in libraries for most of her professional career, I have never asked to see a portfolio nor do I think I would do so. I think this argument points out one of the difference in perspectives between those who teach and those who do [not]." Several respondents discussed how the design of the portfolio is key to the value of the outcome. Their concerns were that good design is vital. Another noted that the "main advantage to the student is the process rather than the product".

In commenting why portfolios were their top choice, most respondents listed reasons having to do with students, both what it requires of students and what students take away from it. The most common reason given was that it requires students to reflect on their program experience. Additionally, students have to discriminate in their selection of items to include and integrate knowledge across courses. It demonstrates to students and faculty growth over time and compares student process to learning outcomes. The portfolio is flexible and is a professional-type activity blending or bridging academic and professional skills. Many respondents stated that it could be used in job seeking either as a tool in and of itself or in how it prepares students to talk about their skills in oral presentations. It is realistic, "shows depth and range," and is "appropriate for a variety of students." It is comprehensive. Two additional respondents noted that it can be used for program assessment and accreditation.

Comprehensive Exams

For the listed advantages of comprehensive exams (comps), about half to two-thirds of respondents agreed that standardization across students and terms was an advantage. For disadvantages, around half agreed that some students do not test well and the exams do not measure student ability to succeed in the profession. Fewer than half of respondents agreed with any of the other advantages or disadvantages listed for comps.

In open-ended responses several people mentioned that exams require students to integrate, synthesize, and summarize knowledge acquired in the program, and exam outcomes and quality of responses help faculty evaluate the program and suggest curricular modifications. Other advantages mentioned were that exams allow students to demonstrate writing skills and professional knowledge, they are a rite of passage, and the stress of preparing for the exams mirrors stressful situations they will face as professionals. One stated that there were no disadvantages to comps.

In the open-ended answers for "advantages" of comps, five respondents stated "none" or expressed that they did not support the use of comps. One stated that they "cannot think of any pedagogically sound reason" to use comps. For the "disadvantages" open-ended question, several respondents indicated issues with administering the exam: the need to develop different question sets each term, and location and timing-related problems. Other responses criticized the value of comps: they are "not typically 'comprehensive'," "distill a complex field down to a pitiful exam," the "length of time" a student is in the program can "make it difficult to recall concepts." One person stated, "There are better forms of assessment that fit into an overall learning process".

General comments about comps fell into four categories. First, a number of people stated that they had had good experiences with comps. Next, people commented about the design and administration of the exam. While one stated that a plus is that all faculty were involved in the design, another noted that not all faculty actively contribute to the questions. Two said it was easy to administer and one stated that it was "extremely time consuming and stressful for faculty". Next, respondents criticized the essence of comps, stating that they do not adequately measure student success in the program and they are a "less-than-optimal pedagogical method." The last category concerned the student experience. Comments included: It is an equal experience for students, objective, gives students a chance to show mastery, synthesize knowledge and demonstrate critical thinking and writing ability. It is the "least drawn out of other options. Get it over with".

Theses

Respondents were very likely to agree with the listed advantages of theses. Over 80% agreed that theses provide experience in research methods and can result in a publication and around three-quarters agreed with the other advantages such as preparation for doctoral studies, gain familiarity with literature on a topic, and improving writing and critical thinking skills. Concerning stated disadvantages, half to two-thirds of respondents agreed that they are very time consuming for faculty and students and can delay graduation. One respondent commented that these listed items were not really disadvantages, but merely excuses.

In the open-ended responses people stated that theses allow students to focus on an area of personal interest, facilitate their ability to interact with researchers, practice applying theories, collecting evidence, and complex thinking. One stated that a thesis "gives the student the self-confidence to tackle a challenging long-term project". One respondent noted that the valuable feedback students receive in thesis work is worth the effort. A number of respondents emphasized that doing a thesis should be optional and not required as it is not appropriate for all students. Many stated there are no meaningful disadvantages to theses if they are optional.

Open-ended responses concerning disadvantages of theses included several comments that theses are not really EPAs and do not demonstrate knowledge of LIS, students are often unprepared to do research, and there is not much time particularly for studies needing Human Subjects approval. Several also lamented the possibility of mediocre theses devaluing the MLIS degree, research done for theses "usually isn't actually useful," the thesis process does not guarantee a good outcome, and some students perform badly. Also mentioned was that the process may work well if a professor has a few theses to supervise, but does not work well with many. One respondent stated that their school discouraged theses as faculty members did not get credit for their supervisory work. Under advantages of theses as EPAs, one respondent said, "none."

For those respondents that chose theses as their top preference, most focused on the skills students hone in the process: writing, thinking, research, analysis, summarization, and mastery of a specialty area. A few others focused on how these skills strengthened the profession. One stated that theses are "standard in graduate education" and "universally accepted in academia". Both quantitative and qualitative analysis showed that respondents had very positive attitudes towards theses. They just felt that a thesis should be an option and not a required EPA.

Field Placements

The majority of respondents were in agreement with all of the advantages for field placements such as internships and practica listed in the survey. Nearly all agreed that practical work experience is valuable on a resume. Another 90% thought that the experience can help students decide if they really want to work in the area, and that experience-based EPAs apply knowledge and skills gained in coursework in real-world settings. For listed disadvantages, around three-fourths of respondents agreed that disadvantages to field placements are that some site supervisors create better learning experiences for their students than others, and field placements are difficult for students who already work full-time.

Advantages offered by respondents included: Introduces students to the culture of the profession, develops practical skills, helps build relationships between the school and the industry, and may be a source of recommendation letters for students. Several responded that in their programs fieldwork is not considered an EPA, for many it is an optional course that students can take. One respondent supported field placements "as part of a basket of measures and not as a single measure". Some respondents noted that in their program practica are required, but not considered EPAs, another stated that all students should do a practicum, but not necessarily as an EPA. Concerning the use of field placements as EPAs, some faculty noted that they do not assess program achievement, integrate learning, or require reflection from students. Additionally "with full-time working students: it just doesn't work".

The majority of respondents who chose internships and practica as their top EPA preferences cited the practical, real-world, professional experience. Some noted that such experience is "clearly important in a professional program." The experience helps students move from theory to practice. In addition to the practice itself, other useful aspects included basic abilities to function in a workplace such as appropriate social skills and the development of realistic career expectations.

Other EPAs

There was some EPA overlap, particularly concerning capstone courses. For several respondents, the capstone course included the creation of a scholarly or research paper, poster presentation, research project, analytic project, or design project. In one case, the capstone course was a research methods course. In another program, students had to identify a client and design a project to solve a problem for them. In some cases, students created their portfolios in a capstone course. Poster presentations included online presentations, presentation at an actual conference, and presentation at an in-house show or research symposium. Capstone courses were non-credit at some institutions and one to three credits at others. Time frames for the capstone course ranged from one-half term or short summer term to two full terms.

Common expectations of capstone courses expressed in the comments included individual and collective reflection, integration of knowledge across courses, synthesis of experiences, and the fostering of professional competency. These outcomes are arrived at through a variety of means including writing, discussing, and creating projects. Advantages of capstones given by respondents included: "expectations are standardized but the content ... can vary," "meets new state system of higher education requirements", "reduces the granularity of the topics students examine". Disadvantages were related to excessive time commitments, lack of student preparation or aptitude necessary for meaningful reflection and buy-in from the faculty and program as a whole.

For some respondents, the "assessment" aspect of "end-of-program assessments" was a sticking point. For these respondents, culminating experiences such as capstone courses, portfolios, theses, field placements, and projects were not EPAs because there was no assessment of student learning the way one might have with a comprehensive or oral exam. In other cases, some activities such as theses and field placements were optional choices that students did not have to complete in order to graduate. Concerning choices, many respondents stated that students had options in their programs for type of culminating experience, some had to do more than one, and sometimes student concentration dictated the choice such as school library media students having to do a portfolio.

On the Use of EPAs

Respondents were asked if their program uses EPAs. A majority, two-thirds, required an EPA as part of the master's degree. However, a sizable minority of one-quarter of programs did not have an EPA requirement and 9.4% stated some degree tracks required EPAs, while others did not.

Several respondents used to have a required EPA, but now did not. For others, they had not been requiring one, but were going to start. Still others said they used to require one type but had switched to another. These comments suggest that EPA requirements are fluid over time, perhaps as faculty examine their pedagogical purposes, and as governing and accrediting bodies change their requirements.

Not all faculty members support the use of EPAs. 31 respondents felt that LIS master's programs should not have, or do not need, EPAs. The most commonly cited reason for not supporting EPAs was that course work is a sufficient measure for student success. Time issues were the next most common reason: not enough time in the master's program to include an EPA, and too much additional work for faculty.

In the open-ended remarks about not having EPAs, respondents mentioned time pressures and problems with faculty buy-in. One respondent did not believe EPAs serve any useful purpose, while another declared that "assessment activity is anti-intellectual conservative corporatization of education". One stated that "evaluation metrics" should be in place throughout the program, and if they are not, an EPA is "not a viable alternative". About half of the comments in this section were actually in support of EPAs, such as, "I believe they are needed," and, "It is good to have some type of EPA". One opinion was that coursework-only programs can become paper mills. Two responses viewed the EPA discussion from a wider lens. "In an age of accountability, it may be the most effective and efficient way to demonstrate what a graduate can do upon graduation". Another said, "The fact is that accrediting bodies require outcomes based assessment and such assessment is critical to considered program evaluation".

Several respondents discussed the politics of EPAs on a broader level. Some decried the EPA process as cumbersome and driven by external concerns such as accreditation or an effort by LIS schools to "justify and protect themselves from criticism". One pointed to current debates in the media and in state legislatures concerning "measures to prove that taxpayers got their money's worth". One suggestion was that, unless the EPA is standardized across accredited LIS schools, it is meaningless, while another was there should be an ALA librarian exam in lieu of EPAs. A criticism of EPAs was that they only provided a gatekeeping function and were not rigorous. Two respondents suggested that the only way to know how well the program is functioning is to assess the careers of graduates.

In contrast, several respondents recalled their own MLS experiences of graduating from programs without EPAs and they felt it was "anti-climactic" with "no real synthesis, summation or closure". One person stated about EPAs, "I wish we used them here," while another said their program tried to institute EPAs and were turned down by the university over faculty salary issues for faculty time spent on EPAs. Others reiterated that their university determined whether EPAs were required and what EPAs they could use.

Discussion and Conclusions

According to the 2010 ALISE Statistical Report the portfolio was the second most common of the "Special Requirements for Graduation with a Master's Degree" with 15 schools requiring it (Wallace & Naidoo, 2010, 259). Portfolio use has increased in LIS over the past decade and this may be related to the shift towards a requirement for outcomes-based measures by the infrastructures within which LIS programs function such as the American Library Association Committee on Accreditation, other accrediting bodies, and federal agencies such as the Institute of Museum and Library Services. In this study full professors were more likely than their more junior colleagues to choose portfolios as their preferred EPA. This might result from their recognition of the value of outcomes-based measures due to their greater experience with accreditation, funding, and administrative activity.

Comprehensive exams were the most common "special requirement" indicated by the ALISE report with 20 programs requiring them. Exams did have their proponents among the respondents in this study, but the majority was not satisfied with the value of the exams and few chose exams as their top preference for EPAs. While exams carry the weight of tradition in graduate education, their use has been questioned repeatedly over time.

The ALISE report showed that theses were required for a master's degree by 8 schools and are an option at 33. An additional 26 did not offer theses. While few respondents chose theses as their top EPA preference, overall comments concerning them were highly positive. Many were in favor of offering a thesis option for students who have the aptitude and desire to pursue one. Several respondents disagreed with the concept of a thesis as an EPA, instead casting them as an experience appropriate for some students but perhaps restricted by the time limitations of a master's program.

Fieldwork as a for-credit class was required by 27 schools per the ALISE report. Like the thesis, many respondents had very positive opinions about fieldwork, but they were unlikely to think of it as an EPA and many thought it should be an option rather than a requirement.

The lists of advantages and disadvantages in the survey were gleaned from administrative documents and most were echoed in the literature. Many of these items, however, were not supported by the faculty who responded to this study. Therefore, faculty responses to these items might be useful for those writing future administrative documents.

In general terms, respondents judged EPAs on their usefulness to students and their soundness as a pedagogical practice. They were concerned with how much faculty time was involved in various EPAs and many questioned whether the items listed were truly assessments. Some expounded on the purpose of EPAs in the broader picture outside of their departments or universities. What was most clear was the range of opinions and lack of consensus. A number of faculty were in favor of having coursework only with no required culminating assessment or experience. Of the ones who were in favor of having EPAs, many advocated having options so students could choose what best fit their skills and career plans.

As programs are continually revisiting, revising, and attempting to improve their program requirements such as EPAs, it may be useful to have some insight into professional opinions in a broader context than a single program. This study provides that. Future studies on LIS EPAs may benefit from conducting in-depth interviews with senior faculty and administrators who can share insights into historical contexts of how end of program requirements evolved at various schools. Since portfolios have become increasingly popular as EPA options or requirements, it would be useful to examine aspects of portfolio use such as why senior faculty are more likely to prefer them, and collect data from students and graduates about whether they found their portfolio experience to be a helpful tool in professional acculturation.

References

Anderson, W. P., Krauskopf, C. J., Rogers, M. E., & Neal, G. W. (1984). Reasons for comprehensive examinations: A re-evaluation. Teaching of Psychology, 11, 78-82. Retrieved from Academic Search Premier.

Bartle, L. & Brodwin, M. G. (2006). The comprehensive examination: Study groups and their effectiveness--A message to counselor education faculty and graduate students. Education, 127, 69-72. Retrieved from Academic Search Premier.

Beck, B., & Becker, H. S. (1969). Modest proposals for graduate programs in sociology. The American Sociologist, 4, 227-234. Retrieved from http://www.jstor.org/stable/27701504

Benor, S. B. & Phillips, B. A. (2009). Pedagogy and impact: The SJCJ master's thesis. Journal of Jewish Communal Service, 84, 228-231. Retrieved from Academic Search Premier.

Berelson, B. (1960). Graduate education in the United States. New York: McGraw-Hill Book Company, Inc.

Council of Graduate Schools. (1994). Master's education: A guide for faculty and administrators, a policy statement. Washington, DC: Council of Graduate Schools.

Fitch, D., Reed, B. G., Peet, M., & Tolman, R. (2008). The use of eportfolios in evaluating the curriculum and student learning. Journal of Social Work Education, 44, 37-54. Retrieved from MasterFILE Premier.

Hamilton, P., Johnson, R., & Poudrier, C. (2010). Measuring educational quality by appraising theses and dissertations: Pitfalls and remedies. Teaching in Higher Education, 15, 567-577. doi: 10.1080/13562517.2010.491905

Latrobe, K. & Lester, J. (2000). Portfolio assessment in the MLIS program. Journal of Education for Library and Information Science, 41, 197-206. doi:10.2307/40324069

Lindquist, J., Wortman, S., & Francis, C. (2011). Adding value to graduate education: The comprehensive exam. NACTA Journal, December, 106-107. Retrieved from Proquest.

Loughead, T. O. (1997). The doctoral comprehensive examination: Fine-tuning the process. Counselor Education & Supervision, 37, 140-149. Retrieved from Professional Development Collection.

Mawn, B. E. & Goldberg, S. (2012). Trends in the nursing doctoral comprehensive examination process: A national survey. Journal of Professional-Nursing, 28, 156-162. doi: 10.1016/j.profnurs.2011.11.013

McNamara, T. L. & Bailey, R. L. (2006). Faculty/ staff perceptions of a standards-based exit portfolio system for graduate students. Innovative Higher Education, 31, 129-141. doi: 10.1007/ s10755-006-9013-9

Neumann, R. (2009). Policy driving change in doctoral education: An Australian case study. In D. Boud & A. Lee (Eds.), Changing practices of doctoral education (pp. 211-224). London: Routledge.

Oehrtman, S. J., Smolen, D., Hoblet, K., & Phillips, K. A. (2010). The comprehensive examination: A viable master's of science in nursing capstone course. Journal of Professional Nursing, 26, 360-365. Retrieved from http://www.sciencedirect.com/science/article/pii/S8755722310000888

Ponder, N., Beatty, S. E., & Foxx, W. (2004). Doctoral comprehensive exams in marketing: current practices and emerging perspectives. Journal of Marketing Education, 26, 226-235. doi: 10.1177/0273475304268778

Pressey, S. L., Pressey, L. C., & Barnes, E. J. (1932). The final ordeal: Two minor experiments regarding reliability and validity of graduate examinations. The Journal of Higher Education, 3, 261-264. Retrieved from http://www.jstor.org/ stable/1974483

Prus, J. & Johnson, R. (1994). A critical review of student assessment options. In T. H. Bers & M. L. Mittler (Eds.), Assessment and testing: Myths and realities (pp. 69-83). San Francisco: Jossey-Bass Publishers.

Ryan, M. (2011). Evaluating portfolio use as a tool for assessment and professional development in graduate nursing education. Journal of Professional Nursing, 27, 84-91. Retrieved from http://www.sciencedirect.com/science/article/pii/ S8755722310001183

Schafer, J. A. & Giblin, M. J. (2008). Doctoral comprehensive exams: Standardization, customization, and everywhere in between. Journal of Criminal Justice Education, 19, 275-289. doi: 10.1080/10511250802137648

Scott, S. G. (2010). Enhancing reflection skills through learning portfolios: An empirical test. Journal of Management Education, 34, 430-157. doi: 10.1177/1052562909351144

Storr, R. J. (1953). The beginnings of graduate education in America. Chicago: University of Chicago Press.

Thomas, R. M., & Brubaker, D. L. (2000). Theses and dissertations: A guide to planning, research, and writing. Westport, CT: Bergin & Garvey.

Thyer, B. A. (2003). A student portfolio approach to conducting doctoral social work comprehensive examinations. Journal of Administration in Social Work, 23, 117-126. doi: 10.1300/ J067v23n03_10

Wallace, D. P. & Naidoo, J. (Eds.). (2010). Library and information science education statistical report 2010. Chicago: Association for Library and Information Science Education. Retrieved from http://www.alise.org/statistical-reports

Susan K. Burke

School of Library and Information Studies, University of Oklahoma, 401 W. Brooks, Old Bizzell Library, Room 120, Norman, OK 73019-6032, Email: sburke@ou.edu

John T. Snead

School of Library and Information Studies, University of Oklahoma, 401 W. Brooks, Old Bizzell Library, Room 120, Norman, OK 73019-6032, Email: jsnead@ou.edu
Table 1. Top EPA Preference by Faculty Rank.

                      Full Professor   Associate   Assistant
                            %          Professor   Professor
                                           %           %

Portfolio                  48.0          39.1        29.0
Research EPAs              16.0          17.4        25.8
Fieldwork EPAs             16.0          17.4        25.8
Comprehensive exams        12.0          17.4         6.5
Capstone course            8.0            8.7        12.9
Total                     N = 25        N = 23      N = 31

Table 2. Top EPA Preference by PhD.

                      LIS/IS PhD   Other PhD
                          %            %

Portfolio                33.3        44.0
Research EPAs            24.5        12.0
Fieldwork EPAs           22.8        16.0
Capstone course          12.3         4.0
Comprehensive exams      7.0         24.4
Total                   N = 57      N = 25

Table 3. Faculty Opinions on Portfolios as EPAs.

                                          f      %
                                               N = 69

Advantages
  The finished portfolio can be           54    78.3
    used in job seeking
  The portfolio process teaches           50    72.5
    students to be professionally
    self-reflective
  Requires students to think in           48    69.6
    terms of goals and/or objectives
  The portfolio requires students         47    68.1
    to synthesize their learning
    experiences from their master's
    program
  Reflective statements could give        29    42.0
    faculty information about student
    satisfaction with the program
  The portfolio process is similar to     25    36.2
    types of reports a professional
    in this field might write
  It is less stressful for students       22    31.9
    than some other types of EPAs
    (like comps)
  Highlights students' strengths and      21    30.4
    weaknesses
  The portfolio process improves          18    26.1
    critical thinking skills
  It is less time consuming for           13    18.8
    students than some other types of
    EPAs (like thesis)

Disadvantages
  Content may vary widely                 36    52.2
  The portfolio is very time consuming    32    46.4
    for faculty
  The structure is too open to            32    46.4
    interpretation
  Grading portfolios is too subjective    31    44.9
  The portfolio is very time consuming    20    29.0
    for students
  The structure is too rigid              4     5.8

Table 4. Faculty Opinions on Comprehensive Exams as EPAs.

                                                      f      %
                                                           N = 71

Advantages
  Fairly standardized across students                 47    66.2
  Fairly standardized across terms                    34    47.9
  Takes less total time for students                  26    36.6
  Does not take the place of an elective course       26    36.6
    (like a thesis often does)
  Takes less total time for faculty                   25    35.2
  Comprehensive exams are common across many          21    29.6
    disciplines
  Comprehensive exams are traditionally part of       21    29.6
    graduate education

Disadvantages
  Some students do not test well                      41    57.7
  Exams are not a good measure of student ability     40    56.3
    to succeed in the profession
  It is not a good measure of student success in      29    40.8
    the program
  It is overly stressful for the students             29    40.8
  It is too time consuming for the faculty            19    26.8
  It is too distracting for the students when they    17    23.9
    have other coursework concerns

Table 5. Faculty Opinions on Theses as EPAs.

                                             f      %
                                                  N = 60

Advantages
  Provides experience in research methods    53    88.3
  Can result in a publication                50    83.3
  Helpful if student decides to go on to     46    76.7
    a doctoral program
  Helps student become very familiar with    46    76.7
    the literature in a topic area
  Improves professional writing skills       45    75.0
  Improves critical thinking skills          45    75.0
  Opportunity to become highly               43    71.7
    knowledgeable in a specific topic

  Disadvantages
  Very time consuming for faculty            37    61.7
  Very time consuming for student            33    55.0
  Can delay student graduation               31    51.7
  Teaches skills that may not be relevant    23    38.3
    to many LIS jobs

Table 6. Faculty Opinion of Fieldwork as EPAs.

                                                       f      %
                                                            N = 71

Advantages
  Practical work experience is valuable on a resume    52    98.1
  The experience can help students decide if they      48    90.6
    really want to work in that area
  Applies knowledge and skills gained in coursework    48    90.6
    in real-world settings
  Builds students' confidence in their abilities       46    86.8
  Graduates are sometimes hired at the sites where     41    77.4
    they did their internship/practicum
  Helps point out areas in which the student needs     38    71.7
    to work on their knowledge or skills

Disadvantages
  Some site supervisors create better learning         40    75.5
    experiences for the students than others
  Difficult for students who are already working       38    71.7
    full time
  Poorly designed internships/practica may result      30    56.6
    in students spending most of their
    time doing non-professional work (like filing)
  Difficult to find an appropriate internship          21    39.6
  Requires a great deal of hours                       12    22.6

Table 7. Does Your School/College Require EPAs?

                                              f      %
                                                   N = 96

Yes, all students are required to complete    63    65.6
  an EPA to receive a master's degree
No, EPAs are not used to determine whether    24    25.0
  a student receives a master's degree in
  our program
It depends on which degree track students     9     9.4
  choose, some tracks require EPAs and
  others do not

Table 8. Opinions on Why Not Use EPAs.

                                f      %
                                     N = 96

Course work is a sufficient     19    61.3
  measure of student success
There is not enough time in     14    45.2
  a master's program to
  include an EPA
It is too much additional       14    45.2
  work for faculty
It is too much additional       10    32.3
  work for students
EPAs are not required by our    9     29.0
  Graduate College/University
EPAs are not a good measure     7     22.6
  of student success in the
  program
COPYRIGHT 2014 Association for Library and Information Science Education
No portion of this article can be reproduced without the express written permission from the copyright holder.
Copyright 2014 Gale, Cengage Learning. All rights reserved.

Article Details
Printer friendly Cite/link Email Feedback
Author:Burke, Susan K.; Snead, John T.
Publication:Journal of Education for Library and Information Science
Article Type:Report
Date:Jan 1, 2014
Words:7160
Previous Article:RDA: training and continuing education needs in academic libraries.
Next Article:Technology competency requirements of ALA-accredited library science programs: an updated analysis.
Topics:

Terms of use | Privacy policy | Copyright © 2020 Farlex, Inc. | Feedback | For webmasters