Faculty opinions on the use of master's degree end of program assessments.
The majority of Library and Information Studies (LIS) master's degree programs require students to successfully complete an end-of-program assessment (EPA) to receive a master's degree. Other LIS master degree programs do not use EPAs to determine whether a student earns a master's degree. Some EPAs may assess students' knowledge and abilities more directly than others and there is a wide range of opinions among LIS faculty on the use and value of these assessments. This study examines faculty opinions of the advantages and disadvantages of different EPAs. It also reports preferences for the use of EPAs in LIS education.
Final assessments are standard in many disciplines. Comprehensive exams and theses or dissertations have been part of graduate education in the United States for at least 150 years (Storr, 1953). Portfolios have become popular more recently.
Master's degree requirements are variable both across disciplines and across universities (Berelson, 1960). While U.S. doctoral degree components became standardized in the 1930s to generally include coursework, comprehensive exams, two foreign languages, dissertation, and oral exam (Schafer & Giblin, 2008), a similar process has not standardized Master's degrees. While some Master's programs focus on research and discipline-specific knowledge acquisition, the Council of Graduate Schools reported that the majority are practice-oriented professional programs (Council of Graduate Schools, 1994). Whether an EPA is a required component of master's education may be dictated by university requirements or accreditation standards (Oehrtman, Smolen, Hoblet, & Phillips, 2010).
While few studies on EPAs exist in the LIS literature, there are several such studies in the literature of other fields. Many of the papers reviewed pointed to Bloom's Taxonomy of Educational Objectives as a rubric for designing effective EPAs, particularly the cognitive aspects of the taxonomy (Fitch, Reed, Peet, & Tolman, 2008; Loughead, 1997; Ponder, Beatty, & Foxx, 2004; Schafer & Giblin, 2008). For LIS education, Latrobe and Lester (2000) reported that there was a variety of EPA options including theses, capstone courses, various kinds of projects or papers, and portfolios, with about 40% of programs requiring comprehensive exams. Following, are brief lists of advantages and disadvantages of EPAs as gathered from the literature.
Portfolios are relative newcomers to graduate education and are hailed as a means to spur self-reflective analysis by students. In Latrobe and Lester's (2000) piece on the adoption of portfolios at University of Oklahoma, they reported that, by 1999, there were four LIS programs using portfolios. By 2010, there were 15 LIS programs doing so (Wallace & Nai doo, 2010).
Not all portfolios are the same and different types result in different products and learning experiences. Fitch, Reed, Peet, and Tolman (2008) identify the following portfolio types: Assessment or evaluative, reflective, integrative, structured, process or learning, and showcase or professional. Additionally, they can be paper or electronic.
Portfolios have a number of advantages. Unlike other EPAs they offer a longitudinal assessment that represents the educational process (Fitch, Reed, Peet, & Tolman, 2008; McNamara & Bailey, 2006; Prus & Johnson, 1994; Ryan, 2011). They also illustrate multiple components like writing, critical thinking, and ability to integrate knowledge (Fitch, Reed, Peet, & Tolman, 2008; McNamara & Bailey, 2006; Prus & Johnson, 1994; Thyer, 2003) and demonstrate student achievement of program objectives and competencies (Latrobe & Lester, 2000; McNamara & Bailey, 2006; Ryan, 2011; Thyer, 2003). They teach self-reflection (Fitch, Reed, Peet, & Tolman, 2008; Latrobe & Lester, 2000; Ryan, 2011; Scott, 2010). Reputed to be more related to professional expectations than other EPAs (Fitch, Reed, Peet, & Tolman, 2008; Mc Namara & Bailey, 2006; Prus & Johnson, 1994; Ryan, 2011; Thyer, 2003), they give a more accurate picture of abilities and potential success in a career (McNamara & Bailey, 2006). They can also be useful as a product to show future employers (Ryan, 2011). Offering a degree of flexibility (Latrobe & Lester, 2000; Thyer, 2003), they minimize test anxiety and other one-shot measurement problems (Prus & Johnson, 1994; Thyer, 2003). Lastly, portfolios help faculty get a broad picture of the program (Fitch, Reed, Peet, & Tolman, 2008; McNamara & Bailey, 2006; Prus & Johnson, 1994).
Several disadvantages to portfolios were also given. Reliability and validity might be issues in grading (Ryan, 2011) and there may be subjective, uneven standards across committees because of qualitative evaluation (McNamara & Bailey, 2006; Thyer, 2003). Students may feel uncertain about what the portfolio entails (Ryan, 2011) and the process is time consuming and laborious (McNamara & Bailey, 2006; Neumann, 2009; Prus & Johnson, 1994; Ryan, 2011). Thyer (2003) reported when his graduate program switched to portfolios, each successive class felt it needed to exceed the efforts of previous cohorts which increased expectations of students and put a heavier grading load on faculty.
Issues with comprehensive exams in graduate education were discussed in the academic literature of the 1930s and 1940s and again in the last decade. Concern with appropriate design and grading of exams was central to this discussion. An early study found that grading of exams did not demonstrate inter-rater reliability and thus the validity of the exam as an assessment tool was questioned (Pressey, Pressey, & Barnes, 1932). Beck and Becker (1969) also raised doubt about the reliability and the usefulness of comprehensive and qualifying exams for graduate students. Schafer and Giblin (2008) emphasized that in order to design effective exams it is important to clearly define the objectives for the exams to address. They stated, "The extent to which coherent educational objectives, as opposed to ritual and tradition, govern comprehensive examination practices remains unclear" (p. 276).
The advantages of comprehensive exams listed in the literature fell into several categories. Most of the advantages concerned student knowledge and skills that the exam responses illustrated. For example: Demonstrates mastery of knowledge, literature, and research in a discipline (Lindquist, Wortman, & Francis, 2011; Loughead, 1997; Mawn & Goldberg, 2012; Oehrtman, Smolen, Hoblet, & Phillips, 2010; Ponder, Beatty, & Foxx, 2004; Shafer & Giblin, 2008; Thyer, 2003); requires students to integrate, analyze, and synthesize knowledge (Anderson, Krauskopf, Rogers, & Neal, 1984; Loughead, 1997; Oehrtman, Smolen, Hoblet, & Phillips, 2010; Ponder, Beatty, & Foxx, 2004; Shafer & Giblin, 2008); fosters creative, critical thinking (Ponder, Beatty, & Foxx, 2004.); hones problem solving skills (Anderson, Krauskopf, Rogers, & Neal, 1984; Loughead, 1997; Ponder, Beatty, & Foxx, 2004); assesses students' basic skills and abilities (Shafer & Giblin, 2008). Other advantages concerned what students gain by experiencing the process: Imparts a sense of mastery (Anderson, Krauskopf, Rogers, & Neal, 1984); prepares students for scholarly, academic life (Ponder, Beatty, & Foxx, 2004.); rite of passage (Anderson, Krauskopf, Rogers, & Neal, 1984; Loughead, 1997; Ponder, Beatty, & Foxx, 2004; Shafer & Giblin, 2008). An additional advantage for both students and faculty is that exams are less time consuming than a thesis or professional paper (Oehrtman, Smolen, Hoblet, & Phillips, 2010). Exams also function for the department to weed out weak students (Anderson, Krauskopf, Rogers, & Neal, 1984; Shafer & Giblin, 2008) and give feedback about effectiveness of the program (Anderson, Krauskopf, Rogers, & Neal, 1984).
Disadvantages of comprehensive exams listed in the literature concern problems with the exams themselves such as potential issues with reliability and validity (Beck & Becker, 1969; McNamara & Bailey, 2006; Pressey, Pressey, & Barnes, 1932) and a vagueness of purpose (Anderson, Krauskopf, Rogers, & Neal, 1984). Disadvantages for students include the considerable study time and effort (Ponder, Beatty, & Foxx, 2004), vague guidance on how to effectively prepare (Anderson, Krauskopf, Rogers, & Neal, 1984), and the high level of stress and anxiety generated by the process (Anderson, Krauskopf, Rogers, & Neal, 1984; Bartle & Brodwin, 2006; Ponder, Beatty, & Foxx, 2004).
Theses and Dissertations
The traditional academic purpose of theses and dissertations is to provide guided research practice to graduate students and to contribute to knowledge in the field (Thomas & Brubaker, 2000). Regardless of the long-standing use of theses and dissertations in graduate education, few articles were found examining their pedagogical value as EPAs.
A major research and writing project such as a thesis or dissertation is a traditional rite of passage in graduate education. It gives students practice with many of the expectations common in academia such as choosing a topic, managing a self-directed project, reviewing the literature of the field, conducting research, and writing research reports (Benor & Phillips, 2009). It also functions as the final identifiable milestone for a degree (Hamilton, Johnson, & Poudrier, 2010).
Disadvantages of theses and dissertations are that their subjective grading may compromise reliability and validity (Hamilton, Johnson, & Poudrier, 2010) and they are labor intensive for faculty and students (Oehrtman, Smolen, Hoblet, & Phillips, 2010). They also require faculty to have graduate standing (Oehrtman, Smolen, Hoblet, & Phillips, 2010) and may not reflect skills learned in the program (Hamilton, Johnson, & Poudrier, 2010).
Faculties continually review and revise master's degree programs in an attempt to keep the educational experience relevant and up to date with current needs and expectations. Such decisions are generally based on the professional experiences and opinions of the faculty members. While each department can draw on the expertise of its own faculty, the value of this study is that it aimed to collect professional opinions about EPAs from the faculty base of the ALA-accredited schools of LIS in North America, thus presenting a broader opinion and experience base than a single faculty for considering EPAs. Such a collection of professional opinions may provide useful data for informing decision-making in LIS master's programs. This study queried respondents about their experiences with comprehensive exams, theses, portfolios, internships/practicums, design or research projects (other than thesis), scholarly papers (other than thesis), capstone courses, and other types of EPA to be defined by the respondent.
In 2009-2010, the Graduate Studies Committee at the University of Oklahoma SLIS discussed changing the format of the master's comprehensive exam. Discussion turned to questions about the use of EPAs in LIS education which became the impetus for this study. The purpose of the study was to collect information on LIS faculty members' professional experiences and views concerning types of end of program assessments. Base study questions included:
1. What do LIS faculty members see as the advantages and disadvantages of different types of end of program assessments?
2. Do characteristics of faculty such as rank and area of PhD impact their opinions on EPAs?
The survey for this study was developed by combing through numerous administrative documents from different disciplines and different universities for assertions about the advantages and disadvantages of different EPAs. These statements were synthesized to create typologies that were used to create closed-ended answer categories. Acknowledging that these categories were not exhaustive, each question also had an open-ended "other" response for respondents to contribute additional thoughts.
Names and email addresses for faculty were obtained from the websites of ALA accredited LIS programs excluding Puerto Rico and Quebec for language reasons. In February 2011, all faculty members listed on these programs' websites were emailed an invitation to participate in an electronic survey posted to Survey Monkey. Two weeks later a follow-up email was sent to generate additional responses. Of the total number of invitations sent, 37 either bounced or we received a response to remove them from the list as they were no longer involved in LIS education. This resulted in 961 invitations. A total of 125 faculty members responded to the survey, a 13.0% response rate. One respondent filled out the survey, but indicated in the initial question that they did not give consent for their responses to be used. Therefore, that respondent was deleted from the dataset. Respondents were asked to complete sections of the survey that related to their experience. Consequently, not all respondents filled out all sections.
Responses were downloaded into SPSS for analysis and are described below using descriptive statistics. Since the population was surveyed rather than a random sample, inferential statistics are not appropriate for this study and were not used. Limitations of the study concern the lack of generalizability of the results. While the entire population of the LIS faculty was surveyed, the response rate was low. No assumption can be made that the opinions of those who did respond is similar to the opinions of those who did not.
Respondents had been teaching from 1 to 48 years with a mean of 14 and a median of 11 years. The majority had doctorates in LIS/IS (63.8%) while 27.6% had doctorates in other fields. An additional eight did not have doctorates. As to rank, 30.0% were full professosr, 26.0% were associates, and 36.0% assistants, with four Professor Emeriti, two administrators, two instructors, and two adjuncts.
A key question of this study was, "If you had to choose one type of master's degree end of program assessment as your top preference, which would it be?" One-third of respondents chose portfolio (36.7%) with most of the other EPA choices receiving 10 to 12.2% of the votes. Preference varied by faculty rank with half of full professors compared to 29% of assistant professors choosing portfolio. Assistant professors were somewhat more likely to choose research (thesis, scholarly paper, research projects) and fieldwork (internship, practicum) EPAs than associate and full professors.
The field of the respondent's doctorate was also related to EPA top choice with LIS/IS PhDs being somewhat more likely to choose research and experience EPAs while PhDs in other fields were much more likely to choose comprehensive exams. Portfolio was still the most popular choice for both groups.
For those respondents who were unable to choose a top preference for an EPA, most said that they support having options so students can choose the EPA that best fits their own goals and abilities. Some thought that completion of coursework should be enough for students to receive the master's degree if the state, college, or university does not require an EPA. Others support the use of multiple EPAs such as exams and a paper, a portfolio and an internship, or a capstone course that includes projects and scholarly papers.
Although the closed-ended answer categories were gleaned from professional documents, respondents did not agree with most of the statements. They were most likely to agree that the finished portfolio can be used in job seeking, and the portfolio fosters self-reflection, thinking in terms of goals and objectives, and synthesis of learning experiences. Fewer than half of respondents agreed with the majority of advantages and disadvantages listed.
Advantages listed by respondents in open-ended responses were both program-related and student-related. Several respondents thought that portfolios help faculty assess instruction, diagnose problems with the program or courses and evaluate whether program competencies are being met. For students, it taps into skills such as creativity, ability to choose among products to include, and technology skills. One respondent stated, "I don't see any negatives with this approach."
Concerning disadvantages, several respondents thought that the resulting product was "so what" and held no real value, nor was the experience valuable as it did not teach skills or require critique. Other disadvantages mentioned included issues with technical skills, difficulty keeping students on track, grading difficulties, time-consuming for faculty and students and some faculty were not engaged in the process. One person noted, "No matter how clear the instructions, some students will misinterpret them."
Quite a number of respondents made general comments about portfolios. For many, the portfolio was a new option in their program or their process was under revision. There was clear disagreement about the potential for students using their portfolios as a job seeking tool. Many commented that the portfolios are useful in job seeking, but others stated "prospective employers are not interested in seeing them." One declared, "Although the argument is usually made that a portfolio can be used for job seeking, as one who has been active in libraries for most of her professional career, I have never asked to see a portfolio nor do I think I would do so. I think this argument points out one of the difference in perspectives between those who teach and those who do [not]." Several respondents discussed how the design of the portfolio is key to the value of the outcome. Their concerns were that good design is vital. Another noted that the "main advantage to the student is the process rather than the product".
In commenting why portfolios were their top choice, most respondents listed reasons having to do with students, both what it requires of students and what students take away from it. The most common reason given was that it requires students to reflect on their program experience. Additionally, students have to discriminate in their selection of items to include and integrate knowledge across courses. It demonstrates to students and faculty growth over time and compares student process to learning outcomes. The portfolio is flexible and is a professional-type activity blending or bridging academic and professional skills. Many respondents stated that it could be used in job seeking either as a tool in and of itself or in how it prepares students to talk about their skills in oral presentations. It is realistic, "shows depth and range," and is "appropriate for a variety of students." It is comprehensive. Two additional respondents noted that it can be used for program assessment and accreditation.
For the listed advantages of comprehensive exams (comps), about half to two-thirds of respondents agreed that standardization across students and terms was an advantage. For disadvantages, around half agreed that some students do not test well and the exams do not measure student ability to succeed in the profession. Fewer than half of respondents agreed with any of the other advantages or disadvantages listed for comps.
In open-ended responses several people mentioned that exams require students to integrate, synthesize, and summarize knowledge acquired in the program, and exam outcomes and quality of responses help faculty evaluate the program and suggest curricular modifications. Other advantages mentioned were that exams allow students to demonstrate writing skills and professional knowledge, they are a rite of passage, and the stress of preparing for the exams mirrors stressful situations they will face as professionals. One stated that there were no disadvantages to comps.
In the open-ended answers for "advantages" of comps, five respondents stated "none" or expressed that they did not support the use of comps. One stated that they "cannot think of any pedagogically sound reason" to use comps. For the "disadvantages" open-ended question, several respondents indicated issues with administering the exam: the need to develop different question sets each term, and location and timing-related problems. Other responses criticized the value of comps: they are "not typically 'comprehensive'," "distill a complex field down to a pitiful exam," the "length of time" a student is in the program can "make it difficult to recall concepts." One person stated, "There are better forms of assessment that fit into an overall learning process".
General comments about comps fell into four categories. First, a number of people stated that they had had good experiences with comps. Next, people commented about the design and administration of the exam. While one stated that a plus is that all faculty were involved in the design, another noted that not all faculty actively contribute to the questions. Two said it was easy to administer and one stated that it was "extremely time consuming and stressful for faculty". Next, respondents criticized the essence of comps, stating that they do not adequately measure student success in the program and they are a "less-than-optimal pedagogical method." The last category concerned the student experience. Comments included: It is an equal experience for students, objective, gives students a chance to show mastery, synthesize knowledge and demonstrate critical thinking and writing ability. It is the "least drawn out of other options. Get it over with".
Respondents were very likely to agree with the listed advantages of theses. Over 80% agreed that theses provide experience in research methods and can result in a publication and around three-quarters agreed with the other advantages such as preparation for doctoral studies, gain familiarity with literature on a topic, and improving writing and critical thinking skills. Concerning stated disadvantages, half to two-thirds of respondents agreed that they are very time consuming for faculty and students and can delay graduation. One respondent commented that these listed items were not really disadvantages, but merely excuses.
In the open-ended responses people stated that theses allow students to focus on an area of personal interest, facilitate their ability to interact with researchers, practice applying theories, collecting evidence, and complex thinking. One stated that a thesis "gives the student the self-confidence to tackle a challenging long-term project". One respondent noted that the valuable feedback students receive in thesis work is worth the effort. A number of respondents emphasized that doing a thesis should be optional and not required as it is not appropriate for all students. Many stated there are no meaningful disadvantages to theses if they are optional.
Open-ended responses concerning disadvantages of theses included several comments that theses are not really EPAs and do not demonstrate knowledge of LIS, students are often unprepared to do research, and there is not much time particularly for studies needing Human Subjects approval. Several also lamented the possibility of mediocre theses devaluing the MLIS degree, research done for theses "usually isn't actually useful," the thesis process does not guarantee a good outcome, and some students perform badly. Also mentioned was that the process may work well if a professor has a few theses to supervise, but does not work well with many. One respondent stated that their school discouraged theses as faculty members did not get credit for their supervisory work. Under advantages of theses as EPAs, one respondent said, "none."
For those respondents that chose theses as their top preference, most focused on the skills students hone in the process: writing, thinking, research, analysis, summarization, and mastery of a specialty area. A few others focused on how these skills strengthened the profession. One stated that theses are "standard in graduate education" and "universally accepted in academia". Both quantitative and qualitative analysis showed that respondents had very positive attitudes towards theses. They just felt that a thesis should be an option and not a required EPA.
The majority of respondents were in agreement with all of the advantages for field placements such as internships and practica listed in the survey. Nearly all agreed that practical work experience is valuable on a resume. Another 90% thought that the experience can help students decide if they really want to work in the area, and that experience-based EPAs apply knowledge and skills gained in coursework in real-world settings. For listed disadvantages, around three-fourths of respondents agreed that disadvantages to field placements are that some site supervisors create better learning experiences for their students than others, and field placements are difficult for students who already work full-time.
Advantages offered by respondents included: Introduces students to the culture of the profession, develops practical skills, helps build relationships between the school and the industry, and may be a source of recommendation letters for students. Several responded that in their programs fieldwork is not considered an EPA, for many it is an optional course that students can take. One respondent supported field placements "as part of a basket of measures and not as a single measure". Some respondents noted that in their program practica are required, but not considered EPAs, another stated that all students should do a practicum, but not necessarily as an EPA. Concerning the use of field placements as EPAs, some faculty noted that they do not assess program achievement, integrate learning, or require reflection from students. Additionally "with full-time working students: it just doesn't work".
The majority of respondents who chose internships and practica as their top EPA preferences cited the practical, real-world, professional experience. Some noted that such experience is "clearly important in a professional program." The experience helps students move from theory to practice. In addition to the practice itself, other useful aspects included basic abilities to function in a workplace such as appropriate social skills and the development of realistic career expectations.
There was some EPA overlap, particularly concerning capstone courses. For several respondents, the capstone course included the creation of a scholarly or research paper, poster presentation, research project, analytic project, or design project. In one case, the capstone course was a research methods course. In another program, students had to identify a client and design a project to solve a problem for them. In some cases, students created their portfolios in a capstone course. Poster presentations included online presentations, presentation at an actual conference, and presentation at an in-house show or research symposium. Capstone courses were non-credit at some institutions and one to three credits at others. Time frames for the capstone course ranged from one-half term or short summer term to two full terms.
Common expectations of capstone courses expressed in the comments included individual and collective reflection, integration of knowledge across courses, synthesis of experiences, and the fostering of professional competency. These outcomes are arrived at through a variety of means including writing, discussing, and creating projects. Advantages of capstones given by respondents included: "expectations are standardized but the content ... can vary," "meets new state system of higher education requirements", "reduces the granularity of the topics students examine". Disadvantages were related to excessive time commitments, lack of student preparation or aptitude necessary for meaningful reflection and buy-in from the faculty and program as a whole.
For some respondents, the "assessment" aspect of "end-of-program assessments" was a sticking point. For these respondents, culminating experiences such as capstone courses, portfolios, theses, field placements, and projects were not EPAs because there was no assessment of student learning the way one might have with a comprehensive or oral exam. In other cases, some activities such as theses and field placements were optional choices that students did not have to complete in order to graduate. Concerning choices, many respondents stated that students had options in their programs for type of culminating experience, some had to do more than one, and sometimes student concentration dictated the choice such as school library media students having to do a portfolio.
On the Use of EPAs
Respondents were asked if their program uses EPAs. A majority, two-thirds, required an EPA as part of the master's degree. However, a sizable minority of one-quarter of programs did not have an EPA requirement and 9.4% stated some degree tracks required EPAs, while others did not.
Several respondents used to have a required EPA, but now did not. For others, they had not been requiring one, but were going to start. Still others said they used to require one type but had switched to another. These comments suggest that EPA requirements are fluid over time, perhaps as faculty examine their pedagogical purposes, and as governing and accrediting bodies change their requirements.
Not all faculty members support the use of EPAs. 31 respondents felt that LIS master's programs should not have, or do not need, EPAs. The most commonly cited reason for not supporting EPAs was that course work is a sufficient measure for student success. Time issues were the next most common reason: not enough time in the master's program to include an EPA, and too much additional work for faculty.
In the open-ended remarks about not having EPAs, respondents mentioned time pressures and problems with faculty buy-in. One respondent did not believe EPAs serve any useful purpose, while another declared that "assessment activity is anti-intellectual conservative corporatization of education". One stated that "evaluation metrics" should be in place throughout the program, and if they are not, an EPA is "not a viable alternative". About half of the comments in this section were actually in support of EPAs, such as, "I believe they are needed," and, "It is good to have some type of EPA". One opinion was that coursework-only programs can become paper mills. Two responses viewed the EPA discussion from a wider lens. "In an age of accountability, it may be the most effective and efficient way to demonstrate what a graduate can do upon graduation". Another said, "The fact is that accrediting bodies require outcomes based assessment and such assessment is critical to considered program evaluation".
Several respondents discussed the politics of EPAs on a broader level. Some decried the EPA process as cumbersome and driven by external concerns such as accreditation or an effort by LIS schools to "justify and protect themselves from criticism". One pointed to current debates in the media and in state legislatures concerning "measures to prove that taxpayers got their money's worth". One suggestion was that, unless the EPA is standardized across accredited LIS schools, it is meaningless, while another was there should be an ALA librarian exam in lieu of EPAs. A criticism of EPAs was that they only provided a gatekeeping function and were not rigorous. Two respondents suggested that the only way to know how well the program is functioning is to assess the careers of graduates.
In contrast, several respondents recalled their own MLS experiences of graduating from programs without EPAs and they felt it was "anti-climactic" with "no real synthesis, summation or closure". One person stated about EPAs, "I wish we used them here," while another said their program tried to institute EPAs and were turned down by the university over faculty salary issues for faculty time spent on EPAs. Others reiterated that their university determined whether EPAs were required and what EPAs they could use.
Discussion and Conclusions
According to the 2010 ALISE Statistical Report the portfolio was the second most common of the "Special Requirements for Graduation with a Master's Degree" with 15 schools requiring it (Wallace & Naidoo, 2010, 259). Portfolio use has increased in LIS over the past decade and this may be related to the shift towards a requirement for outcomes-based measures by the infrastructures within which LIS programs function such as the American Library Association Committee on Accreditation, other accrediting bodies, and federal agencies such as the Institute of Museum and Library Services. In this study full professors were more likely than their more junior colleagues to choose portfolios as their preferred EPA. This might result from their recognition of the value of outcomes-based measures due to their greater experience with accreditation, funding, and administrative activity.
Comprehensive exams were the most common "special requirement" indicated by the ALISE report with 20 programs requiring them. Exams did have their proponents among the respondents in this study, but the majority was not satisfied with the value of the exams and few chose exams as their top preference for EPAs. While exams carry the weight of tradition in graduate education, their use has been questioned repeatedly over time.
The ALISE report showed that theses were required for a master's degree by 8 schools and are an option at 33. An additional 26 did not offer theses. While few respondents chose theses as their top EPA preference, overall comments concerning them were highly positive. Many were in favor of offering a thesis option for students who have the aptitude and desire to pursue one. Several respondents disagreed with the concept of a thesis as an EPA, instead casting them as an experience appropriate for some students but perhaps restricted by the time limitations of a master's program.
Fieldwork as a for-credit class was required by 27 schools per the ALISE report. Like the thesis, many respondents had very positive opinions about fieldwork, but they were unlikely to think of it as an EPA and many thought it should be an option rather than a requirement.
The lists of advantages and disadvantages in the survey were gleaned from administrative documents and most were echoed in the literature. Many of these items, however, were not supported by the faculty who responded to this study. Therefore, faculty responses to these items might be useful for those writing future administrative documents.
In general terms, respondents judged EPAs on their usefulness to students and their soundness as a pedagogical practice. They were concerned with how much faculty time was involved in various EPAs and many questioned whether the items listed were truly assessments. Some expounded on the purpose of EPAs in the broader picture outside of their departments or universities. What was most clear was the range of opinions and lack of consensus. A number of faculty were in favor of having coursework only with no required culminating assessment or experience. Of the ones who were in favor of having EPAs, many advocated having options so students could choose what best fit their skills and career plans.
As programs are continually revisiting, revising, and attempting to improve their program requirements such as EPAs, it may be useful to have some insight into professional opinions in a broader context than a single program. This study provides that. Future studies on LIS EPAs may benefit from conducting in-depth interviews with senior faculty and administrators who can share insights into historical contexts of how end of program requirements evolved at various schools. Since portfolios have become increasingly popular as EPA options or requirements, it would be useful to examine aspects of portfolio use such as why senior faculty are more likely to prefer them, and collect data from students and graduates about whether they found their portfolio experience to be a helpful tool in professional acculturation.
Anderson, W. P., Krauskopf, C. J., Rogers, M. E., & Neal, G. W. (1984). Reasons for comprehensive examinations: A re-evaluation. Teaching of Psychology, 11, 78-82. Retrieved from Academic Search Premier.
Bartle, L. & Brodwin, M. G. (2006). The comprehensive examination: Study groups and their effectiveness--A message to counselor education faculty and graduate students. Education, 127, 69-72. Retrieved from Academic Search Premier.
Beck, B., & Becker, H. S. (1969). Modest proposals for graduate programs in sociology. The American Sociologist, 4, 227-234. Retrieved from http://www.jstor.org/stable/27701504
Benor, S. B. & Phillips, B. A. (2009). Pedagogy and impact: The SJCJ master's thesis. Journal of Jewish Communal Service, 84, 228-231. Retrieved from Academic Search Premier.
Berelson, B. (1960). Graduate education in the United States. New York: McGraw-Hill Book Company, Inc.
Council of Graduate Schools. (1994). Master's education: A guide for faculty and administrators, a policy statement. Washington, DC: Council of Graduate Schools.
Fitch, D., Reed, B. G., Peet, M., & Tolman, R. (2008). The use of eportfolios in evaluating the curriculum and student learning. Journal of Social Work Education, 44, 37-54. Retrieved from MasterFILE Premier.
Hamilton, P., Johnson, R., & Poudrier, C. (2010). Measuring educational quality by appraising theses and dissertations: Pitfalls and remedies. Teaching in Higher Education, 15, 567-577. doi: 10.1080/13562517.2010.491905
Latrobe, K. & Lester, J. (2000). Portfolio assessment in the MLIS program. Journal of Education for Library and Information Science, 41, 197-206. doi:10.2307/40324069
Lindquist, J., Wortman, S., & Francis, C. (2011). Adding value to graduate education: The comprehensive exam. NACTA Journal, December, 106-107. Retrieved from Proquest.
Loughead, T. O. (1997). The doctoral comprehensive examination: Fine-tuning the process. Counselor Education & Supervision, 37, 140-149. Retrieved from Professional Development Collection.
Mawn, B. E. & Goldberg, S. (2012). Trends in the nursing doctoral comprehensive examination process: A national survey. Journal of Professional-Nursing, 28, 156-162. doi: 10.1016/j.profnurs.2011.11.013
McNamara, T. L. & Bailey, R. L. (2006). Faculty/ staff perceptions of a standards-based exit portfolio system for graduate students. Innovative Higher Education, 31, 129-141. doi: 10.1007/ s10755-006-9013-9
Neumann, R. (2009). Policy driving change in doctoral education: An Australian case study. In D. Boud & A. Lee (Eds.), Changing practices of doctoral education (pp. 211-224). London: Routledge.
Oehrtman, S. J., Smolen, D., Hoblet, K., & Phillips, K. A. (2010). The comprehensive examination: A viable master's of science in nursing capstone course. Journal of Professional Nursing, 26, 360-365. Retrieved from http://www.sciencedirect.com/science/article/pii/S8755722310000888
Ponder, N., Beatty, S. E., & Foxx, W. (2004). Doctoral comprehensive exams in marketing: current practices and emerging perspectives. Journal of Marketing Education, 26, 226-235. doi: 10.1177/0273475304268778
Pressey, S. L., Pressey, L. C., & Barnes, E. J. (1932). The final ordeal: Two minor experiments regarding reliability and validity of graduate examinations. The Journal of Higher Education, 3, 261-264. Retrieved from http://www.jstor.org/ stable/1974483
Prus, J. & Johnson, R. (1994). A critical review of student assessment options. In T. H. Bers & M. L. Mittler (Eds.), Assessment and testing: Myths and realities (pp. 69-83). San Francisco: Jossey-Bass Publishers.
Ryan, M. (2011). Evaluating portfolio use as a tool for assessment and professional development in graduate nursing education. Journal of Professional Nursing, 27, 84-91. Retrieved from http://www.sciencedirect.com/science/article/pii/ S8755722310001183
Schafer, J. A. & Giblin, M. J. (2008). Doctoral comprehensive exams: Standardization, customization, and everywhere in between. Journal of Criminal Justice Education, 19, 275-289. doi: 10.1080/10511250802137648
Scott, S. G. (2010). Enhancing reflection skills through learning portfolios: An empirical test. Journal of Management Education, 34, 430-157. doi: 10.1177/1052562909351144
Storr, R. J. (1953). The beginnings of graduate education in America. Chicago: University of Chicago Press.
Thomas, R. M., & Brubaker, D. L. (2000). Theses and dissertations: A guide to planning, research, and writing. Westport, CT: Bergin & Garvey.
Thyer, B. A. (2003). A student portfolio approach to conducting doctoral social work comprehensive examinations. Journal of Administration in Social Work, 23, 117-126. doi: 10.1300/ J067v23n03_10
Wallace, D. P. & Naidoo, J. (Eds.). (2010). Library and information science education statistical report 2010. Chicago: Association for Library and Information Science Education. Retrieved from http://www.alise.org/statistical-reports
Susan K. Burke
School of Library and Information Studies, University of Oklahoma, 401 W. Brooks, Old Bizzell Library, Room 120, Norman, OK 73019-6032, Email: email@example.com
John T. Snead
School of Library and Information Studies, University of Oklahoma, 401 W. Brooks, Old Bizzell Library, Room 120, Norman, OK 73019-6032, Email: firstname.lastname@example.org
Table 1. Top EPA Preference by Faculty Rank. Full Professor Associate Assistant % Professor Professor % % Portfolio 48.0 39.1 29.0 Research EPAs 16.0 17.4 25.8 Fieldwork EPAs 16.0 17.4 25.8 Comprehensive exams 12.0 17.4 6.5 Capstone course 8.0 8.7 12.9 Total N = 25 N = 23 N = 31 Table 2. Top EPA Preference by PhD. LIS/IS PhD Other PhD % % Portfolio 33.3 44.0 Research EPAs 24.5 12.0 Fieldwork EPAs 22.8 16.0 Capstone course 12.3 4.0 Comprehensive exams 7.0 24.4 Total N = 57 N = 25 Table 3. Faculty Opinions on Portfolios as EPAs. f % N = 69 Advantages The finished portfolio can be 54 78.3 used in job seeking The portfolio process teaches 50 72.5 students to be professionally self-reflective Requires students to think in 48 69.6 terms of goals and/or objectives The portfolio requires students 47 68.1 to synthesize their learning experiences from their master's program Reflective statements could give 29 42.0 faculty information about student satisfaction with the program The portfolio process is similar to 25 36.2 types of reports a professional in this field might write It is less stressful for students 22 31.9 than some other types of EPAs (like comps) Highlights students' strengths and 21 30.4 weaknesses The portfolio process improves 18 26.1 critical thinking skills It is less time consuming for 13 18.8 students than some other types of EPAs (like thesis) Disadvantages Content may vary widely 36 52.2 The portfolio is very time consuming 32 46.4 for faculty The structure is too open to 32 46.4 interpretation Grading portfolios is too subjective 31 44.9 The portfolio is very time consuming 20 29.0 for students The structure is too rigid 4 5.8 Table 4. Faculty Opinions on Comprehensive Exams as EPAs. f % N = 71 Advantages Fairly standardized across students 47 66.2 Fairly standardized across terms 34 47.9 Takes less total time for students 26 36.6 Does not take the place of an elective course 26 36.6 (like a thesis often does) Takes less total time for faculty 25 35.2 Comprehensive exams are common across many 21 29.6 disciplines Comprehensive exams are traditionally part of 21 29.6 graduate education Disadvantages Some students do not test well 41 57.7 Exams are not a good measure of student ability 40 56.3 to succeed in the profession It is not a good measure of student success in 29 40.8 the program It is overly stressful for the students 29 40.8 It is too time consuming for the faculty 19 26.8 It is too distracting for the students when they 17 23.9 have other coursework concerns Table 5. Faculty Opinions on Theses as EPAs. f % N = 60 Advantages Provides experience in research methods 53 88.3 Can result in a publication 50 83.3 Helpful if student decides to go on to 46 76.7 a doctoral program Helps student become very familiar with 46 76.7 the literature in a topic area Improves professional writing skills 45 75.0 Improves critical thinking skills 45 75.0 Opportunity to become highly 43 71.7 knowledgeable in a specific topic Disadvantages Very time consuming for faculty 37 61.7 Very time consuming for student 33 55.0 Can delay student graduation 31 51.7 Teaches skills that may not be relevant 23 38.3 to many LIS jobs Table 6. Faculty Opinion of Fieldwork as EPAs. f % N = 71 Advantages Practical work experience is valuable on a resume 52 98.1 The experience can help students decide if they 48 90.6 really want to work in that area Applies knowledge and skills gained in coursework 48 90.6 in real-world settings Builds students' confidence in their abilities 46 86.8 Graduates are sometimes hired at the sites where 41 77.4 they did their internship/practicum Helps point out areas in which the student needs 38 71.7 to work on their knowledge or skills Disadvantages Some site supervisors create better learning 40 75.5 experiences for the students than others Difficult for students who are already working 38 71.7 full time Poorly designed internships/practica may result 30 56.6 in students spending most of their time doing non-professional work (like filing) Difficult to find an appropriate internship 21 39.6 Requires a great deal of hours 12 22.6 Table 7. Does Your School/College Require EPAs? f % N = 96 Yes, all students are required to complete 63 65.6 an EPA to receive a master's degree No, EPAs are not used to determine whether 24 25.0 a student receives a master's degree in our program It depends on which degree track students 9 9.4 choose, some tracks require EPAs and others do not Table 8. Opinions on Why Not Use EPAs. f % N = 96 Course work is a sufficient 19 61.3 measure of student success There is not enough time in 14 45.2 a master's program to include an EPA It is too much additional 14 45.2 work for faculty It is too much additional 10 32.3 work for students EPAs are not required by our 9 29.0 Graduate College/University EPAs are not a good measure 7 22.6 of student success in the program
|Printer friendly Cite/link Email Feedback|
|Author:||Burke, Susan K.; Snead, John T.|
|Publication:||Journal of Education for Library and Information Science|
|Date:||Jan 1, 2014|
|Previous Article:||RDA: training and continuing education needs in academic libraries.|
|Next Article:||Technology competency requirements of ALA-accredited library science programs: an updated analysis.|