Creating and sustaining a culture of assessment: the guiding principles of institutions that have established a culture of assessment can be described as internally driven.
"Many higher education institutions have managed only small and incremental steps on their journeys toward sustainability, confronted by both complexity and embedded resistance to change." (Tilbury and Wortman 2008, p. 5)Introduction
As the assessment movement gains momentum, colleges, universities, and academic departments continue to seek help with the rudiments of the process and with ways to evaluate their specific programs (U.S. Department of Education 1993). In the United States, over 94 percent of higher education institutions have assessment activities underway (Palomba and Banta 1999). Many of these institutions struggle in "isolation with state accountability mandates, accrediting agency requirements, and internal concerns with declining student performance and the efficacy of the curriculum" (U.S. Department of Education 1993, [paragraph] 3). In response to this and the growing sophistication of assessment in general, the higher education community is now looking for guidance on models and best practices. The American Association for Higher Education and other resources offer a plethora of "how to" information that specifies the stakeholders, assessment rubrics, and critical components needed to establish an assessment process. However, there is limited information on how to establish a culture of assessment that will sustain itself beyond accreditation to continually assess how much students learn and to maintain a quality improvement approach across programs, schools, and institutions.
Culture of Assessment
A culture of assessment is defined as "an organizational environment in which decisions are based on facts, research, and analysis, and where services are planned and delivered in ways that maximize positive outcomes and impacts for customers and stakeholders" (Lakos and Phipps 2004, p. 352). A culture of assessment can guide meaningful change on a continual basis (Lakos and Phipps 2004; Smart and St. John 1996). Creating this culture requires continuous learning, strategic decision making, prioritization of the allocation of scarce resources, and organizational and individual accountability (Lakos and Phipps 2004). An institution with a culture of assessment agrees on what is organizationally meaningful and what practices are required to fulfill its goals (Schein 1999).
As Cameron and Quinn (1999) note, the most commonly shared ingredient of successful organizations is their ability to pay attention to culture. Correspondingly, the success of an assessment system depends on using established principles grounded in organizational culture to govern its practices. Creating an effective assessment system at the school, college, or institutional level requires the articulation of a shared conceptual understanding, a common definition of assessment, and the clear expression of assessment expectations and the use of results (Bresciani 2005). This ensures that the environment is ready to establish an assessment culture that supports and sustains change (Hill 2005). The study presented in this article examines how institutions have established a culture of assessment and how the interplay among organizational practices affects that culture. Since the likelihood of overall institutional agreement and buy-in is rare, particularly initially, the article also discusses the challenges that institutions face when creating a culture of assessment. In addition, the article describes strategies that can be used to sustain a culture of assessment once it has been created, as well as how leadership can support this effort in terms of policy design and implementation.
Theoretical Framework
The assessment culture matrix, published by the Higher Learning Commission/NCA (2003), identifies three stages in developing an assessment culture: beginning, progress, and maturation. The matrix describes each stage of development in relation to seven factors that are pertinent to assessment systems across programs, colleges, and institutions: collective and shared values, mission, faculty, administration and board, students, resources, and structures. Using this framework, the study presented in this article investigates the strategies institutions use to develop a culture of assessment. More specifically, we examine the following five factors: leadership, faculty, resources, students, and access to and systematic use of data. "Collective and shared values" and "mission" are excluded because they are embedded throughout the other factors. In the assessment culture matrix, "structures" pertains to maintaining the established culture; for the purposes of this study, we have replaced structures with access to and systematic use of data. These five factors are further described below:
Leadership. Leadership and vision are the principal components of organizational success (Lakos and Phipps 2004). To build a culture of assessment, the leadership of a small implementation team is necessary (Brock et al. 2007; Hill 2005). Lakos and Phipps (2004) note that
leadership is essential for assessment work to succeed. Leaders must have a clear performance ethic and be visibly and continuously committed to assessment work and understand its importance to the success of the organization. If leadership is perceived to lack commitment, meaningful culture change will be difficult, if not impossible. (p. 353)
Leadership that commits to and financially supports assessment activities can overcome challenges (Lakos and Phipps 2004). As Suskie (2004, pp. 35-36) posits, "If campus leaders are committed to assessment, assessment gets done and it gets done well." The success of any strategies used to facilitate the establishment of a culture of assessment will depend highly on the institution's leadership. It is well known that assessment is not cheap. It requires a substantial commitment of resources (e.g., financial, social, time), and leaders can facilitate access to those resources. Leaders are also expected to head the development of meaningful policies that guide practices and promote open, appropriate channels of communication across different disciplines and stakeholders in order to build a strong community. As Fonte (1994) states,
because a president is ultimately accountable for assessment of an institution's effectiveness and efficiency, he or she should play a proactive role in mobilizing institutional resources for assessment; balancing the apparent conflicts between assessing for effectiveness, for efficiency, and for improvement; and communicating assessment results to various constituencies. (p. 37)
Faculty. Faculty should be involved in the assessment process; their roles include identifying learning objectives and continuously assessing whether current processes can be improved. When necessary, faculty should be trained so they can fulfill their assessment responsibilities (Lakos and Phipps 2004). The Assessment Faculty Fellows Program at the University of Delaware is an exceptional example of faculty involvement. One of the hallmarks of this program is its ability to recruit influential faculty to lead assessment efforts. Faculty fellows work closely with chairs and departmental faculty and are "responsible for coordinating, supporting, and maintaining assessment processes in their respective academic units and for sharing knowledge with their colleagues across the university" (University of Delaware 2006, [paragraph] 1). Whatever method is used to facilitate their involvement in and commitment to assessment, faculty must have ownership of the process (Bresciani 2005) to ensure its sustainability.
Resources. Research has shown that the lack of resources circumvents meaningful assessment efforts (Brock et al. 2007; Hill 2005). For a culture of assessment to survive and succeed in the present environment, there must be a "commitment to the necessity of prioritization of the allocation of scarce resources" (Lakos and Phipps 2004, p. 351). Resources needed for assessment include funds allocated in the operating budget, technological support, physical facilities, knowledgeable staff and/or faculty, and release time or compensation for their involvement. Professional development opportunities and an assessment manual or newsletter are additional valuable resources (Higher Learning Commission/NCA 2003).
Students. Assessment in higher education has shifted from a teacher-centered approach to one that places more emphasis on student involvement (Boud 1995). Student participation in assessment practices is more than responding to surveys and other data collection instruments and includes activities such as analysis of assessment results, administration of assessment instruments, and planning (Froestad and Bakken 2004). Involvement in these activities provides a means for more student and teacher interaction and increases the relevance and legitimacy of the process for students (Froestad and Bakken 2004). Further, student involvement in assessment also improves student learning (Falchikov 2005). The Working Group on Quality Assurance of Student Assessment (2008), based in the Netherlands, contends that "there should be a role for students in all aspects of assessment from its design through to implementation and review" (p. 9). For the purposes of this study, student participation encompasses those activities and initiatives that aim to give students a more active role, one that moves them from a data provider's perspective to an active stakeholder's perspective in an institution's assessment initiatives. This approach will increase student motivation and make assessment more meaningful (Suskie 2004).
Access to and systematic use of assessment data.
Assessment data provide the means by which assessment systems can be effective in student learning and program improvement. By linking data collection with learning outcomes, practices can be improved and adjusted. With systematic use of assessment data and guaranteed access for all, interested parties can practice data-driven decision making; of course, this requires institutions to provide the means for disseminating and interpreting assessment data (Gentemann 2001). Assessment becomes part of the everyday work process when data and user feedback are routinely collected; analyzed; and used to set priorities, allocate resources, and make decisions (Lakos and Phipps 2004).
Creating means to provide access to data and disseminating lessons learned from assessment results are some of the greatest challenges in building a culture of assessment (Bresciani 2005). A study on the development of assessment systems found that after one year, only six of the 27 colleges surveyed showed signs of institutionalizing a culture of assessment. Difficulties were attributed to not knowing how to respond to assessment data and to having weak information systems (Brock et al. 2007). The American Association for Higher Education asserts, "The point of assessment is not to gather data and return 'results'; it is a process that starts with the questions of decision-makers, that involves them in the gathering and interpreting of data, and that informs and helps guide continuous improvement" (American Association for Higher Education 1996, [paragraph] 7). This statement highlights the importance of accessing and using assessment data in developing a culture of assessment.
Purpose of the Study
The purpose of this study was to examine institutional initiatives being used to develop and sustain a culture of assessment. The research questions were:
* What is the current phase of development of a culture of assessment at different schools/ colleges/institutions?
* What are the challenges in building or sustaining a culture of assessment at different schools/ colleges/institutions?
* What has facilitated building or sustaining a culture of assessment at different schools/colleges/institutions?
* What strategies support/sustain a culture of assessment?
Method
An electronic survey on building and sustaining a culture of assessment was administered in the fall of 2007 using SelectSurvey[c] (see the sidebar, "Survey Instrument"). The survey consisted of 15 questions regarding the institution's stage in developing a culture of assessment, institutional practices, institutional characteristics, and professional demographics (the respondent's level of responsibility and length of employment). The survey had 11 structured categorical items, One rating scale item, and three open-ended items. Members of two national Listservs (ASSESS and First-Year Assessment) received an e-mail invitation to participate in the survey along with a link to the survey. The 2,142 members of these Listservs are assessment professionals working in higher education institutions, schools, and colleges.
The majority of respondents were from four-year universities (83.8 percent), while 13.7 percent came from two-year institutions and 2.5 percent reported "other." Respondents were predominantly from public institutions (65.55 percent), compared to 32.77 percent from private institutions and 1.68 percent from "public universities that are privately supported." The majority of the respondents had campus-wide responsibilities (72.27 percent), while 8.40 percent were at the school or college level (e.g., School of Education or Nursing) and 19.33 percent were in Student Affairs.
The researchers recognize that the response rate (5.5 percent, N = 119) is lower than the typical 13.4 percent response rate for electronic surveys (Hamilton 2009). This low response rate might be the consequence of having the same individuals subscribe to both lists. This study therefore emphasizes trends rather than drawing inferences (Cook, Heath, and Thompson 2000). The results focus on those who participated in the study and their institutions' stage of developing a culture of assessment (beginning, progress, or maturation). The researchers used descriptive statistics to analyze the structured survey items and qualitative coding procedures to analyze the open-ended comments. For the comments, the researchers examined the text line-by-line to identify discrete incidents, ideas, or events. Data reduction and data display were then used to draw conclusions.
Results
In congruence with the assessment culture matrix, survey responses were analyzed by stage of development: "have not established shared principles governing assessment" (beginning), "establishing shared principles governing assessment" (progress), and "have established shared principles governing assessment" (maturation). The majority of respondents (48.74 percent) indicated that their institution was working toward establishing shared principles, 26.05 percent declared that their institution had already established a culture of assessment, and 25.21 percent reported that they had not. Results are presented along the main categories of the survey instrument.
Facilitating factors in establishing a culture of assessment. Respondents were asked to indicate whether certain factors (leadership, faculty involvement, resources, student participation, access to data, systematic use of data, and other) were responsible for helping to establish a culture of assessment at their institution. Leadership was the factor most cited by both respondents at institutions that have reached maturation (93 percent) and respondents at institutions that are in progress (83 percent), as shown in figure 1. This 10-point gap between institutions at maturation and those in progress was twice as large with respect to the systematic use of assessment data. Sixty-four percent of the respondents who reported that their institutions have reached maturation listed it as an aid compared to 45 percent of those whose institutions are in progress. Student participation was more of a facilitating factor for institutions in progress (19 percent) than for institutions that have reached maturation (14 percent).
[FIGURE 1 OMITTED]
[FIGURE 2 OMITTED]
Challenges in establishing a culture of assessment.
Faculty involvement, resources, and systematic use of assessment data were the most commonly cited challenges in the establishment of a culture of assessment. Institutions that have reached maturation and those that are in progress reported faculty involvement (61 percent and 76 percent respectively) and systematic use of assessment data (35 percent and 66 percent respectively) as the prevailing challenges (see figure 2). [cr] A lack of resources was the most cited challenge for institutions that are at the beginning stage. Among the six factors, leadership is the third-most prevalent challenge for institutions that are at the beginning stage (68 percent), the fourth-most for institutions in progress (52 percent), and the lowest (sixth) for institutions at maturation (19 percent). Use of assessment data also varies throughout the three stages of development. It is less challenging for institutions that have reached maturation (35 percent) than for those in progress (66 percent) and those that are beginning (68 percent).
<01_TB004>
Strategies used to promote a culture of assessment.
Respondents were asked to describe the strategies used at their institutions to overcome challenges and difficulties in their efforts to establish a culture of assessment. Analysis of the open-ended responses revealed the following four major strategic themes: (1) assessment integrated in daily practice, (2) leadership, (3) use of data, and (4) communication. Institutions were compared along these four themes for each stage of development (see figure 3). Of the 31 respondents answering this question, 18 were from institutions that had reached maturation. The purpose of this comparison is to provide insights on best practices in building and sustaining a culture of assessment so that institutions can borrow from one another as they move along the continuum from beginning to maturation.
* Assessment integrated in daily practice. Institutions that have established shared principles governing assessment across campus used two main strategies to incorporate and foster the integration of assessment into daily practice. First, they reported encouraging faculty and student participation through rewards. One respondent stated, "We have asked the Student Senate to provide additional students to be part of the committee work. Many majors also include students in their assessment implementation and review processes." Rewards might include providing small grants or funding for assessment projects. A second strategy involved piloting new ideas and initiatives on a smaller scale to develop stakeholders' comfort and ownership. One respondent reported that efforts to make assessment part of daily practice included "informal discussions about assessment topics with groups of faculty from different disciplines [and] direct contact with student organizations to encourage student participation." Respondents from institutions in other categories (progress or beginning) focus more on a grassroots approach with the aim of convincing stakeholders of the importance of assessment. Limited experience in data collection and analysis seems to be the major obstacle for institutions in the beginning stage.
* Leadership. When asked whether leaders at their institution devote time to assessment planning and implementation, respondents answered differently depending on their institution's stage of development. In institutions that have reached maturation, leaders implement a systemic approach to assessment that focuses on "closing the loop" Meanwhile, institutions at other stages generally focus on more isolated actions. For example, leaders at institutions that have reached maturation focus on identifying and engaging stakeholders (e.g., students) and using assessment data in future initiatives. A majority of the respondents from these institutions (66 percent) commented on leadership-related strategies, including leaders facilitating collaboration among different units within a school, encouraging shared responsibility among various stakeholders (e.g., student, faculty), making funds and resources available, and clarifying expectations. Leaders at institutions at maturation frequently develop strategies to make assessment a faculty-driven process that also actively engages students. As one respondent stated, "We engage faculty in focused discussions regarding assessment in collaboration with our Faculty Center for Innovative Teaching. We have just begun discussions regarding methods to involve students in this process." Assessment in these institutions is primarily based on, as one respondent put it, "faculty-designed and developed instruments and systems." Another respondent noted that leaders at these institutions continually "try to involve faculty through enticements (course release for leading assessment efforts, professional development funds for participating in some assessment activities such as assessing writing across the curriculum) and through cajoling (social influence)."
We engage faculty in focused discussions regarding assessment.
* Use of assessment data. There were striking differences regarding whether assessment data are systematically available to interested parties at institutions at different stages of development. While institutions that have reached maturation focus on the wide dissemination and use of data, other institutions limit the use and availability of data to budget requests and reporting. There were 18 comments from institutions that have established a culture of assessment, and 72 percent of those referenced the use of data as a major strategy to promote and support a culture of assessment. Analysis of the responses revealed three major differences between institutions at maturation and those in progress. Institutions in the first group have created a central repository for data collection and analysis. Further, as one respondent explained, they support and encourage periodic "short reports on assessment progress and findings from studies for campus leadership." Such a strategy allows the frequent use of data throughout the year as opposed to only in the traditional annual report. Another strategy that differentiates institutions at maturation is that they require annual meetings among departments and university assessment officials to review assessment plans and then use the data collected to develop future plans. Institutions in this category also require the use of assessment data in their five-year program reviews.
We support and encourage short reports on assessment progress.
* Communication. Institutions used different communication strategies depending on which stage of development they were in. While all institutions seem to have established a forum for sharing ideas, their approaches differed. Institutions that have reached maturation distinguished themselves by emphasizing the role and importance of involving students in communication strategies. Further, 55 percent of the comments provided by respondents from these institutions referenced communication and information sharing as a hallmark of their assessment strategy. Communication at institutions at maturation is integrated into a systemic planning effort through workshops, conferences, symposiums, newsletters, committee reporting and discussions, informal discussions across disciplines, and formal reports, and emphasizes maintaining direct contact with student organizations and conducting focus groups to explore ways to get students more involved. As one respondent noted, direct contact occurs through "informal discussions about assessment topics with groups of faculty from different disciplines [and] direct contact with student organizations to encourage student participation." Another respondent stated that frequent communication with students happened by "undertaking a series of focus groups with students to discover ways to involve them more in assessment." While institutions at other stages have developed vague strategies to keep communication ongoing (e.g., offering workshops, building relationships, showcasing best practices), institutions that have reached maturation have developed more specific methods, including publishing an assessment newsletter and encouraging informal discussions across disciplines.
We held focus groups with students to discover ways to involve them more in assessment.
Discussion
Leadership, faculty involvement, resources, student participation, and access to and systematic use of data were found to be important factors in establishing and sustaining a culture of assessment. Notably, leaders from institutions that are making progress in establishing shared principles incorporate learning outcomes and assessment at the course level, but generally offer no specific direction on how these outcomes should be used to promote a culture of assessment. While such strategies appear isolated and remain unincorporated in a more systemic approach, they can still help institutions develop shared principles. We also found that student participation was more of a facilitating factor for institutions in progress than for those that have reached maturation. This could be because institutions that are in progress build more bridges with student organizations to actively promote a culture of assessment, while institutions that have reached maturation may consider this as something they have already accomplished.
Findings from this study are consistent with the literature. Strategies reported (integrating assessment in daily practice, leadership, use of assessment data, and communication) can be aligned with factors that have been previously identified as challenges for institutions in their efforts to promote a culture of assessment. For example, Brock et al. (2007) reports ineffective information systems (communication) and inappropriate ways of responding to data (use of data) as the main challenges institutions face in building a culture of assessment. Suskie (2004) highlights the importance of having leaders on board when trying to establish a culture of assessment. In her work, she reports that leaders must take on a number of key roles as well as "personally commit to assessment" "promote communication," and "encourage a sense of community" (pp. 36-37).
Leaders facilitate the development of strategies that allow assessment to be integrated into daily practice, such as using data consistently and opening and maintaining channels of communication. Establishing and sustaining a culture of assessment generally requires the development of strategies that make assessment a part of daily practice rather than an isolated event. To that end, leaders must support assessment initiatives; more importantly, leaders must ensure that adequate time is devoted to assessment planning and implementation so that assessment is built into the institution's schedule and activities. A culture of assessment should also support ongoing communication among stakeholders and the availability and use of assessment data.
Policy Implications
The strategies reported to promote a culture of assessment must be supported by effective institutional policies. The prominent role of leadership in the strategies previously described implies the development of an environment capable of promoting and building a culture of assessment. Such environments result from establishing meaningful policies and effectively managing the components and factors that must be considered in the development of those policies. This section discusses the implications for policy in terms of a consensus-building policy design model and describes the main factors needed to implement such a model.
Policy design model. The strategies effective in developing and sustaining a culture of assessment (integrating assessment in daily practice, leadership, use of assessment data, and communication) suggest that higher education leaders must be consensus builders. Building a culture of assessment requires all stakeholders to have a voice in the process. Essex (1976) emphasizes the need for unified policy and the importance of a consensus-building model: "It seems quite clear that our nation is able to make rapid progress in education when the agencies, organizations, groups, and individuals concerned with education have reached consensus" (p. 9). Reaching a consensus requires providing both public (e.g., parents) and professional (e.g., faculty, students) stakeholders with the opportunity to articulate policy statements, in the hope that collaboration will lead to clarity and agreement.
The recent emphasis on accountability in higher education and the increasing involvement of partners (e.g., parents, the community in general) provide a reason and foundation for using a consensus-building policy model. The process of designing policies that promote improved higher education effectiveness should include opportunities for stakeholders to play essential roles as advocates for effectiveness as well as promoters of needed change and "the necessity to interface with others, either one-on-one or in groups" (Martinez 2008, p. 636).
The strategies that promote a culture of assessment require stakeholders to adhere to commonly agreed-upon regulations, procedures, and rules that cannot be established without a consensual approach. Leaders must promote and create an environment in which consensus can be built and in which all stakeholders can maintain their individual identities while still contributing through shared governance for the benefit of the whole. Leaders should also promote the complementarities of the various stakeholders so that institutions can be steered in the desired direction (Kennedy 2003).
Factors needed to implement the model. To successfully implement the consensus-building policy model, leaders must promote a number of factors:
* Empowerment of all stakeholders. Leaders of higher education institutions must design policies that empower all stakeholders. As the W. K. Kellogg Foundation (n.d., p. 3) reports, "An effective leader works with others in ways that allow each person to be heard and moved individually, while listening and moving a group collectively toward a desired future. [Leaders] must have the ability to assist communities in creating and articulating a compelling, community-inspired vision of a desired future." This is necessary to promote the information and knowledge sharing that will ultimately facilitate stakeholders' sense of belonging and ownership (Perry 2009). Empowering stakeholders also means giving them flexibility within agreed-upon guidelines. While stakeholders share common goals, they have different individual needs. Policy design can empower stakeholders by allowing them enough flexibility to address their own issues and needs so that they can easily integrate assessment into daily practice and promote the development of internally-driven processes and procedures. When stakeholders are empowered, then assessment has a better chance of focusing on student learning and program improvement, rather than just accreditation. Further, empowered faculty and student stakeholders can help design meaningful ways to use the assessment data.
* Open channels of communication. Policies should promote open and frequent communication among all stakeholders. This means facilitating exchanges among faculty, students, program areas, and higher education leadership to help develop policies to address any conflicts of interest that may arise. An open channel of communication provides the various stakeholders with a venue for expressing their interests and concerns and for providing feedback about how things are being done. This communication channel should guarantee the sense of ownership needed to build a culture of assessment.
Open channels of communication also ensure data usage and promote shared governance. Open channels of communication can facilitate data sharing among stakeholders and, therefore, provide a forum in which to discuss results, practices, and findings. This could, in turn, promote greater data usage and lead to improved effectiveness.
* Innovation diffusion. Results of this study indicate that policies should incorporate an innovation diffusion component that encourages small pilots. Starting assessment on a smaller scale and expanding it later was shown to be an effective strategy in building a culture of assessment. Having new ideas and innovative approaches tested first by those program areas and faculty members who are willing to try something new can facilitate the success and sustainability of a wider assessment initiative. This should be supported by a policy that provides resources to those programs. This approach also allows programs to learn from one another and helps to establish higher levels of comfort with new initiatives. Integrating an innovation diffusion component might help ease faculty concerns and increase their involvement, the lack of which is one of the most commonly noted barriers to the establishment of a culture of assessment.
Future Research
Further research is needed to explore the relationships and linkages between higher education policies and the establishment of a culture of assessment. Additional research is also needed to answer questions such as: What practices can be developed that could lead to assessment being integrated into daily practice and not seen as a separate and irrelevant activity? Who are the main stakeholders in terms of establishing a culture of assessment, and how do we better understand their different interests and needs? How do we make establishing and sustaining a culture of assessment relevant and meaningful to all stakeholders?
Conclusions
This article examined guiding principles and practices among institutions with regard to the three stages in the development of a culture of assessment. We highlighted areas where beginning and progress institutions can learn from institutions that have reached maturation in successfully creating and sustaining a culture of assessment. The results of this study suggest that the guiding principles of institutions that have established a culture of assessment can be described as internally driven. Institutions with a culture of assessment tend to focus on student learning rather than accreditation, the usage of locally developed instruments, and regular communication through means such as workshops. Institutions that focus on creating an internally-driven process are more likely to develop strategies that allow stakeholders' ownership and buy-in, necessary components in building and sustaining a culture of assessment.
References
American Association for Higher Education. 1996. Nine Principles of Good Practice for Assessing Student Learning. Retrieved October 8, 2009, from the World Wide Web: www.gettysburg. edu/about/offices/ees/institutional_analysis/assessment_/files/9 Principles.pdf.
Boud, D. 1995. Enhancing Learning Through Self Assessment. London: Kogan Page.
Bresciani, M. J. 2005. Creating a Learning Culture and Acquiring Faculty Buy-in. Presentation at the Institute for Quality Enhancement and Accreditation Conference, Orlando. Retrieved October 8, 2009, from the World Wide Web: interwork.sdsu.edu/elip/consultation/presentations/fac_buyin.pdf.
Brock, T., D. Jenkins, T. Ellwein, J. Miller, S. Gooden, K. Martin, C. MacGregor, and M. Pih. 2007 Building a Culture of Evidence for Community College Student Success: Early Progress in the Achieving the Dream Initiative. New York: MDRC.
Cameron, K. S., and R. E. Quinn. 1999. Diagnosing and Changing Organizational Culture: Based on the Competing Values Framework. Reading, MA: Addison-Wesley.
Cook, C., F. Heath, and R. L. Thompson. 2000. A Meta-Analysis of Response Rates in Web- or Internet-Based Surveys. Educational and Psychological Measurement 60 (6): 821-36.
Essex, M. W. 1976. Models for Policy Design: Public Consensus Model, Professional Consensus Model, Public/Professional Consensus Model. Retrieved October 8, 2009, from the World Wide Web: www.eric.ed.gov/ERICDocs/data /ericdocs2sql/content_storage_01/0000019b/80/35/8e/1c.pdf.
Falchikov, N. 2005. Improving Assessment Through Student Involvement: Practical Solutions for Aiding Learning in Higher and Further Education. New York: RoutledgeFalmer.
Fonte, R. 1994. Assessment From the President's Perspective. New Directions for Community Colleges 88: 37-45.
Froestad, W., and P. Bakken, eds. 2004. Student Involvement in Quality Assessments of Higher Education in the Nordic Countries. Helsinki: Nordic Quality Assurance Network in Higher Education. Retrieved October 8, 2009, from the World Wide Web: www.noqa.no/graphics/NOQA/Reports/Student %20Involvement%20in%20Quality%20Assessments%20of %20Higher%20Education%20in%20the%20Nordic %20Countries.pdf.
Gentemann, K. M. 2001. Using and Disseminating Assessment Data Information. Presentation given at the University Life Workshop, George Mason University, Fairfax, VA.
Hamilton, M. B. 2009. Online Survey Response Rates and Times: Background and Guidance for Industry. White paper. Longmont, CO: Ipathia, Inc. Retrieved October 8, 2009, from the World Wide Web: www.supersurvey.com/papers /supersurvey_white_paper_response_rates.pdf.
Higher Learning Commission/NCA. 2003. Assessment of Student Academic Achievement: Assessment Culture Matrix. In Restructured Expectations: A Transitional Workbook, 70-80. Chicago: Higher Learning Commission/North Central Association of Colleges and Schools. Retrieved October 8, 2009, from the World Wide Web: www.ncahlc.org/download /AssessMatrix03.pdf.
Hill, J. S. 2005. Developing a Culture of Assessment: Insights From Theory and Experience. Journal of Political Science Education 1 (1): 29-37.
Kennedy, J. S. 2003. Higher Education Governance as a Key Policy Issue in the 21st Century. Educational Research for Policy and Practice 2 (1): 55-70.
Lakos, A., and S. Phipps. 2004. Creating a Culture of Assessment: A Catalyst for Organizational Change. Libraries and the Academy 4 (3): 345-61. Retrieved October 8, 2009, from the World Wide Web: repositories.cdlib.org/cgi /viewcontent.cgi?article=1563&context=postprints.
Martinez, M. 2008. Competencies and Higher Education Policy Analysts. Educational Policy 22 (5): 623-39.
Palomba, C. A., and T. W. Banta. 1999. Assessment Essentials: Planning, Implementing, and Improving Assessment in Higher Education. San Francisco: Jossey-Bass.
Perry, L. B. 2009. Conceptualizing Education Policy in Democratic Societies. Educational Policy 23 (3): 423-50.
Schein, E. H. 1999. The Corporate Culture Survival Guide. San Francisco: Jossey-Bass.
Smart, J. C., and E. P St. John. 1996. Organizational Culture and Effectiveness in Higher Education: A Test of the "Culture Type" and "Strong Culture" Hypotheses. Educational Evaluation and Policy Analysis 18 (3): 219-41.
Suskie, L. 2004. Assessing Student Learning: A Common Sense Guide. Bolton, MA: Anker.
Tilbury, D., and D. Wortman. 2008. Education for Sustainability in Further and Higher Education: Reflections Along the Journey. Planning for Higher Education 36 (4): 5-16.
U.S. Department of Education. Office of Postsecondary Education. 1993. Buttressing Assessment: AAHE's Assessment Forum. Retrieved October 8, 2009, from the World Wide Web: www.ed.gov/about/offices/list/ope /fipse/lessons2/aahe.html.
University of Delaware. 2006. Assessment Fellows. Retrieved October 8, 2009, from the World Wide Web: www.assessment. udel.edu/UD%20Assessment%20Initiative/fellows.html.
W. K. Kellogg Foundation. n.d. Building Leadership Capacity for the 21st Century: A Report from Global Leadership Scans. Battle Creek, MI: W. K. Kellogg Foundation. Retrieved October 8, 2009, from the World Wide Web: www.wkkf.org/Pubs /CCT/Leadership/WKKF.LeadScan_00312_03800.pdf.
Working Group on Quality Assurance of Student Assessment. 2008. Quality Assurance of Student Assessment: Invitation for Discussion. Utrecht: Working Group on Quality Assurance of Student Assessment. Retrieved October 8, 2009, from the World Wide Web: www.enqa.eu/files/QA%20of%20student %20assessment%20INVITATION%20210208.doc.
Survey Instrument
A culture of assessment provides a foundation for a successful assessment system. The culture of assessment is referred to as shared values, behaviors, and principles that (1) govern assessment and (2) facilitate communication among stakeholders. This culture also ensures that the environment is flexible enough to accommodate change. You are invited to participate in this survey that investigates your institution's initiatives to develop a culture of assessment.
1. Please indicate your level of responsibilities
* Campus wide
* School level (e.g., school of education, school of nursing)
* Other (please specify)_
2. Which of the following applies to your institution's ongoing assessment efforts?
* Assessment is only coordinated at the institutional level
* Assessment is coordinated at the individual school/college level
* Assessment is coordinated at the institutional level while schools/colleges implement their own assessment
3. Please indicate the category that best describes your position
* Faculty
* Administrator
* Other (please specify)--
4. How long have you been in your current position?
* Less than a year
* 1 to 3 years
* 4 to 6 years
* 7 to 9 years
* More than 9 years
5. Assessment practices (please rate the following statements) Strongly Agree--Agree--Neutral--Disagree--Strongly Disagree
a. Assessment is an integrated part of my institution/ school/college's daily practice--
b. Assessment data are systematically available to interested parties--
c. There is ongoing communication about assessment at my institution/school/college--
d. My institution/school/college's leadership devotes time to assessment planning and implementation
e. There is shared understanding of the purpose of assessment at my institution/school/college
6. Referring to your answers to the above questions, would you say that your institution/school/college
* Has established shared principles governing assessment across campus/school/college
* Is establishing shared principles governing assessment across campus/school/college
* Has not established shared principles governing assessment across campus/school/college
7. If applicable, please list your institution/school/college's general principles governing assessment.
8. Students at my institution/school/college participate in the following: (Please check all that apply)
* Planning of assessment
* Design of assessment measures
* Implementation of assessment
* Analysis of assessment data
* Reflection on assessment results
* None of the above
* Other (please specify)--
9. Please indicate the factors that have helped establish a culture of assessment at your institution and elaborate on your selection. (Check all that apply)
* Leadership
* Faculty involvement
* Resources (financial, staff, etc.)
* Student participation
* Access to assessment data and information
* Systematic use of assessment data
* Other (please specify)--
10. Please list the challenges you have faced in establishing a culture of assessment at your current college/university and elaborate on your selection. (Check all that apply)
* Leadership
* Faculty involvement
* Resources (financial, staff, etc.)
* Student participation
* Access to assessment data and information
* Systematic use of assessment data
* Other (please specify)--
11. What strategies are used to overcome the challenges that you listed?
12. Do you work at a two-year or four-year university?
* 2-year
* 4-year
* Other (please specify)--
13. What is your institution's Carnegie Classification?
* Doctoral/Research University
* Master College
* Baccalaureate College-Liberal Arts
* Baccalaureate-General
* Associate College
* Other (please specify)--
14. Is your institution public or private?
* Public
* Private
* Other (please specify)--
15. Additional Comments
Abdou Ndoye is the assessment program director at the Watson School of Education, University of North Carolina Wilmington. He earned his doctorate from the University of Connecticut Neag School of Education and was a member of the university's assessment committee. His research interests include learning program design and assessment methods, assessment plan designs, and evaluation.
Michele A. Parker is an assistant professor in the Educational Leadership Department at the University of North Carolina Wilmington's Watson School of Education. She earned her doctorate in research, statistics, and evaluation at the University of Virginia. She worked in the Office of Institutional Assessment and Studies at the University of Virginia and assisted with the regional accreditation process. Her research interests include teacher education assessment and research methodology.
Figure 3 Summary of Strategies by Stage of Developing an Assessment
Culture
Categories Has Established
(Maturation)
Assessment --Reward collaboration,
integrated in daily including student
practice participation
--Pilot new initiatives
Leadership --Seek more active student
involvement
--Pay constant attention to
the use of data in planning
for the future
Use of assessment data --Provide support and data
reporting software
--Hold annual meetings
between departments
and university to review
assessment plans and use
of data
--Incorporate data in five-year
program reviews
--Create central repository for
data collection and analysis
Communication --Encourage informal
discussions between faculty
from different disciplines
--Maintain direct contact
with student organizations
--Publish assessment
newsletter (twice a
semester)
--Conduct focus groups
with students on more
involvement
Categories Is Establishing
(Progress)
Assessment Use grassroots approach
integrated in daily to convey the value and
practice importance of assessment
Leadership Require new course/program
to have learning outcomes and
assessment for curriculum
committee approval
Use of assessment data Tie budget requests to use
of assessment data
Communication --Offer workshops
--Build relationships with
key actors
--Reduce apprehension and
confusion about assessment
--Showcase examples of
successful assessment
projects
Categories Has Not Established
(Beginning)
Assessment Not integrated into daily
integrated in daily practice
practice
Leadership Limited or lack of experience
in data collection and
analysis
Use of assessment data --Limited use of data
--Administration minimizes
faculty involvement and
control
Communication Share ideas on approaches
at the individual department
level
 Printer friendly
 Cite/link
 Email
 Feedback
| |
| Author: | Ndoye, Abdou; Parker, Michele A. |
|---|---|
| Publication: | Planning for Higher Education |
| Article Type: | Report |
| Geographic Code: | 1USA |
| Date: | Jan 1, 2010 |
| Words: | 6634 |
| Previous Article: | Integrating risk management and strategic planning: integrated risk management and strategic planning leverages the benefits of both processes and... |
| Next Article: | Strategic by design: iterative approaches to educational planning: in the push for accountability, colleges and universities resort to simplistic,... |
| Topics: | |