Constructing a grants office review: a case study.

Introduction

Working within an academic context one may find it uncomfortable (and even a little repugnant) to think about the college or university in terms of efficiency, productivity, and bottom lines. The notion of accountability seems more appropriate to he corporate and government sectors and belies the more noble goals of institutions of higher education to inspire young intellects, nurture good citizens, and explore ideas. And yet all institutions now face accrediting reviews that include questions on assessment practices, judging how well the institution is meeting its mission. This nation-wide push for accountability affects the curriculum and the performances of faculty and staff and often encourages our grants and sponsored program office management to expect that the exercise will lead to improvements.

While the movement toward increasing assessment is affecting our office staff and activities, the challenge is to find an appropriate and useful way to evaluate the sponsored program office that will lead to distinct opportunities, challenge the status quo, and establish practices for improvement. This paper describes a distinct model that is useful for organizing an assessment of grants offices. Even though the review process could be constructive for strategic and other planning, the model presented here has inherent value for a grants office.

Because a review systematically investigates activities and rigorously evaluates outcomes, its usefulness depends on the priority it receives. Fundamentally, an assessment should be helpful in gathering information about the current status of activities and products and in pointing at areas that could be improved. When given top priority, the review can serve several goals; it can, for example, a) enhance the capacity of the grants office; b) increase the services offered; c) alter policies, practices, and structures; and d) enable the faculty to seek more grants. And yet a review can provide an opportunity for much more: marketing the grants office, making administration aware of needs, developing a plan for growth that has support from others, and organizing for change. How then can this process be managed so the results are valid and the process serves a variety of goals? Deciding on the framework for the assessment requires some reflection on the level of the analysis--the context--to be researched.

Stockton Context

Even though the academic programs are required to undergo self-studies and reviews every five years at Stockton, the academic service programs had never been assessed. Every year the administrative staff is required to produce a self-evaluation which often involves setting professional goals and evaluating the effects of office management. Since the grants office at Stockton is a one-person shop, a self-evaluation is, by default, an evaluation of context as well.

This situation could suggest that a review of a small grants office might be so narrow as to have limited consequences. And yet the College, as a public liberal arts, primarily undergraduate college with approximately 6,500 students, is poised for institutional transformation: the state allocation has been cut, creating more interest in outside funding; a new president began in June 2003, bringing a robust interest in building relationships outside the college, and inspiring the College community to reflect on ways it meets the mission, and bringing a new mandate to establish goals and achieve results. In other words, the College is ripe for growth, and a well-structured review process could launch the grants office into the mainstream. Whatever the state or context of an institution, a thoughtfully created assessment can serve as an initiative for change.

Model for the Review Process

The literature on assessment is extensive and will provide a wide variety of general terms and practices. Since an assessment review is basically a research project, we can look to research methods to guide the process. A more traditional scientific model of research begins with a theory for which there is no clear answer (appropriate so far for the grants office review); then we devise research questions (still pertinent) and a hypothesis based on logically deduced and observable outcomes (and this is where this model gets tricky for a grants office review). While this research method is focused, it can be too narrow and linear for our purposes. A program review of the grants office needs more flexibility and broader input than a more traditional research model can manage. The social scientists will think of participatory action research (Wadsworth 1998) or a more open-ended and experiential-based approach.

The model outlined here as appropriate for grants offices relies on the theories and goals of a community-based research (CBR) assessment. If the grants office is considered the place that facilitates change, CBR provides guiding principles and practices that are useful (Stoecker, 2002). By adapting our goals to a CBR approach, the grants office review will be collaborative, sensitive to the college culture and context, systematic and rigorous in gathering data, multidimensional, and anticipative of change.

Guiding Questions to Consider

As Patton (1997) wrote, "There is no best way to conduct an evaluation. It depends on the people involved and their situation." Even though each office has its own culture, a program review has characteristics common to most offices which can be discovered through a variety of data-gathering tools that include both qualitative and quantitative data. In designing the framework for the review, the following broad dimensions inform what methods and activities can best document the process: a) What is the goal of the review? b) What are the deeper purposes that might be served in this process? (For example, a grants office can he a change agent, which means including more than charts and bottom lines.) c) What information and data are relevant? d) What role might an outside evaluator play in the process?

Once the general questions are considered that will frame the assessment, the review will take on a more practical and pedestrian pace. From the first meeting with administration to the end of the process, one might spend eight or nine months, and implementing the recommendations and plans for change might require several years. To monitor progress of the office review, outline a process and note the concepts that will characterize the assessment. Concepts might include the following:

1. The grants office mission is basic to the whole review. Is it working? Do staff attend to all it says? Are there other activities outside the mission that occupy the staff?

2. Review job descriptions. What are the big tasks that require staff time? Internal/external grants, compliance issues, fundraising, pre-award and post-award.

3. Who are the constituents? Faculty, student affairs, administration, community partners.

4. What are "best practices" for similar offices? And how can they contribute toward improving this office?

And think about parameters that will guide the kinds of information to gather:

1. What context(s) might be used for comparisons? Other institutions within the same state, across the country, within the same Carnegie classification, with a similarly-sized private/public--all can provide some interesting information.

2. What quantitative data are essential? Has there been an institutional self-study within the past few years? (These documents contain useful information)

3. What data will show activity and trends without becoming mere numbers?

4. What methods are best to gather qualitative information?

5. What timeline is reasonable, given one's other responsibilities?

6. How might an outside consultant best serve the goals of the review?

Process--Road Best Taken

Since the review may take close to a year to collect and analyze all the documentation and write the self-study report, this will be a project with long interruptions while one tends to regular responsibilities. A matrix can keep the process organized by tracking the activities/methods, the processes by which activities will be complete, the resources needed, a target date and completion date.

The review process requires commitment of time and work, so one must gain approval and support from the administration for the plan early in the process. Toward this end, one should craft a preliminary draft of the process to be followed, a timeline, a request for an outside evaluator, and a table of contents for the self-study report. Once a plan is given the go-ahead and a budget established, the review can proceed on several fronts.

The table of contents for the self-study extends in several sections with subsections, so that gathering information and writing can begin. The self-study report expands to include the following sections:

a) Overview--background/history of the office, mission and values;

b) Infrastructure--outreach and marketing activities, technology and information resources, staff; budget, and compliance activities;

c) External award activity--policies and procedures, outcomes, examples;

d) Internal faculty development programs--policies and procedures, outcomes;

e) Comparisons and context--nationally, statewide, interdepartmental;

f) Anticipated trends;

g) Conclusions--policy implications, resources needed, future activities; and

h) Appendices--job descriptions, organizational chart, external grants activity summary, internal grants summary.

Gathering data for comparisons and information from various stakeholders often requires developing surveys for more local distribution and researching outcomes from national studies. National data on grants and sponsored research offices is scarce and often has limited value when compared to one's own institution. (1) The colleges and universities within one's own state frequently provide useful comparisons since these may be familiar to the administration and often have similar traditions. In Stockton's case, a survey was developed and sent to the grants offices at the other eight public colleges within New Jersey. With follow-up phone calls, a 100% response rate was achieved.

If the process is to be collaborative and sensitive to the college culture, the stakeholders within the college also need an opportunity to participate in and inform the review. Another survey can gather information from faculty members via e-mail to ask about their grant-seeking history, experiences with the grants office, their problems and recommendations. Space can be given for written responses for some questions, and a Likert scale response on others. Similarly, a survey developed for the academic deans elicits an administrator's point of view. These data can then be collated and summarized for the report and narrative responses selectively quoted in the Self-Study Report to add richness and other voices.

Professional associations and colleagues' recommendations are effective sources for external evaluators. The evaluator is in a position to give guidance and advice to upper administrators, so select someone from outside the college who has the effect of a top administrator and is also someone who shares your vision and with whom you can collaborate. This person is important to the process for providing balance of opinions. The evaluator's voice carries weight and authority., so the choice must be some one with experience and intelligence. The evaluator for Stockton developed guiding questions for personal and group interviews before arriving on campus to give these conversations some consistency. The consultant should meet with a variety of stakeholders across the college including academic affairs vice presidents, administration and finance, academic deans and group sessions with funded project directors and faculty who are not project directors but have the potential to become so. The variety of opinions and perspectives should give the consultant a more inclusive description of grants office activity. The consultant's report becomes an important reference document during the final stages of the review process. Once the report arrives, the grants office director ought to respond in writing with some objectives to meet the consultant's recommendations and an action plan for implementation. These documents become the basis for later discussions on ways to best develop the grants office.

Lessons Learned

Framing a review within the methods of community based research assessment employs a feedback process in which stakeholders are asked for advice, make recommendations, and then are solicited to respond to the consequences of the review. Thus both the review and its implementation are collaborative. Encouraging all the stakeholders to give voice to their opinions about the grants office will also mean that they have been "prepped" to help implement the recommendations. Changes may be more welcome, because a broad group has already invested in the outcomes.

The review process largely reflects the experiences and practice of the staff in the office--its culture. Hearing from the grants office workers and reflecting on their relationships will help formulate guiding questions and appropriate methods. This is the value-laden premise of a review--to improve the efficacy of the grants office so that the opportunities for grant-seeking can be improved.

Check and recheck the data and information included in the self-study, because the report will become a reference for administration for several years.

Keep the review focused on the process and not the staff. An office review is not an evaluation of personnel. Be vigilant that the review does not deteriorate into complaints and criticisms about personalities.

Staff should request at the outset that the review process include a budget, especially when hiring an outside consultant.

The process takes time to complete. Keep the process alive by sharing "marker moments" with others and advocate that the recommendations remain on the agenda.

Value and Outcomes

Assessment looms over us all as education moves through an active period of new ideas and new structures. The process of review requires some effort but then the documents that accumulate as a result provide fertile groundwork for leading the grants office in new directions. These documents become invaluable records for the college community and for the grants office in planning for change and when they need to show results. Once a review cycle is completed, the baseline data is established and information gathered that will be useful as comparison in the next review cycle. During the year or so of the review, and then another year or so of implementation, the grants office becomes a focus of attention and activity. Especially during this time there will be opportunities to press for change in policy, to market the grants office, to promote the grants office in strategic planning sessions and to position the grants office for growth.

Even though at Stockton we are still in the process of implementing the recommendations that emerged from the review process, the assessment established that the grants office was operating on limited resources that limited its activities. Since the review, the director has been promoted, a new clerical position has been approved which will increase the capacity of the office, a task force has been established with faculty and administrators to redesign the internal award programs, and new incentives are under investigation to encourage more grant-seeking. Perhaps as important is the positive and supportive message widely delivered by the upper administration about the grants office and its value to the institution.

Conclusions

An assessment of the office can be helpful and productive. The reports make a clear statement of need and prioritize what matters most to all the stakeholders. By collecting and documenting the activities of the grants office, the process establishes where things stand and gives some authority to those assumptions. This documentation can then become a vehicle for consensus-building to increase support for the grants office. Even though a review is time-consuming, the process gives legitimacy to the needs of the grants office and can become critical to its growth.

Author's Note: This paper was originally presented as a Roundtable discussion at the American Association of Colleges and Universities, Grants Resource Center Spring 2002 conference. The author appreciates and acknowledges the assistance of the editorial reviewers. Contact Beth Olsen, Director of Grants Development, Office of Academic Affairs, Richard Stockton College of NJ, PO Box 195, Pomona, NJ 08240-0195, USA. Ph: [phone]. Fax: [phone], Email: [e-mail].

References

Greenwood, D.J. & Levin, M.N. (1998). Introduction to action research: Social research for social change. Thousand Oaks, CA: Sage Publications.

Kirby, W. & Waugaman, P. (2003). Moving to Best Practices in Sponsored Programs Administration. Retrieved August 16, 2005, from http://www.t-c-group.com/tt5.pdf

Leavitt, J. (1999). Implementation-Designing and Conducting an Evaluation. Retrieved August 16, 2005, from www.si.umich.edu/Community/con nections/archives/evaluationImple ment.html

Lowry, P. & Hansen, S. (2001, Winter/Spring). Reconsidering Research Administration at Predominantly Undergraduate Colleges and Universities. Research Management Review, 12(1), 11-15.

Patton. M (1997) Utilization-focused evaluation: the new century text. Thousand Oaks, CA: Sage Publications.

Stoecker, R. (2002). Thinking about CBR: Some questions as we begin. Retrieved August 16, 2005, from http://comm-org.utoledo.edu/drafts/cbrqs.htm

Wadsworth, Y. (1998). What is participatory action research? Action Research International, Paper 2. Retrieved August 16, 2005, from http://www.scu.edu.au/schools/gcm /ar/ari/p-ywadsworth98.html

According to the Middle States Commission on Higher Education. (2002). Designs for Excellence: Handbook for Institutional Self-Study. Philadelphia, PA: Middle States Commission on Higher Education.

(1) See Kirby and Waugaman (2003) for national benchmarking study of sponsored programs offices.

Beth Olsen

Richard Stockton College of NJ