A Case Study: Engineering a Plan of Action for ERA.

Abstract

Over an 11-year period, Virginia Commonwealth University (VCU) recognized the need for electronic research administration (ERA), developed specifications to meet its need and selected and procured a system. This paper reviews the process of evaluating five ERA systems and winning executive-level commitment to ERA at VCU. In pursuit of this goal, VCU reviewed perceptions of need for ERA at VCU, reviewed the overall research administration structure, identified five potential ERA systems, designed a needs-list for VCU, published a Request for Information and prepared comparative documents for system review. A system was selected with executive support. Implementation is under way.

Introduction

The introduction of modern electronic office and communications technology has made significant changes in university research administration over the last two decades. Perhaps this shift began with the page-to-page storage typewriter, which allowed proposals to be heavily and frequently edited without expenditure of excessive clerical effort. This was a modern-day wonder, just over a decade ago. Or perhaps it truly began as the engineering and "hard" science disciplines harnessed the mainframe computer for their research administrative purposes. Within a few short years, this technology had moved from "the computer suite" to the administrator's desk.

What do you remember of the 1980s? Pop music? Economic decline? At the same time these things were occurring, researchers were investing in technology, and it is this technology that has catapulted research and research administration to a new level.

Today, researchers do not have to run down the hall to the mainframe to update results. Increasingly user-friendly languages, programs and interfaces have made computing a tool for individuals with little or no technical expertise. The development of networking, from the office local area network (LAN) to the Internet, has made it possible for individuals to create, access, manipulate and share massive amounts of information. Proposals and award information can now be transmitted electronically. We have entered the age of Electronic Research Administration (ERA).

Perceptions of Need for ERA at VCU

Virginia Commonwealth University's (VCU) introduction to ERA was an article on electronic proposal processing at Clemson University (Latimer & McCracken, 1988). However, upper-level administration at VCU did not readily embrace the idea of ERA and held to the notion that ERA would require additional staffing and resources, beyond its benefits.

As early as 1987, VCU's Research Advisory Council reported faculty dissatisfaction with the timeliness and comprehensibility of project account financial information provided to principal investigators (PI's) from the enterprise accounting system, itself designed in the late '60s. There was a significant delay between the time a PI requested a budget action and the commitment of costs. Frequently, only a transaction number identified costs, and printouts used accounting jargon unfamiliar to most researchers.

PI's at VCU adapted to these difficulties in one of two ways. In research intensive departments, accounting technicians were employed and shadow accounting systems were developed. This enabled PI's to receive reports on their project's financial status, information necessary for good project management. In units that typically administered fewer awards, PI's were left to fend for themselves, with varying degrees of success.

The VCU Strategic Plan of 1993, developed by the Commission on the Future of the University, called for a significant reduction in the university's administrative costs, the elimination of unnecessary paperwork and a greater reliance on information technology. A study by KPMG Peat Marwick concluded that VCU should implement a new financial information system and make a number of other dramatic changes to the research support systems.

In response, a new Financial Access Network (FAN) was initiated in 1996. It was to be an enterprise-wide system linking all financial/administrative activities through a data warehouse, with the goal of decentralizing processes and accountability. For several reasons, this venture collapsed, with a loss of significant time and other resources. As a result, VCU's senior officials became somewhat distrustful of complex, highly specialized, electronic systems for administration.

The next step in reengineering processes to support researchers' needs and reduce administrative costs was a two-part study to identify ways to strengthen VCU's pre-award and post-award sponsored program processes, initiated in 1997. The Process Owners Team, comprised of users of these services, was charged with the task of mapping the processes and identifying factors reducing efficiency. Based on the 1997 Process Owners Team report of current VCU practices, an Envisioning Team, consisting of users and deliverers of these services was created. The Envisioning Team was responsible for making recommendations to increase research at VCU, a goal in VCU's second long-range plan.

The Envisioning Team released a report on their improvement recommendations in 1998. The report recommended that VCU increase its support for research administration at both the central and decentralized levels and develop electronic proposal capacity in the near future. Approval and dissemination of this report was delayed several months, possibly because its recommendations regarding increased administrative activity and investment ran counter to executive expectations.

As VCU struggled internally, federally mandated Electronic Research Administration (ERA) became a reality. This, along with increasing competition for research funding, pressure for compliance accountability, and other factors, forced VCU, as well as many other institutions of higher education, to reevaluate its support for the research enterprise. The following section describes how VCU has dealt with this challenge.

VCU's Research Administration Structure

Two separate offices administer VCU's pre-award and post-award functions. Focusing largely on pre-award matters, the Office of Sponsored Programs Administration (OSPA) reviews and approves sponsored program proposals, negotiates agreements with sponsors and interprets regulatory and contractual requirements. It reports to the vice president for research and is located on the Medical Campus of VCU, approximately two miles from the Academic Campus. OSPA used a homegrown database in Microsoft[R] Access to record proposal and award demographic and financial data. Enhancements to this program helped staff deal with the preparation of agreement and sub-agreement documents. For reasons of security, this database was not viewable, or useful, outside of the office.

The office of Grants and Contracts Accounting (G&CA) at VCU establishes accounts and maintains accounting records for all sponsored research. It also prepares financial reports for sponsors, prepares bills in order to recover direct and indirect costs and interprets financial requirements. G&CA reports to the vice president for finance and administration and is located near the Academic Campus. Although OSPA's database could provide at least half of the information electronically that is needed to establish an account, resources were not made available to generate the necessary interface to the legacy accounting system.

Regulatory Compliance Offices such as the Institutional Review Board, the Animal Care and Use Committee and the Radiation Safety Committee are independent of one another, reporting directly to the vice president for research. Their administrative offices are located exclusively on the Medical Campus. On these two urban campuses, parking is expensive and frequently unavailable. Even though there is a shuttle bus running twice an hour, moving people and papers is inconvenient, at best.

Thus, in 1997, there were significant tensions affecting VCU's research enterprise. Researchers wanted more support and administration wanted less cost. Researchers wanted faster and more convenient support; research administrators faced increasing demands from sponsors without any prospect of adding more personnel. Communication among constituencies was poor, and duplicative data entry was endemic. Electronic communication was hindered by the variances in equipment; PCs did not reliably talk to Macs; desktops did not easily talk to mainframes. The 21st century was almost upon us, and we were not ready for it!

The ERA Systems Review at VCU

This confusion was leading different groups within VCU to seek and to advocate for separate solutions to their own perceived (and conflicting) ERA needs. This, and the perception that electronic proposal submission would soon be mandatory, led VCU to seek a study of research administration software systems available or under development. The objectives of this study were to identify current and future capabilities, implementation costs and ongoing support costs.

It was clear that evaluating VCU's needs would require more staff time and energy than was currently available. Also, it was apparent that all factions within VCU needed to be certain that the evaluation process would be objective. The charge was to hire an outside consultant to design and prepare an analysis of available ERA systems so that VCU could choose those they wished to see demonstrated and, finally, to purchase.

The following plan was proposed to the VCU administration by the outside consultant. With the plan approved, the tasks were carried out and the resulting deliverables formed the body of a guidebook for identifying appropriate ERA technology:

Task 1: Interview the process owner team members regarding specific requirements within their areas and where they see opportunities to share data electronically.

Deliverable: Interview log and summary of area needs by category.

Task 2: Draft and execute a Request for Information (RFI) targeting a select group of five ERA Systems Engineers and Consultants.

Deliverable: RFI publication including distribution and management (log) of all inquiries

Task 3: Collect individual vendor proposals (responses to the RFI) drafted in response to VCU's unique needs.

Deliverable: RFI individual responses, collective response document (comparing responses by area and category)

Task 4: Compare responses objectively

Deliverable: Collective response document comparing responses by area and category (e.g., area=security, category=password functions)

Task 5: Develop a tool for summary review of systems

Deliverable: Using a matrix of VCU needs and system specifications, the RFI responses were registered and scored according to how each particular need was addressed: (function or feature available as described, available with similar function, in development, considered as additional cost/add on, or not available). This data was also compiled into a Systems Review Summary Report.

Task 6: Compile a final report and executive summary

Deliverable: Compilation of the following in reference manual form. This reference tool became the mechanism for selecting vendors to interview, baseline knowledge of system capabilities and dramatically reduced the time spent reviewing marketing materials, system specification lists and product descriptions.

Designing a Needs List for VCU

Although their reports had not been released, the process owners team and the envisioning team were adamant: "Electronic submission software should be explored immediately." Based on their input, as well as that of OSPA, the minimum requirements included:

1. Formalized faculty training to include on-line guides and templates;

2. Automated matching of faculty interests with funding opportunities, including electronic notification of funding agency deadlines and sponsorship areas;

3. Tracking and scheduling research risk applications, proposals and the associated deliverables;

4. Collaborative editing and review of research risk applications, proposals, subcontracts and project results;

5. Automated routing of documents for the review and approval process;

6. A security system that would meet all department, institutional, industry and federal requirements;

7. Electronic proposal submission;

8. Electronic processing of award, including notifications, updating of terms and conditions and account set-up;

9. Electronic award management, including subcontract management, modifications and reallocations;

10. Electronic deliverables and reporting management;

11. Integration with legacy accounting system FRS (DB/2);

12. Integration with human resources system (internal); and

13. Electronic closeout procedures.

From these 13 items, interviews with each of the team members resulted in 141 specific requirements, which were easily grouped into the categories listed in Table 1.

With each specific requirement detailed and categorized, one approach would have been to review each vendor's literature regarding its capabilities in each area. However, the sheer enormity and inconsistency of the marketing literature was daunting. VCU opted for using a directive approach. We put our goal, objectives and needs out in the marketplace in the form of a Request for Information or RFI and requested vendors to make a direct response.

Designing a Request for Information

Armed with very specific minimum requirements, drafting the Request for Information (RFI) document was a matter of translating the above needs into capability inquiry statements, developing a timeline and process for respondents, setting a deadline for questions to be answered and developing a tool for summarizing system capabilities.

The "Request for Information" document solicited very specific information from a group of vendors who had marketed their capabilities for electronic research administration. We presented a brief introduction and background information about the VCU system. To encourage vendor response, we also described our plan to review ERA systems for the purpose of identifying a vendor and purchasing ERA tools based on our needs.

RFI's must contain certain information. For example, vendors need to know how to reach you regarding questions that they may have about not only your specific needs, but also about the appropriateness of their investment in providing a response. Some of the necessary sections includes:

1. Information about your organization

2. The deadline for responses from vendors

3. The deadline to receive all questions

4. Acceptable methods of response

5. Disclosure restrictions

6. RFI instructions

7. Application description

Organizational Information: How do your pre-award and post-award systems function? What is your organizational structure? Are there legacy systems in place that your institution is "married to" versus "simply settling with for now"? On the part of VCU, of greatest concern was using a consultant to devise and manage the RH. It was necessary to have many joint meetings with the consultant and work closely when responding to specific vendor queries during the question period.

Deadline: The importance of setting an appropriate deadline and making this information immediately known is quite obvious. But, what is a reasonable period of time for vendors to respond? VCU allowed for five weeks. We determined that several of our targeted vendors would be able to respond within a two-to-three-week period of time, so allowing for double the time seemed not only fair, but also certainly adequate.

Interestingly enough, one company was able to respond within two weeks and the remaining responses were, literally, down to the wire. Regardless, the imperative is not to make exceptions, thus allowing ample time for all your target vendors. A great way to do this is simply call all targeted vendors and announce the upcoming RFI and advise that the response time will be limited to X weeks. This affords the vendor with an opportunity to make you aware of any "new product releases" or other extenuating circumstances which might conflict with providing you with up-to-date information within their response.

Questions: Because all of us are fallible, it is important to provide an opportunity for RFI respondents to request clarification regarding the description of your needs or key details regarding your technological environment. We found that each vendor did take an opportunity to ask questions within the period allowed (up until the due date, in our case). The simple rule exists with RFI's, the better information put out, the better the information brought in.

Methods of Response: VCU allowed for two methods of response, electronic and hard copy. In other words, the vendors needed to respond to the direct questions of the RFI in one of those two ways. It was made clear that simply providing a "demonstration copy" of the ERA system software would not meet the requirements of the RH, unless provided in addition to the response to the direct inquiry statements (outlined in the RFI).

Disclosure Restrictions: VCU wanted as much detail as possible about each vendor's capabilities for providing a system that would meet the institution's specific needs. In some cases, this meant that the company had to reveal capabilities recently developed or under development (disclosing the status of availability). For the purpose of confidentiality, we provided a disclosure statement indicating that all information provided in response to the RFI would be subject to internal review at VCU only for the purposes of leading to the identification of an ERA software system. Furthermore, we requested that all information that should be held strictly confidential be so marked. Information so marked would not be disclosed beyond the purpose described within the RFI, unless required by state or federal law.

RFI Instructions: Within the RFI we included overall instructions and specific instructions for each section. This information is presented in Table 2.

Compiling Responses/Final Report

Once the RFI responses were received, compiling them was relatively simple, largely because all but one vendor provided an electronic document in conjunction with a hard copy. Two documents were made using the electronic responses, a Collective Response Document and a System Analysis Grid.

The Collective Response Document contained all the VCU requirements, section by section, and outlined each vendor response. In this way, "apples were compared to apples," so to speak. This document served as the primary reference document for a number of individuals and groups at VCU who had been informed by one vendor or another -- or by their advocates -- that only their preferred system satisfied some crucial need. Sometimes a claim proved true, but frequently it became clear that another system could provide satisfactory results.

The System Analysis Comparison Grid (see Table 3) was a more structured document in which a coding system was used to "grade" the function and availability of each feature required by VCU. This became a very powerful tool for detailed analysis of the data acquired from the RFI. A sample page is attached, with individual vendors' names omitted. The coding system that allowed us to "tally" results and help support system preferences is presented in Table 4.

The final report (Ballance, 1997) provided VCU a comprehensive review of five ERA systems. This report dramatically reduced the time VCU's personnel spent reviewing marketing materials, conducting site visits, evaluating recommendations and advancing toward a decision. This reference tool was made available to VCU's decision-makers as a simple three-ring binder containing the following:

1. Executive Summary

2. ERA Systems Review Summary Report

3. Collective Response Document

4. System Analysis Comparison Grid

5. Supporting Documents

* Process Owner Team Results

* RFI Distribution List

* Request for Information

* Interview Log

* Grant Application Transaction Set-194 (partial)

* Internal Reference Resources

Final Procurement Decision

With the final report in hand, VCU's OSPA personnel rapidly were able to determine that three of the systems could not reasonably be used by VCU. One system, for example, involved participation in a multi-institution consortium during development. However good the final result might be, VCU was now gun-shy of that approach. Another vendor required use of a specific enterprise-wide database, and VCU's Office of Information Technology had committed to a different one, thus, no technical support could be made available.

In late 1997, an ad hoc committee was developed to advise the vice president for research on which system VCU should procure. It included such users as researchers, departmental administrators, OSPA, G&C Accounting and such support or other interested parties as OIT and the financial advisors to the vice presidents of each campus.

These groups reviewed the Executive Summary and the ERA Systems Review Summary Report, with the entire report on hand if detailed answers were needed. Two vendors were invited to give on-campus presentations. These presentations demonstrated that in at least two crucial areas, one vendor's product better suited VCU's needs. The vendor that best suited the specific and unique needs of VCU was invited to make a quotation on the desired portions of its system. On April 29, 1998, the vice president for research accepted this quotation.

The vendor was clearly aware that the funding for such a significant purchase would not be available until VCU was operating on the following year's budget. The need for an ERA system had been perceived by researchers and their supporting constituencies, and had been brought to executive attention in the process of executive search for cost reductions! It was only by convincing executive management that proposals to federal agencies would need to be submitted electronically in a few years, that executive management was willing to consider the expense. During that year, the final report was referred to frequently in demonstrating reasons for selection of the final vendor.

On July 1, 1999, funds for procurement of the selected system were made available. Executive management continued to have doubts of the wisdom of the procurement. Finally, in August 1999, a group of five VCU senior personnel visited the vendor's establishment. A further presentation, discussion with referenced institutions, and the development of a trustworthy relationship persuaded the vice president for research to commit to the purchase.

Then, of course, it was necessary to process a sole source procurement. VCU's procurement system had to meet state requirements. The final report was made available to the purchasing agent as reference material. "You've really done your homework!" was one of the most rewarding comments heard!

Oh, yes! The purchase order was issued on September 29, 1999, and implementation of VCU's ERA system is under way as the last words of this paper are being written.

Lisa R. Ballance, BA, CCRA, ("A Case Study: Engineering a Plan of Action for ERA") is Associate Professor and Administrative Director of the Virginia Commonwealth University (VCU) Clinical Trials Institute and Outcomes Research Institute, located in the Virginia Biotechnology Research Park in Richmond, Va. She was recently named the Director for Restructuring of the VCU Institutional Review Board. Ms. Ballance has worked with a group of colleagues (including SRA, DIA and ACRP members) to found the Academic Health Center Clinical Research Working Group.

Herbert B. Chermside, CRA, ("A Case Study: Engineering a Plan of Action for ERA") is Director of Sponsored Programs Administration for Virginia Commonwealth University. A member of SRA since 1969, he has held previous research administration positions at the University of San Francisco, the Veterans Administration Hospital, San Francisco, the University of Virginia and Virginia Tech. He is on the Board of Directors and Treasurer of the Research Administrators Certification Council, serves as VCU's Primary Representative to COGR and is a long-time member of NCURA.

References

(1.) Latimer, D.A. & McCracken, A.L. (1988) Electronic Proposal Processing at Clemson University, SRA Journal, fall, 23-29.

(2.) Ballance, Lisa R., Electronic Research Administration Systems Review Final Report, October 2, 1997, in VCU OSPA files.

Table 1: Categories of Concern

General Requirements            18 items
Security                         4 items
Sponsored Programs Development  36 items
Pre-Award Tasks                 31 items
Research Risk Management        24 items
Post-Award Tasks                12 items
Reporting                        8 items
Technical                        8 items

Table 2: RFI Instructions

INSTRUCTIONS:

The ERA Systems Provider should respond to any or all parts of this RFI. This RFI refers to the request for information regarding a "total" business need including hardware requirements, software, training, documentation and other areas as required to develop a total solution. Alternative solutions (e.g., system contractor) will be considered.

Section 1.0 Summary

A brief description of your product, its major strengths, how it meets VCU's requirements and a synopsis of support options.

Section 2.0 Solution Description

A technical overview of your product, including laymen's terminology as well as technical descriptions. Existing product literature may be included here.

Section 3.0 Hardware/Software Requirements

Responses must include a description of all hardware and software required. In addition, recommendations must be included for network, size and configuration of server, storage type and capacity, and other system requirements to operate the system to meet VCU performance requirements.

Section 4.0 Service and Support

Describe service and support offerings and price. Indicate options available.

Section 5.0 Company Profile

A brief description and history of your company, including the date founded, whether it is publicly or privately held, and any business partnerships pertinent to this RFP. Describe the experience, capabilities and qualifications of your company within the industry. Include an overview of your current ERA activities and customers.

Section 6.0 Pricing

Provide detailed pricing information for components proposed. Provide software pricing broken down by module and/or feature. Include itemized costs for installation, data conversion, training and maintenance. Describe any other costs associated with the system purchase (e.g., integration utility and/or programming costs). Also, describe any potential payment schedules.

Section 7.0 Vendor Experience

Responses must include information about the vendor's past experience with computing systems (hardware and software), research and sponsored programs, contracts and grants, training, documentation and other areas related to this effort.

The names of three clients from a university or research environment for which similar work has been completed must be attached. This list should include a contact person, address and telephone number. The list should also include a brief description of the solution provided.

Section 8.0 Application Description

In reference to the Application Description document, attached, responses should be clearly stated, with comments describing function (or alternative approach), as necessary. In this case, providing more information, within the "Comments" section is strongly encouraged, as it may clarify your approach.

Table 3: The System Analysis Comparison Grid

available:    standard system feature  3
planned:      described for future     2
limited:      applicability to VCU     1
fee appl:     will chg fee to build    0
not avail:    feature not described    0
n/a:          function not available   0
alternative:  not as described         1
a/p:          planned exp.             2
font chg:     need to review
                    SYSTEM       SYSTEM       SYSTEM       SYSTEM
                       A            B            C            D

General
Requirements

        1         available 3  available 3  available 3  available 3
        2         planned   2  available 3  available 3  available 3
        3         available 3  available 3  fee-appl  0  available 3
        4         available 3  Available 3  available 3  available 3
        5         available 3  Available 3  available 3  available 3
        6         available 3  Available 3  available 3  available 3
        7         available 3  Available 3  available 3  available 3
        8         available 3  a/p       3  available 3  available 3
        9         available 3  Available 3  available 3  available 3
       10         available 3  a/p       3  available 3  available 3
       11         available 3  Available 3  available 3  available 3
       12         planned   2  Available 3  available 3  fee appl  0
       13         available 3  Available 3  available 3  fee appl  1
       14         available 3  Available 3  n/a       0  fee appl  0
       15         available 3  Available 3  available 3  available 3
       16         planned   2  Planned   2  planned   2  planned   2
       17         available 3  Available 3  available 3  available 3
       18         available 3  Available 3  available 3  not avail 0
                      51           52           47           41

Security Minimum
Requirements

        1         available 3  Available 3  available 3  available 3

        2         available 3  net only  1  available 3  available 3


                     SYSTEM
                        E

General
Requirements

        1         available   3
        2         planned     2
        3         available   3
        4         available   3
        5         available   3
        6         available   3
        7         available   3
        8         not avai    0
        9         alternative 1
       10         available   3
       11         ?
       12         ?
       13         ?
       14         ?
       15         available   3
       16         planned     2
       17         available   3
       18         available   3
                       35

Security Minimum
Requirements

        1         planned     2
        2
        2         available   3
Table 4: Coding System

TEXT CODE            DESCRIPTION OF FEATURE

Available            Standard system feature/function
Planned              Confirmed for future release/development
Alternative/Planned  Function expansion planned for future
Limited              Not specific to VCU requirements
Alternative          Function available, not with features
Fee Applied          Will charge a fee to build or modify
Not Available        Function not available/not planned
Italics              Needs further review

TEXT CODE            VALUE ASSIGNED

Available                  3
Planned                    2
Alternative/Planned        2.5
Limited                    1
Alternative                1
Fee Applied                0
Not Available              0
Italics