Application of standard project management tools to research--a case study from a multi-national clinical trial.

Introduction

Although the pharmaceutical industry has well-developed project management methodologies for research, it is unusual for academic researchers working in the education and public sectors to do so. The discipline that these tools impose can appear alien initially and often require cultural change for the potential value they can bring to be recognised.

This paper examines the experience of introducing a standard project management tool, PRINCE2, to the management of a large Phase III clinical trial, the Microbicides Development Programme (MDP). Phase III clinical trials are usually undertaken by the pharmaceutical industry. Somewhat unusually, the MDP is publicly funded and managed by a partnership of academic bodies. Funding is provided by the UK Department for International Development (DFID) and the programme is coordinated by the Medical Research Council Clinical Trials Unit, UK and Imperial College London, UK. The trial sites themselves are in Africa.

The complexity of this particular trial, and the need to communicate and monitor progress against budget in a standard format to the funder, DFID, prompted senior academic staff to modify their approach to management and reporting through adopting elements of PRINCE2. This has proved beneficial for both the trial team and DFID.

The paper describes what was done in the MDP case and discusses the costs and benefits of adopting a similar approach more widely in conducting academic-led clinical trials.

The Microbicides Development Programme

The Microbicides Development Programme (MDP) is a partnership to develop vaginal microbicides for the prevention of HIV transmission, funded by the UK Department for International Development (DFID) and the UK Medical Research Council, and coordinated by the Medical Research Council Clinical Trials Unit, UK and Imperial College London, UK. The central goal of the Partnership is to complete a Phase III trial of candidate microbicides in Africa. Phase III trials are randomised controlled trials on large patient/healthy volunteer groups (often enrolling several thousand individuals), and are aimed at definitively assessing the efficacy of a new therapy or prevention. Phase III trials are invariably expensive, time-consuming and complex to design and run. These trials look at whether the new treatment works and at any side effects it may cause.

The MDP budget is GBP 42M (USD 75M) and involves thirteen principal scientific partner institutions, six of which are African. A large number of scientists and clinicians are involved in programme management functions in addition to their own areas of particular expertise. There are also a number of people focusing on specific areas, such as trial management and communication.

Given the size of the management "burden" of a Phase III clinical trial and the need to communicate progress in a standard format to the funder, DFID, in a way that would reflect both the customary approaches to trial management and DFID's usual approach to reporting on projects (not designed specifically for clinical trials), senior academic staff opted to adopt an approach to project management based on PRINCE2 methodology.

Features of PRINCE2

PRINCE is a structured method for achieving effective project management that has evolved in the UK. It was first established in 1989 by the UK Central Computer and Telecommunications Agency as a standard to be used for all government IT projects, and was subsequently developed as an approach to project management for all projects. Since 1996 it has been a standard requirement that UK public sector projects are run using this version of the approach, PRINCE2.

Key features of the PRINCE2 approach include: 1) a clear business case, which sets out the aims of the project; 2) a defined and measurable set of "products" or results, together with the activities to achieve them; 3) defined resources linked to activities; and 4) an organisational structure, with defined responsibilities to manage the project (UK Office of Government Commerce) (Figure 1).

[FIGURE 1 OMITTED]

Typically, these features are captured in a set of project documents against which aims and progress are monitored, risks identified and managed, and changes to aims or activities controlled. The set of documents includes a project initiation document (PID), risk register, issues log, project plan and statement of success criteria.

The PRINCE2 approach is not intended to cover all aspects of management for every project, and the techniques and tools may vary according to the type of project and organisation carrying it out. Some aspects of project management are well covered by other well-proven methods, including people management techniques, generic planning approaches (e.g., Gantt charts, critical path analysis) and methods for controlling budgets. PRINCE2 is a coherent set of project management concepts and processes that provides a minimum set of requirements for a properly run project. But the approach fully recognises that each project may vary substantially, and that the particular approach to effective project management will require tailoring of the overall method.

PRINCE2 requires the production of a summary reporting document to a steering group and a related set of supporting documents and processes. The set of documents that were considered most appropriate for use in the MDP case included the following:

1. A Project Initiation Document (PID)--to summarise in one place the aims of the project, an outline project plan for all activities and deliverables by all parties, resources and budgets, key project dependencies (including critical external dependencies, e.g,, supply of the gel and ethics committee approvals), reporting processes and governance structure, risks, and change control procedures.

2. Detailed Project Plan--a consolidated overall plan of key deliverables, milestones and timescales.

3. Financial Controls and Reporting Procedures--these include financial profiles that link budgets and expenditures to activities, as well as the associated monitoring and corrective action procedures. In the MDP case the approach that was already being taken was modified to provide clearer reporting on progress with the trials (e.g., recruiting trial participants at the trial sites), and matching this progress against proportion of budget used. Financial spreadsheets were adapted to produce automatically graphical summaries for use by the project team and in reporting to DFID (Figure 2).

4. Risk Register--the majority of risks in the MDP case were already being anticipated and recorded by the trial management team, but not easily communicated to DFID. This document collated this information according to the groups of activities. Probability and severity of risks were also noted so that priorities could be determined. A "traffic light" warning system was employed to readily prioritise any risks.

5. Issues Log--to record risks that have become reality and identify what is being done to address them and by whom.

[FIGURE 2 OMITTED]

As the PRINCE2 approach was being applied to a project that had already started, it was decided not to create the PID (as the relevant documentation already existed, albeit not in one single document). Emphasis was therefore placed on modifying the approach to financial monitoring and reporting, and on identifying and reporting risks and issues.

Costs: The MDP Case

In the MDP case an existing approach to project management was modified. The information required was already being collected and, to a large extent, all of the project management functions implied by PRINCE2 were being implemented. However, these were not organised in a way that lent itself readily to linking progress and planning of expenditures to activities, or to reporting in a transparent way that could be easily communicated to the funder in a standard format.

Table 1 summarises the costs associated with modifying the existing project management approach for MDP.

It is likely that these costs would have been lower had a PRINCE2 (or similar) approach been adopted at the outset. But this is with the benefit of hindsight, and it must be recognised that even for the funder there was limited familiarity with this approach and hence an iterative process of learning and familiarisation.

Benefits: The MDP Case

The benefits of adopting the PRINCE2 approach in the MDP case are summarised in Table 2. Some of these are prospective, as the trial is still in progress and some of the modifications have yet to be fully implemented.

Although these benefits are qualitative (we do not attempt to put a financial measure against them), we believe the most important of these will result from more effective project management and from the enhanced relationship with the funding body rather than from quantifiable cost savings.

Nevertheless, one of the consequences of streamlining the quarterly reports to the steering committee and funder (DFID) has been that the time required to produce these reports has been reduced.

Wider Benefits

In the MDP case, a primary reason for reviewing the project office documents was the request from DFID to improve the link between expenditure and activities in high level reporting. As the modifications to the reports were explored, wider benefits to the project managers and the team as a whole from the proposed modifications became apparent.

Our view is that there are substantial potential benefits for funding bodies, project managers, and research teams as a whole if project management is recognised as an explicit cost item at the project proposal stage and project management approaches are then adopted at the outset of a project.

Table 3 suggests where the main benefits might arise. The list of benefits reflects to a large extent what has already been shown in MDP. We also add potential benefits to the research team itself.

Wider Costs

While the ratios are not rigid, a common rule of thumb in management consulting is that project management costs represent on average 3% of total fees. Typically a full-time project manager is required on projects of 3m [pounds sterling] and over per year. In construction projects a common expectation is that project management will require 1.5% - 3% of the capital costs, but much depends on size and complexity and on which functions are included in project management.

As in the MDP case, it can be expected that cost will be lower if a coherent approach is built in from the beginning (from the proposal stage).

loaded on projects. Once the PID and project documents are set up, their maintenance by team members who understand the process is relatively economical. Properly administered, they also save administrative time on other tasks.

Clearly, in assessing costs adjustments, one needs to account for: (a) Scope--what will be included in project management, e.g., which of the PRINCE2 documents and processes are considered appropriate in each case; (b) Geographical spread of sites (consultant or client) and number of different parties involved (entities/team size); (c) Number of key decision points envisaged (need to take stock of progress and adjust plan); (d) Number of different disciplines (or work-streams) involved in the project; and (e) Size and duration of the project

Conclusion

The use of PRINCE2 to help manage MDP and use it to report back to DFID was fairly novel for all concerned--scientists, clinicians, and administrators alike. Indeed, Ove Arup and Partners Ltd. was commissioned to work with the team to develop the tools and techniques needed. The approach adopted gave a transparent and robust tool for managing risks and budgets that enabled DFID to readily review a number of key performance indicators relevant to the trial and thereby obtain assurance about programme management and trial progress.

The tools and terminology were alien to the majority of those involved and therefore required overcoming a steep learning curve. It was also resource-intensive for a number of key staff as changes were made to the existing approach. Familiarising the consultants on clinical trial methodologies, on MDP itself and how the new tools should be scoped, developed, and managed required a significant amount of time from key MDP staff members. This was essential to download structure and intelligence to ensure the tools were accurate and fit for purpose. It is doubly essential, therefore, that tools and templates provide obvious efficiencies in the medium to long term.

It is important that techniques and tools of this kind are seen as adding value in terms of oversight and scrutiny rather than becoming an additional bureaucratic burden, particularly for the researchers. Whether this achievement can always be proven is a moot point. The mandates of Good Clinical Practice (GCP), FDA and other authorities require that data collection and storage, trial management, and processes per se must be of the highest quality. Project management, however, is less well defined--the majority of researchers, quite rightly, need to be convinced of the merit and benefit of incorporating these tools in order to accept the costs and resource implications of doing so. We suspect, however, that as the number, size, and complexity of research projects continue to increase, the need for formal project management will become more critical. It is likely that as research managers, we will be required to undertake the necessary training to support and work closely with our academic colleagues.

Authors' Note

We recognise the contribution and efforts of a number of key individuals whose work on MDP ensures it is such a well-managed study and who were intimately involved in the development of the tools and reports described in this paper: Dr. Sheena McCormack (MDP Chief Investigator), Professor Janet Darbyshire, Emmanuel Harding, Terry Kenvyn, Clare Rutterford (all from Medical Research Council Clinical Trials Unit, UK), Professor Jonathon Weber, Dr. Lorua Colquhoun, Kathryn Taylor, Ruth Tipples, Claire Puddephatt (all from Imperial College London, UK), Professor Helen Rees (Reproductive Health and HIV Research Unit, University of Witwatersrand, South Africa), Professor Heiner Grosskruth (MRC/Uganda Virus Research Institute, Uganda), Stephen Phillips, Andrew Butt (both from Ove Arup & Partners Ltd, UK), investigators and staff at the trial sites in Africa too numerous to mention by name. MDP is jointly funded by the UK Department for International Development and Medical Research Council, UK. David Langley was previously Director of Research Services, Imperial College London, UK.

Reference

UK Office of Government Commerce. (Third edition Crown copyright 2002, Twelfth Impression 2005). Managing successful projects with PRINCE2.

Peter Gist, Ph.D.

Ove Arup & Partners Ltd, UK

[e-mail]

Ove Arup & Partners Ltd

13 Fitzroy Street

London WIT 4BQ, UK

David Langley, Ph.D.

Director of Research and Enterprise Development,

University of Bristol, UK

Table 1
Indicative Costs to Modify Project Office Documents and
Processes on MDP

Activity                                      USD

Review of modifications to                   15,000
project office documents

     * Review current project office
       documents against PRINCE2 standard
     * Discussions with PI, MRC Clinical
       Trials Unit (CTU)
     * Note to PI, MRC recommending
       changes to documents
     * Note to steering committee and
       DFID on recommended changes

Implementation of changes                    20,000
to project office documents

     * Note on current process for
       financial reporting and
       recommendation on changes
     * Production of worked of modified
       financial example quarterly
       reports (linking expenditures to
       activities)
     * Preparation of outline overall
       project plan
     * Adjustments to MRC CTU financial
       reporting spreadsheets to automate
       production of quarterly reports
     * Modifications to clinical site
       financial reporting templates and
       automation of data transfer
       to CTU reports

Additional time for project                  40,000
office to implement changes
("one-off costs" only)

Total additional cost (USD)                  75,000

Table 2
Benefits to MDP-from Modified Project Management

 *     Improved understanding of the trial process at the
       funding body (DFID) and confidence in
       financial management

 *     Ongoing savings in time for team members through
       standardisation of site reporting and
       automatic flagging of variances (for monitoring)
       between actual and forecast/budgeted expenditures

 *     Improvements to project management effectiveness
       through explicit linking of activities and
       expenditure (objective measures aiding process
       of making future revisions to budgets)

 *     Improvements in risk management from modifications
       to the risk register and linking of the risk
       register to the high level reports

 *     Ongoing savings in team project management time
       from streamlining and automating financial
       reports

Table 3
Wider Benefits of Project Management
in Research

Funder

* Provides assurance of value
  for money through:
* Clear reports of progress and
  plans for the next period.
* Clear governance structure
  for decision making and for
  assignment of responsibilities.
* Improving confidence that
  expenditure is well managed
  (i.e. tied to activities-
  milestones and deliverables).
* Improving confidence that
  any risks to milestones or
  deliverables will be mitigated.

Researchers

* Improves research team
  chances of winning research
  funds.
* Aids knowledge transfer
  between team members
  (standard reporting and data
  accessibility).
* Reduces time and risk
  in conveying knowledge
  between team members.
* Potentially influences
  scientific outcomes by
  providing objective criteria
  for targeting efforts/avoiding
  or managing risks.

Research Managers

* Improves management of
  relationship with funders.
* Clarifies the plan, roles and
  responsibilities--reduces
  ambiguity in the project
  management task.
* Assists with budgetary
  control (e.g. requires
  academics to engage more
  closely with the link between
  activities and resources).
* Provides audit trail.