Doctor of Nursing Practice program evaluation and beyond: capturing the profession's transition to the DNP.
Deoxyribonucleoprotein; a complex of DNA and protein that usually yields DNA upon cell disruption and isolation.
2,4-dinitrophenol. ) educational preparation for advanced practice nursing and DNP role development are innovations that require intense scrutiny through formative and summative Adj. 1. summative - of or relating to a summation or produced by summation
additive - characterized or produced by addition; "an additive process" evaluation. It is imperative to move beyond standard curriculum evaluation approaches Evaluation approaches are conceptually distinct ways of thinking about, designing and conducting evaluation efforts. Many of the evaluation approaches in use today make truly unique contributions to solving important problems, while others refine existing approaches in some way. to capture the transformation of advanced nursing practice and the profession. The University of Washington School of Nursing designed a comprehensive DNP evaluation plan that focuses on both program effectiveness and the experience of transition as described by faculty and students. The evaluation plan is outlined, and lessons learned regarding data collection, student and faculty participation, and institutional review board approval are discussed. Examples of instruments are included.
MORE THAN 100 SCHOOLS OF NURSING HAVE IMPLEMENTED DOCTOR OF NURSING PRACTICE (DNP) PROGRAMS, AND AN ADDITIONAL 50 PROGRAMS ARE CONSIDERING STARTING A DNP PROGRAM (AMERICAN ASSOCIATION OF COLLEGES OF NURSING [AACN], 2009). This transition in the education of advanced practice nurses (APNs) is targeted to meet the increasingly complex needs of patients, families, and communities in a rapidly changing health care environment. DNP education also has the potential to transform the nursing profession in a variety of ways. These include:
* Creating and adopting new roles in nursing practice
* Increasing the influence of APNs in health care and policy development
* Promoting leadership by APNs in their workplace and health care organizations
* Enhancing the self-concept of advanced practice nurses
* Strengthening interprofessional relationships and collaborations.
DNP education and role development are innovations that require intense scrutiny through formative and summative evaluation. It is imperative, however, to move beyond standard curriculum evaluation approaches to capture the transformation of advanced nursing practice and the profession. The University of Washington (UW) School of Nursing designed a comprehensive DNP evaluation plan that focuses on both program effectiveness and the experience of transition as described by faculty and students. THIS ARTICLE DESCRIBES BOTH ASPECTS OF THE PLAN.
The Program Evaluation Program evaluation is a formalized approach to studying and assessing projects, policies and program and determining if they 'work'. Program evaluation is used in government and the private sector and it's taught in numerous universities. Planning for the UW DNP began in December 2003, with final approval obtained in June 2006. The first cohort of 16 postmaster's students matriculated in January 2007 for the winter quarter. The second cohort of 20 postmaster's and I5 postbaccalaureate students began in the summer and fall quarters of 2007. Subsequent cohorts were admitted in 2008 and 2009.
The curriculum consists of a minimum of 90 quarter-credits distributed in three curricular areas: practice inquiry, leadership, and advanced practice. Program development and implementation continue with the evolution of the program (Brown et al., 2006; Draye, Acker, & Zimmer, 2006; Magyary, Whitney, & Brown, 2006).
Program evaluation is based on outcome objectives, which are generally based on the Essentials of Doctoral Education for Advanced Nursing Practice, published by the AACN in 2006. Development of the UW program objectives preceded publication of the AACN Essentials. Once the Essentials were available, minor revisions were made to the outcome objectives.
Development of the UW evaluation plan addressed the goal of conducting a standard academic program evaluation as well as capturing the phenomenon of transition of the profession. The standard academic aspects monitor student progression and the program's quality, effectiveness, and alignment with AACN Essentials. The transition aspect explores student and faculty experiences as the DNP role develops.
Components of the standard academic evaluation include: student progression benchmarks; faculty evaluation of student performance in the didactic di·dac·tic
Of or relating to medical teaching by lectures or textbooks as distinguished from clinical demonstration with patients. and clinical areas; preceptor evaluation of students; and student evaluation of courses. Other aspects of program evaluation include: evaluation of DNP course syllabi syl·la·bi
A plural of syllabus. for congruence con·gru·ence
a. Agreement, harmony, conformity, or correspondence.
b. An instance of this: "What an extraordinary congruence of genius and era" with the AACN Essentials; evaluation of DNP courses for inclusion of content about cultural competence cultural competence Social medicine The ability to understand, appreciate, and interact with persons from cultures and/or belief systems other than one's own ; and community collaboration for DNP clinical practica and capstone clinical investigative projects. End-of-program and postprogram assessments with graduates and their employers are also incorporated.
Bridges (2001) articulates the transition process as "the process of letting go of the way things used to be and then taking hold of the way they subsequently become" (p. 2). This evaluation plan capitalizes on the unique opportunity to conduct a prospective longitudinal study longitudinal study
a chronological study in epidemiology which attempts to establish a relationship between an antecedent cause and a subsequent effect. See also cohort study. of one school's development and implementation of the DNP role. This prospective approach is likely to provide a more accurate and valid representation of the transition phenomenon than the commonly used retrospective approach.
The first aspect of this part of the evaluation seeks to understand student and graduate perceptions of the effect of the DNP on their APN APN
advanced practice nurse role and practice. The second aspect explores faculty support for the DNP program and their perception of the DNP role and practice. The third aspect analyzes both faculty and student/graduate perceptions of the effect of the DNP on nursing education, the nursing profession, and the improvement of health care outcomes.
Data collection utilizes various approaches, including surveys, interviews, and review of student files and academic documents, to allow for triangulation triangulation: see geodesy.
The use of two known coordinates to determine the location of a third. Used by ship captains for centuries to navigate on the high seas, triangulation is employed in GPS receivers to pinpoint their current location on earth. of the data. Use of multiple methods yields both quantitative and qualitative data that enhance an understanding of the phenomena evaluated (Burns & Grove, 2005).
PROGRAM BENCHMARKS Most programs collect benchmark data such as grade point average (GPA GPA
grade point average
Noun 1. GPA - a measure of a student's academic achievement at a college or university; calculated by dividing the total number of grade points received by the total number attempted ), retention rates, and graduation rates. Successful completion of a general examination; a Tripartite TRIPARTITE. Consisting of three parts, as a deed tripartite, between A of the first part, B of the second part, and C of the third part. Immersion immersion /im·mer·sion/ (i-mer´zhun)
1. the plunging of a body into a liquid.
2. the use of the microscope with the object and object glass both covered with a liquid. Practicum practicum (prak´tikm),
n See internship. that synthesizes leadership, practice inquiry, and advanced practice; a practice inquiry capstone project; and a final examination are benchmarks specific to the DNP.
STUDENT PERSPECTIVES Student feedback provides a vital component of any educational assessment. Aspects of student input include: assessment of satisfaction with the program; course evaluation A course evaluation is a paper or electronic questionnaire, which requires a written or selected response answer to a series of questions in order to evaluate the instruction of a given course. ; evaluation of preceptors; self-evaluation of the capstone clinical investigative project and the Tripartite Immersion Practicum; and satisfaction with post-DNP employment. Self-evaluation plays a stronger role in the DNP program than in the master's program as DNP students' self-reflection skills are actively cultivated as part of the leadership component of the program.
FACULTY AND PRECEPTOR PERSPECTIVES As with all advanced practice programs, faculty and preceptors assess the academic achievement and practice competence of students. UW DNP students select a supervisory committee that guides the development of their program of study. The chairperson chairperson Chairman The head of an academic department. See 'Chair.', Cf Chief. assumes the major responsibility for implementation and evaluation of the student's capstone clinical investigative project. Faculty also assess student learning designed specifically for the DNP program, such as the Tripartite Immersion Practicum. Preceptors and agency personnel are responsible for providing feedback to students and faculty about student competence.
CURRICULUM EVALUATION Eight elements are used in determining the quality and effectiveness of the DNP curriculum:
1. Student satisfaction with the DNP courses (entered verbatim ver·ba·tim
Using exactly the same words; corresponding word for word: a verbatim report of the conversation.
adv. from course evaluations)
2. Successful completion of Capstone Clinical Investigative Project
3. Successful completion of Tripartite Immersion Practicum
4. Successful completion of the Synthesis of Advanced Practice Nursing course
5. Successful completion of Advanced Practice Precertification Clinical course work
6. Preceptor evaluation of the clinical performance of DNP students
7. Faculty evaluation of clinical performance of DNP students
8. Student satisfaction with preceptors Congruence of curriculum with the DNP Essentials is analyzed through review of syllabi and each student's program of study (see Figure 1).
Figure 1. Doctor of Nursing Practice Course Syllabus Evaluation Tool Using AACN Essentials Course # Course Title Number of Credits Type of Course (check all that apply)  Practice Inquiry  Leadership  Advanced Practice Review the course syllabus to determine which of the eight essentials are included in the course objectives and content outline. Note below, under each essential, the number of the course objective and/or the item in the content outline that reflects the following essentials. A single course may not address all eight essentials. 1. Scientific underpinnings for practice 2. Organizational and systems leadership for quality improvement and systems thinking 3. Clinical scholarship and analytical methods for evidence-based practice 4. Information systems/technology and patient care technology for the improvement and transformation of health care 5. Health care policy for advocacy in health care 6. Interprofessional collaboration for improving patient and population health outcomes 7. Clinical prevention and population health for improving the nation's health 8. Advanced nursing practice
Creating opportunities for students to conduct the capstone and advanced practice synthesis experience facilitates program development. Review of these experiences provides an opportunity for faculty to create norms and expectations. The review also includes evidence documenting the influence of capstone clinical investigative projects on health care outcomes, policy, and practice.
COLLABORATION Interprofessional collaboration is a core element of the AACN Essentials. Review of each student's individualized in·di·vid·u·al·ize
tr.v. in·di·vid·u·al·ized, in·di·vid·u·al·iz·ing, in·di·vid·u·al·iz·es
1. To give individuality to.
2. To consider or treat individually; particularize.
3. program of study provides evidence of the frequency of enrollment in courses offered by other health professions and departments. Review of the practice inquiry and advanced practice synthesis experiences provides evidence of the extent to which students impact the health care system through interprofessional collaboration. In addition, collaboration with community partners is important to ensure stakeholder stakeholder n. a person having in his/her possession (holding) money or property in which he/she has no interest, right or title, awaiting the outcome of a dispute between two or more claimants to the money or property. involvement in the DNP program. Data regarding the number and types of community collaborations is obtained from the contract database, reports of the School of Nursing Advisory Board meetings, and student practice inquiry proposals.
END-OF-PROGRAM AND POSTPROGRAM EVALUATION Interviews with faculty and students and an employer survey contribute data to the standard academic end-of-program and postprogram evaluations. Postprogram evaluations are conducted one, three, and five years after graduation. Student interviews include areas such as the extent to which the DNP program met expectations, enhanced leadership and professional involvement, developed practice inquiry skills, and improved practice expertise. To obtain faculty perceptions of the competencies of DNP graduates, interviews with faculty address areas similar to those discussed with students.
Employers of graduates of the UW DNP program are asked to participate in evaluating the program's quality and effectiveness by completing a self-administered questionnaire. Questions address areas such as: the type of positions in which the DNP graduate is employed; the ways that the DNP graduate improves health care quality and reduces health disparities
Health disparities (also called health inequalities in some countries) refer to gaps in the quality of health and health care across racial, ethnic, and socioeconomic groups. ; and the level of cultural competence of the DNP graduate.
Capturing the Profession's Transition Both qualitative and quantitative methods are used to explore the nursing profession's transition to DNP education as the process unfolds. Data collection began prior to student matriculation ma·tric·u·late
tr. & intr.v. ma·tric·u·lat·ed, ma·tric·u·lat·ing, ma·tric·u·lates
To admit or be admitted into a group, especially a college or university.
n. and is an ongoing component of the program.
Students in the first cohort and selected faculty were interviewed at the initiation of the program. (See Figure 2.) The interviews provided baseline information about the students' professional backgrounds and activities as well as their goals and vision for the DNP role. To document students' professional transformation, additional interviews are conducted throughout their program and following graduation. Faculty interviews focused on areas such as the level of support for the program, working with colleagues on the DNP, and expectations about the influence of the DNP on nursing education and the profession. Subsequent program evaluation with faculty explored changes over time.
Figure 2. Interview Guides FACULTY * Please describe your involvement in the development and implementation of the DNP program. * How have your views of the DNP changed over time? What was your initial image of the DNP? * How does that compare to your current understanding of it? * What challenges have you experienced and/or do you anticipate in the implementation of the DNP program? * What has been your experience working with your colleagues on the DNP? Have your colleagues been supportive, resistant, ambivalent? * How well designed are the curricular plans to meet the DNP program outcomes? * Please address the three curricular threads of advanced practice, practice inquiry, and leadership. * How do you feel the DNP will influence nursing education in general? * How do you feel the DNP will influence the nursing profession in general? * At this time, what are your current concerns about the DNP program? * Please share with us anything else you want to say about the DNP. STUDENTS, PREPROGRAM FOR POSTMASTER'S STUDENTS * Please describe your nursing background such as the type of nursing roles you have had, areas of specialization. * What activities have you been involved with professionally? * Please discuss the extent to which you have been involved in research activities. * Describe your experiences collaborating with other advanced practice nurses. * How did you learn about the development of the Doctor of Nursing Practice degree? What do you see as the benefits of having a professional doctorate? * What are your goals for obtaining a DNP? What made you decide you needed the DNP to attain these goals? * Are there any supports and/or incentives that are making it possible for you to enroll in the DNP program? * Describe your experience and skill level working with computers and information technology. * How do you envision graduates of the DNP program will improve health care quality and contribute to reducing health disparities? * How are people (professionally and personally) responding to your decision to obtain a DNP? * Are there any questions you have about the DNP program? * Is there anything else you would like to tell us about your decision to obtain a DNP?
All UW School of Nursing faculty were invited to participate in an online survey. They were asked about their level of support for the DNP as entry into advanced practice by 2015 and whether they supported the UW implementation of the DNP. The survey also contained questions about the expected competencies of DNP program graduates in comparison with master's program graduates. Another component of capturing the profession's transition will be data from a survey of employers to determine the contributions of DNP graduates.
Lessons Learned The development and initial implementation of the evaluation plan yielded some important lessons. One lesson concerned Institutional Review Board (IRB IRB
See: Industrial Revenue Bond ) approval. Although not typically required for standard academic evaluations, IRB approval was required because of plans for data dissemination dissemination Medtalk The spread of a pernicious process–eg, CA, acute infection Oncology Metastasis, see there . Conducting a longitudinal study with different groups and multiple instruments contributed to the complexity and difficulty of the approval process.
Other complicating com·pli·cate
tr. & intr.v. com·pli·cat·ed, com·pli·cat·ing, com·pli·cates
1. To make or become complex or perplexing.
2. To twist or become twisted together.
1. factors involved data collection and student and faculty participation. The evaluation plan began with admission of the first cohort of students and is ongoing. There are numerous data collection points, each involving one or more different instruments and three different groups: students, faculty, and employers of the DNP graduates.
Instruments were created for each group and sampling point. A total of seven tools are used for extracting data from student records, syllabi, and program contracts for practice and research. In addition, there are five interview guides for students, one interview guide and one questionnaire for faculty, and one questionnaire for employers.
The evaluation plan is a matrix encompassing 15 measures, multiple groups, and multiple time points for data collection, making it a challenge to clearly delineate the relationships among the metrics metrics Managed care A popular term for standards by which the quality of a product, service, or outcome of a particular form of Pt management is evaluated. See TQM. . A recommended strategy to clarify this complex process is to create a visual representation of the matrix using figures, flow charts, and/or tables. In addition, developing a process of informed consent acceptable to the IRB was particularly challenging. The solution involved additional steps and exchanges between study personnel and potential respondents to establish a consistent and standardized approach.
Ensuring the integrity and validity of the data collection was another key focus. Having only an internal evaluator who is a faculty colleague and who teaches DNP courses could lead to concern about a power differential, diminish participation, and inhibit full expression by participants. To avoid these problems, both an external and an internal evaluator were used, each with different responsibilities. One of the external evaluator's responsibilities is to conduct faculty and student interviews without creating a sense of coercion coercion, in law, the unlawful act of compelling a person to do, or to abstain from doing, something by depriving him of the exercise of his free will, particularly by use or threat of physical or moral force. to participate. Participants may be more likely to be honest and open during the interview process with someone who is not a faculty colleague or course instructor. An external evaluator strengthens confidentiality and diminishes the sense of vulnerability should a participant wish to make negative comments.
Strategies for optimizing the participation of students and faculty focused on the interview and survey approaches. Telephone interviews were particularly convenient for both the participants and the evaluator; they allowed for flexible scheduling, without having to travel to an interview location, and reduced costs. It is also possible that, for some people, a telephone interview increases comfort and facilitates disclosure.
Initially, an outside service was used to conduct the online faculty survey rather than relying on university-based technology. The intent was to increase the faculty's confidence that participation would be anonymous, particularly because of controversy surrounding implementation of the DNP at the UW School of Nursing. Other benefits of this approach were the ease of setup, self-administration, and tabulation of the results. A second online faculty survey conducted two years later used university-based technology.
Implications for Nursing Education and Practice An educational innovation requires intense scrutiny to ensure quality and effectiveness. A consistent and standardized approach to evaluation will facilitate the validity, reliability, and generalizability of data, which can be used to demonstrate the quality and effectiveness of DNP programs. This information will be very useful in working with numerous stakeholders Stakeholders
All parties that have an interest, financial or otherwise, in a firm-stockholders, creditors, bondholders, employees, customers, management, the community, and the government. who include:
* Schools of nursing that have not adopted the DNP
* University and college administrators and faculty colleagues
* Health professionals with whom DNP graduates collaborate
* Regulators such as Boards of Nursing and Boards of Higher Education
* Potential employers of DNP graduates
* Health care consumers
* Health plans and insurers
In conclusion, the DNP stimulates the creation and adoption of new advanced practice roles. Thus, it is crucial to capture this major transition in nursing education and the profession. Bridges (2001) describes transition "as the way we come to terms with change" (p. 3). His three-stage process of transition involves an ending, a neutral zone, and beginning again. Implementation of DNP programs has set into motion Stage 1, the ending that focuses on the loss of old values, self-image, perspectives, and realities. This stage is characterized by the challenge of acknowledging the reality that things are no longer as they were. Stage 2, the neutral zone, is the state of being in limbo limbo
In Roman Catholicism, a region between heaven and hell, the dwelling place of souls not condemned to punishment but deprived of the joy of existence with God in heaven. The concept probably developed in the Middle Ages. , when things are neither the old nor the new. Stage 3 is a beginning that embraces a new reality, outlook, or self-image.
While a growing number of schools have initiated DNP programs, only a handful have postbaccalaureate entry into DNP education. The profession is on the cusp of Stage 1. As more programs are planned and implemented, the status qua has been disrupted. Some may see this as an exciting opportunity. Others may be experiencing disequilibrium disequilibrium /dis·equi·lib·ri·um/ (dis-e?kwi-lib´re-um) dysequilibrium.
linkage disequilibrium or loss. The University of Washington evaluation plan allows us to detail the profession's transition through each of these three stages.
Key Words Doctor of Nursing Practice--Practice Doctorate--Program Evaluation--Advanced Practice Nursing--Nursing Education
American Association of Colleges of Nursing. (2006). The essentials of doctoral education for advanced nursing practice. Retrieved from www.aacn.nche.edu/DNP/pdf/Essentials.pdf
American Association of Colleges of Nursing. (2009). Doctor of Nursing Practice Programs. Retrieved September 2, 2009, from www.aacn.nche.edu/DNP/DNPProgramList.htm
Bridges, W. (2001). The way of transition: Embracing life's most difficult moments. Cambridge, MA: Perseus Books Group.
Brown, M.A., Draye, M.A., Zimmer, RA., Magyary, D., Woods, S. L., Whitney J. D., et al. (2006). Developing a practice doctorate in nursing: University of Washington perspectives and experience. Nursing Outlook, 54(3), 130-138.
Burns, N., & Grove, S.K. (2005). The practice of nursing research: Conduct, critique and utilization. St. Louis: Elsevier.
Draye, M.A., Acker, M., & Zimmer, P.A. (2006). The practice doctorate in nursing: Approaches to transform nurse practitioner nurse practitioner
n. Abbr. NP
A registered nurse with special training for providing primary health care, including many tasks customarily performed by a physician. education and practice. Nursing Outlook 54(3), 123-129.
Magyary, D., Whitney J. D., & Brown, M.A. (2006). Advancing practice inquiry: Research foundations of the practice doctorate in nursing. Nursing Outlook, 54(3), 139-151.
Louise Kaplan, PhD, ARNP ARNP Advanced Registered Nurse Practitioner , is an associate professor at Washington State University, College of Nursing, Vancouver, and Marie-Annette Brown, PhD, ARNP, FAAN FAAN
Fellow of the American Academy of Nursing , is a professor at the University of Washington. Contact Dr. Kaplan for more information at firstname.lastname@example.org.