The right way: staff training and competency assessment.
Accrediting and regulatory agencies have different requirements for competency assessment. Discussed here are the requirements of the American Association of Blood Banks (AABB), the College of American Pathologists (CAP), the Clinical Laboratory Improvement Act of 1988 (CLIA '88), and the Joint Commission for Accreditation of Healthcare Organizations (JCAHO), as well as ways to meet these requirements through staff training and competency assessment programs.
In general, the effectiveness of orientation, training, and education should be evaluated and reported through a quality improvement (QI) program. Similarly, staff competency should be evaluated and reported regularly so that problems may be identified and solved in a timely way.
It seems everyone is mandating that employees demonstrate competency; however, most guidelines are not specific. So how should this be done? It's pretty much left up to you to decide what is the most efficient yet effective way to proceed.
Some employees may perceive competency assessment as a threat. It is very important to explain to employees exactly what is going to occur and why. You may wish to give employees copies of accreditation regulations, if necessary.
Employees must understand the goal of competency assessment is to improve performance, hence improve patient care. Whenever possible, corrective action should not be punitive. Education should be stressed.
Another problem may occur during actual testing. Some employees may feel someone is looking over their shoulder or checking up on them. This may be true, especially if observation is used by the supervisor to assess competency. Employees may feel their supervisor does not trust them, or worse, is hoping they'll make a mistake.
Once again, continue to emphasize the goal of the program. Inform employees in advance of what skills are to be tested. If possible, let them know when they will be tested so they will feel more prepared. Since everyone should demonstrate competence, don't forget to assess supervisors and support personnel, not just technologists.
Here's what is required by each agency:
AABB. AABB requires a quality assessment and improvement program to ensure personnel are knowledgeable and skilled in their assigned duties. The Association demands a quality management program that includes periodic evaluation and documentation of competence of personnel to perform assigned duties.
At least annually, the employees must demonstrate their abilities to carry out every test or procedure they may be called upon to perform. The AABB standards suggest use of proficiency testing, written and oral exams, and observance of daily work. Corrective action to improve substandard performance must be documented.
CAP. CAP requires a sufficient work force with adequate documented training and experience to meet the needs of the laboratory. Periodic evaluations are required.
CLIA '88. CLIA '88 requires a mechanism for periodically evaluating the effectiveness of policies and procedures to ensure employee competence (see box, "CLIA's required procedures for evaluating laboratory staff competency"). The procedures for evaluating competency must include those listed in the box. The laboratory director must employ competent personnel to perform and report tests, and the technical supervisor is responsible for evaluating competency. The technical supervisor also is responsible for conducting performance evaluations semiannually for new employees during their first year in the lab and annually thereafter. The technical supervisor may delegate these periodic evaluations to the general supervisor. Corrective action to improve performance must be documented.
JCAHO. JCAHO requires individual competency to perform tests safely and accurately and to prevent transfer of infection. The laboratory director must maintain competency of staff initially and continuously. Proficiency must be demonstrated and documented. JCAHO also requires performance evaluations and suggests appraisals of routine test performance, recording and reporting of results, quality control, proficiency testing, calibration performance, and assessment of problem-solving skills. Blind testing also is suggested by JCAHO.
There are many ways to demonstrate competence, some more complex and involved than others. Likewise, there are several methods by which competency can be assessed, including the use of blind samples, proficiency testing, written and verbal exams, direct observance, daily reviews of test results, delta checks, and use of physician alert values.
Frequency of assessment. So, how often should an assessment be done? An assessment should occur upon employment and at least annually thereafter. AABB standards require performance assessments semiannually during the first year of employment. One way to comply with this standard is to do a follow-up skills assessment, perhaps at the end of the probationary period three to six months after the date of hire.
We assess competency during orientation using a detailed checklist that we call our validation of skills checklist [ILLUSTRATION FOR FIGURE 1 OMITTED]. This list contains every procedure, test, and skill required of the employee, and is updated at least annually and whenever a new test or procedure is introduced. (Incidently, procedure manuals also should be reviewed during orientation, annually, and whenever a new test or procedure is introduced as a proactive step toward maintaining employee competency.)
We conduct performance appraisals three months after the date of hire and annually thereafter, and competency assessment is included with each performance evaluation. Supervisors use a skills competency assessment form [ILLUSTRATION FOR FIGURE 2 OMITTED] to select the most important criteria of the individual position for evaluation (e.g., customer service, specimen collection, administrative ability, problem antibody work-up).
Criteria should be reviewed and revised at least annually, and employees need to be informed of revisions to the set of skills to be assessed.
Quantifying performance and characterizing skills
The methods used to measure whether the employee meets the criteria also are listed. Tools you can use to quantify performance and characterize skills include continuing education credits; written documentation, such as copies of QA logs; observation; peer review; QA findings; a lab practical in which the employee performs tests for evaluation; and verbal or written tests.
Each criterion used to measure employee job performance usually falls into one or more of three skills categories: critical thinking, interpersonal relations, and technical. We decided these three categories were the most important ones for us. Some criteria can be used to evaluate more than one skill. For example, preparation of blood components requires both critical thinking and technical skill, and how well an employee performs this task is a reflection of the sophistication of those skills.
We score performance using a scale of 1-3, 1 being unsatisfactory performance that meets less than 80% of the criteria, 2 being standard performance that meets 80% to 90% of the criteria, and 3 being excellent performance that meets more than 90% of the assessment criteria. The lower limit of 80% for satisfactory work was chosen because CLIA uses 80% as the lower limit for satisfactory scores on proficiency tests.
There also is a column to document the date of assessment and the evaluator's name or initials. Testing may occur on the same day or may occur over a period of time, whatever works best for you and your department.
At the end of the assessment, you want to total your scores and divide by the number of criteria for an average. (If one criteria tests more than one skill, these scores should be averaged so that you will have just one score per criterion.)
What happens when an employee scores a 1 in any area or the overall score is less than 2.0? We initiate a performance improvement plan. At this point, it is necessary to involve the Human Resources Department. Employees should not perform tests or procedures for which they have been unsatisfactorily appraised until corrective action is taken and performance is improved adequately. Competence must be demonstrated before the employee is reassigned those responsibilities.
When a problem is identified, try to determine the cause. Is it a lack of knowledge? If so, educate. Is it a performance error? If so, provide training and allow practice. Is it a system problem? If so, provide supplies or change the structure or policies.
Because improvement is the intent of performance appraisals, the focus of corrective action should be on education, and punitive action should be avoided whenever possible.
Also, fix the system or process rather than place blame on the individual. Look critically at the employee, but also look critically at how the problem evolved.
Our skills competency assessment form also has space to document performance improvement plans, as well as a space for the review date, and signatures for the employee and supervisor.
If a performance improvement plan is needed, the supervisor draws one up and reviews it with the lab manager. We document the competency score in the performance appraisal, and a copy of the skill competency assessment is included with the appraisal. The lab manager signs and dates the skills competency assessment forms; then the quality compliance supervisor reviews all assessments, looks for trends, and reports findings to the QA committee. The committee may make recommendations regarding our assessment procedure or the criteria being used for assessment.
Competency testing is important and can be achieved easily with an organized, comprehensive plan. Communicating your expectations to staff members is essential for a successful competency program. Remember that the purpose of competency testing is to improve performance resulting in positive patient outcomes.
RELATED ARTICLE: CLIA's required procedures for evaluating laboratory staff competency
1. Direct observations of routine patient test performance, including patient preparation, if applicable, specimen handling, processing, and testing
2. Monitoring the recording and reporting of test results
3. Review of intermediate test results or worksheets, quality control records, proficiency testing results, and preventive maintenance records
4. Direct observation of performance of instrument maintenance and function checks
5. Assessment of test performance through testing previously analyzed specimens, internal blind testing samples or external proficiency testing samples
6. Evaluating and documenting the performance of individuals responsible for high complexity testing at least semiannually during the first year the individual test patient specimens. Thereafter, evaluations must be performed at least annually unless test methodology or instrumentation changes, in which case, prior to reporting patient test results, the individual's performance must be reevaluated to include the use of the new test methodology or instrumentation.
Source: Standard: Technical Supervisor responsibilities. Section 493.1451 Federal Register. 1992;57(40):7180-7182.
Karen A. George is laboratory administrative director for the University of Louisville Hospital in Louisville, Ky.