Training and Competency: They’re the Same Thing, Right?

As a fitting end to my previous 3-part series on how to prepare for and survive your regulatory inspections, one of the hospitals we provide consulting services to was just visited by The CAP. Overall we did great and I’m proud of everyone there, but the inspectors found a weak area for us to improve upon that others may be struggling with as well: documentation of training and competency.

It is a common misnomer that training and competency are equivalent and essentially the same thing. Whether you’re subject to CLIA, CAP or your local state DOH requirements, you will be required to provide proof (documentation) of both training AND competency for each employee, for each task that they perform. This is not just limited to your technical staff, but also includes non-technical personnel (phlebotomists, lab assistants, LIS personnel,transport couriers, etc.), as well as staff outside of the immediate laboratory testing area (respiratory clinics with blood gas analyzers, Point of Care testing, etc.).

Simply put, training is coaching, mentoring, and teaching someone step-by-step how to perform a specific task. Proper documentation of this training includes:

  • Objectives for the training (i.e., “After completing training, staff will understand howto successfully perform maintenance tasks on the hematology CBC analyzer.”)
  • Identification of the methods to be used during the training (direct observation, monitoring recording & reporting of results, review of worksheets & preventive maintenance records, evaluation of problem solving skills)
  • Identification of the materials to be used during the training (cleaning agents, QC samples, previously tested & scored proficiency testing material)
  • Criteria used to assess the effectiveness of the training (minimum score of 90% on critical thinking quiz, ±10% correlation with previously tested sample)
  • Signature of both the trainee and trainer confirming that training was completed, and when

In addition to the obvious routine tasks a lab professional will need to perform (running QC, instrument maintenance, running patients), don’t forget to document their training for the low frequency tasks performed as well. Based on an employee’s job description, they may be involved in additional tasks such as specimen handling, safety precautions, packing and shipping of samples to reference labs, computer system training, telepathology training, and supervisory functions. These tasks too will require documentation of training.

Documentation of all of these tasks can be organized through the use of a departmental orientation checklist. This will help you keep track of what each staff members’ specific job junctions will include that they need to be trained on, and which tasks have been completed by each trainer. Depending on the task, training can be completed quickly after several minutes of demonstration (waived urine hCG testing), or may take several weeks for staff to fully understand and master the task (flow cytometry leukemia work-up). Keep in mind that until a staff member has documented training followed by successful assessment of competency of that task, they should not be permitted to perform or result patient testing independently of their trainer.

Once training has been completed and documented, you must then assess each staff member’s ability to successfully perform these tasks. This is their competency, where you assess if the training was successful and staff are able to perform each assigned task correctly. To fully demonstrate successful competency of non-waived tests, all 6 of the following elements must be documented for each employee, for each task:

  1. Direct observation of patient test performance, including patient identification, specimen collection, handling, processing and testing.
  2. Monitoring the recording & reporting of test results, including when appropriate the handling of critical results.
  3. Review of testing worksheets, QC records, proficiency testing results, and preventive maintenance records.
  4. Direct observation of performance of instrument maintenance and function checks.
  5. Assessment of test performance through testing previously analyzed specimens, internal blind testing samples or external proficiency testing samples.
  6. Evaluation of problem-solving skills.
  7. Observation of compliance with safety protocols (based upon your specific local state DOH regulations).

The documentation of your competency elements should include the date each item was evaluated, as well as a way to identify and recreate the test performance if asked by an inspector. This is most easily accomplished with the specimen ID number, or PT survey name so records can be located or reprinted.

Be mindful of your local state regulations regarding the specific requirements for who can perform a competency assessment. In many cases, assessors will need an additional supervisor competency for themselves to confirm they are able to successfully assess the performance of their peers. If weaknesses are identified during the competency assessment, additional training should be performed with appropriate corrective actions documented. Competency should be reassessed to ensure staff are correctly performing all duties, prior to them resuming patient testing.

So to summarize:

During training, I am showing you how to do something. I will document all aspects of the training steps that I reviewed with you. When I assess your competency, you are showing me that you know how to do the task correctly. You will document your results as you were trained how to do, and I will validate the accuracy of your work.

-Kyle Nevins, MS, MLS(ASCP)CM is one of ASCP’s 2018 Top 5 in the 40 Under Forty recognition program. She has worked in the medical laboratory profession for over 18 years. In her current position, she transitions between performing laboratory audits across the entire Northwell Health System on Long Island, NY, consulting for at-risk laboratories outside of Northwell Health, bringing laboratories up to regulatory standards, and acting as supervisor and mentor in labs with management gaps.

What’s Your Number?

When I ask “what is your number,” I mean “what is the percentage of time you spend on the bench versus time in the office doing administrative work.” In my laboratory we currently have a chemistry supervisor on extended leave so myself and the hematology supervisor are 90/10 bench/office. I know what you’re thinking, how do you do the administrative portion while working the bench that much? The answer is simply, we can’t. Unless I work 60 hour work weeks there is no humanly possible way to be able to work on the bench and get done what I need to in the office. Prioritizing the administrative duties and taking advantage of slow days on the bench is the number one reason why I am somewhat successful at getting things done.

The biggest challenge I face is competency. One of CLIA’s six competency guidelines is physically watching each technologist/technician perform each test that is on our test menu. So let me break this down. I have 15 employees that need to complete competencies. We perform gel testing in blood bank as well as LISS, PEG, and Pre-warm screens in tube if necessary. Let’s round all of those tests to 45 minutes. 15 employees times 4 different methods is 60, times the 45 minutes is 2700 minutes, or 45 hours. Now don’t forget that I am in the hospital setting which is 24/7 so I have to be available for second and third shift if I want to make sure those employees are competent. So for simplicity sake I’m going to round those hours up to 48 which makes six 8 hour days of competency. If I get two office days a month, competency will take me four months to accomplish and that isn’t counting my other duties. This is also assumes uninterrupted time which we all know happens once a millennia. These are just four tests as well; we perform many others such as antigen testing, long crossmatching, etc. When looking at this it can be overwhelming but there are ways to accomplish the impossible without killing yourself.

One way to help yourself out is to look at the CLIA definitions of who can be a technical supervisor for competency testing. Chances are you have more than one employee who qualifies and as long as you sign them off as competent they can sign off other employees. This can drastically cut down the amount of time you need to spend individually with each employee. Another way is to have the employee’s videotape themselves performing the tests while explaining out loud what they are doing. This not only confirms their competency but you can watch back the tape at your leisure as well as show inspectors if they ask how you satisfy the observation portion of the competency. I now ask you in the comment section below write down your numbers and let me know what your biggest challenges are and some possible solutions. We can help each other out, and share in the continuous struggle. So, what’s your number?

 

Herasuta

Matthew Herasuta, MBA, MLS(ASCP)CM is a medical laboratory scientist who works as a generalist and serves as the Blood Bank and General Supervisor for the regional Euclid Hospital in Cleveland, OH.

Point of Care Testing Competency Assessment

Healthcare facilities can use point of care testing (POCT) devices to improve healthcare delivery. The majority of these POCT devices are waived tests, such as glucometers and hemoglobinmeters. In recent years, however, more facilities have moved toward moderate complexity testing such as troponin cardiac biomarker, blood gasses, prothrombin time, and bilirubin. Most POCT devices are operated by non-laboratory clinical personnel, such as nurses and respiratory therapists. Under CLIA regulations, these individuals are classified as testing personnel (an individual who perform laboratory testing and report test results for patient care) and must comply with competency assessment regulations.

CLIA does not have any requirement for assessing personnel competency on waived testing; however, CLIA requires that waived testing results are accurate to assist in making accurate patient diagnosis. This requires that personnel follow all manufacturer’s instructions, which generally includes personnel training, skill assessment for QC and patient testing, device maintenance and infection control procedures as the device is moved from patient to patient. When waived tests are included in the main laboratory’s CLIA license, most accrediting agencies (CAP, Joint Commission, COLA, HFAP, etc.) will also require fully established and documented personnel competency testing on all personnel who performed waived tests. It’s not uncommon for a healthcare facility to receive findings of non-compliances on POCT personnel competency assessment due to incomplete QC (both internal and external) skill assessments for each personnel. Some healthcare facilities erroneously have the laboratory POCT coordinator or supervisor perform the external QC on their POCT. All testing personnel must know how to perform all QC and their corrective action for out of control situations and documentation requirement pursuant to the organization policy.

Reference: CLIA Brochure #10.

 

Information on policies or practices are solely from my personal experience ONLY and have NO relation to my affiliation with any regulatory or government agency.

satyadi

-Caroline Satyadi, MT(ASCP), SM, DLM, SLS, MBA, MS, CQA (ASQ) has been a laboratory management professional for over 25 years. She has worked with several different medical industries for CLIA/CMS, FDA/ICH/ISO, TJC/CAP/COLA/HFAP accreditation survey readiness.

Personnel Competency Assessment

On December 21, 2012, the Center for Clinical Standard and Quality/Survey & Certification group (part of the United States Centers for Medicare and Medicaid Services) published S&C-13-07-CLIA. This letter outlines competency assessment requirements for compliance with federal regulation 42 CFR, Part 493, Subpart M. The Clinical Laboratory Improvement Amendments (CLIA) law for personnel competency applies to testing personnel, clinical and technical consultants, and technical and general supervisors. Competency assessments can only be performed by personnel qualified as technical consultants, technical supervisors, or general supervisors. The competency assessment must cover a minimum of six required procedures in addition to their federally regulated responsibility:

  1. Direct observations of testing procedure, including specimen preparation, and handling/ pre-analytical as appropriate,
  2. Monitor of recording and reporting of results,
  3. Review of intermediate test results or worksheets, QC results, preventive maintenance and proficiency testing results,
  4. Direct observation of actual preventive maintenance or function checks procedures,
  5. Assessment of test performance on external proficiency testing samples, internal blind unknowns or previously analyzed samples,
  6. Assessment of problem solving skills (42 CFR 493.1413 (b)(8) and 493.1451(b)(8)).

It’s important to note that peer reviews among testing personnel are NOT acceptable for regulatory personnel competency assessment.

An update to this policy involves the laboratory director’s competency assessment requirement. When the director is the only individual testing and reporting test results (such as a pathologist in a small hospital), the organization must establish and document a minimum level of competency via external proficiency testing or peer reviews for the tests reported on patients. This competency must be performed a minimum of three events throughout the year.

The competency assessment procedure is further differentiated by the regulations for moderate and high complexity testing. For moderate complexity testing, the technical consultant can perform the competency assessment, whereas in high complexity this can be performed by the technical supervisor, which can be further delegated to a general supervisor as long as the general supervisor is qualified for high complexity testing. All delegations must be delineated appropriately by the laboratory director as a written policy. Testing personnel requirements might additionally vary from state to state because some states (CA, FL, HI, LA, MT, NV, ND, RI, TN, WV, and Puerto Rico [PR]) require licensures for their laboratory practitioners.

For new employees, competency assessments must be performed six months and one year after employment; this assessment happens annually thereafter. Whenever testing errors serious adverse events occur, management should conduct personnel retraining and requalification to establish personnel competency. Also, establish limits on how many times such events can occur before prompting a transfer for other work duties or termination.

How can you measure the effectiveness of your organization’s personnel competency assessment? Since personnel competency encompasses multi factors, developing a balance score card (BSC) for each department or specialty can be useful. Percentages can be assigned for external testing (such as CAP proficiency tests), attending internal education sessions, number of serious adverse events or errors, and accrediting agency findings on on-site personnel technical competency during licensing surveys. This BSC can aid management in producing score of effectiveness on each individual and department.

Reference: CLIA brochure #10.

 

Information on policies or practices are solely from my personal experience ONLY and have NO relation to my affiliation with any regulatory or government agency.

satyadi

-Caroline Satyadi, MT(ASCP), SM, DLM, SLS, MBA, MS, CQA (ASQ) has been a laboratory management professional for over 25 years. She has worked with several different medical industries for CLIA/CMS, FDA/ICH/ISO, TJC/CAP/COLA/HFAP accreditation survey readiness.

Does Your Laboratory Personnel Meet the CLIA Regulation and Accreditation Requirements?

On December 21, 2012, the Centers for Medicare and Medicaid Services (CMS) published Survey & Certification letter: (S&C-13-07-CLIA) with Brochure #10 on Personnel Competency. It cites the six minimal regulatory requirements for assessment of competency for testing personnel:

  1. Direct Observation of testing procedure
  2. Monitoring the recording and reporting of test results
  3. Review of test results
  4. Performance of instrument Preventive Maintenance
  5. Test performances, i.e. blind unknowns, etc.
  6. Assessment of problem solving skills.

Since early 2013, some CMS surveyors have joined accrediting agencies such as The Joint Commission (TJC) and College of American Pathologists (CAP) in their bi-annual lab accrediting surveys to check testing personnel’s qualification. This qualification includes proof of a minimum education requirement (usually a college diploma). Healthcare organizations need to assure the qualifications for testing personnel are fully met pursuant to the federal, state, and accrediting agency requirements. This includes personnel that perform point-of-care testing (POCT). Because this testing might be performed by non-laboratory staff, these personnel might not have the required education requirements for moderate complexity testing pursuant to the CLIA requirements.

In such cases where the testing personnel licensure is required (for example, by the state) copies of staff diplomas would not be necessary. Licensure is different than certification, which is sponsored by a private sector, nongovernmental institution, such as American Society of Clinical Pathology (ASCP). By requiring personnel licensure, the state government has taken additional measures to protect harm to the public in the state. Currently, there are 12 states with laboratory personnel licensure requirements (California, Florida, New York, North Dakota, Rhodes Island, Tennessee, Louisiana, Nevada, West Virginia, Montana and Georgia). The components of the laws for personnel licensure vary from one state to another; it usually involves a fee, a continuing education provision, and a minimum education requirement, and a professional competency requirement.

Due to the economic situation in the United States and the laboratory workforce shortage, some state government agencies have not actively enforced all compliance aspects among healthcare organizations. For example, some laboratories allow medical lab technicians (MLTs) to perform high-complexity testing due to the medical lab technologists’ (MTs) shortages across the country. However, according to CLIA regulations, MLTs should only work in moderate complexity testing areas that require limited independent judgments. The high complexity testing environments that include blood banking, microbiology identification procedures, microscopy, etc. require a high level of independent judgment and should only be performed by MTs.

Technical supervisory positions in the clinical laboratory require a minimum of five years technical experience. It would not be in compliance with the regulations when the clinical laboratory general supervisors are chosen from MLTs or related field, such as cytology or histology or radiology. Healthcare organizations need to assess their lab personnel qualification and competency to assure that they are meeting the standard requirements pursuant to CLIA regulations, state regulations, and accreditation requirements.

If you’d like more information about these regulations, please read the Code of Federal Regulations part 493.

 

Information on policies or practices are solely from my personal experience ONLY and have NO relation to my affiliation with any regulatory or government agency.

satyadi

-Caroline Satyadi, MT(ASCP), SM, DLM, SLS, MBA, MS, CQA (ASQ) has been a laboratory management professional for over 25 years. She has worked with several different medical industries for CLIA/CMS, FDA/ICH/ISO, TJC/CAP/COLA/HFAP accreditation survey readiness.

Musings After the RISE

So how did you fare on your recent resident in-service exam, aka the RISE? For most of the residents I know, they did better on the AP portion over the CP portion. I would have to say that not surprisingly, I feel that I did the opposite. Last year, I definitely did much better on the CP portion than the AP portion but my overall percentile was still good.

Even though I usually narrowed down the answers on the AP section to the correct one and a distractor, when I looked up content after I got home, I discovered that I often picked the wrong answer. But even then, I feel that the AP section was fair and not overwhelmingly difficult for someone who is probably better at AP than me.

For me, I thought that the CP section was not that difficult but most other residents that I talked with thought the opposite. They felt that many of the questions were esoteric and possibly not relevant to the practice of pathology once we are out of residency.

What are your thoughts after taking the RISE? Did you feel that it was a fair test? Did you feel that the questions asked are relevant to what we need to learn in residency and for our practice as real-world pathologists?

In other specialties like surgery and anesthesiology, in-service exams have a greater importance and scores are often asked for on fellowship applications. For pathology, this is not the case but it still is important that we test ourselves yearly to pinpoint our strengths and weaknesses in some manner. Do you think that the RISE is the answer or does it need a revamp?

 

Chung

Betty Chung, DO, MPH, MA is a second year resident physician at the University of Illinois Hospital and Health Sciences System in Chicago, IL.

The Utility of the RISE…Or Not…

So, that time of year is now again upon us…the Resident In-Service Exam, aka the RISE. Even though this test is meant to assess our knowledge of what we’ve learned thus far, the competitive natures that have brought to this point in our careers push us to do well on it. We feel an impulse, regardless of whether we act on it or not, to cram some knowledge into our brains at the last minute for a test that we are told we are not supposed to study for. This is quite a paradoxical conundrum.

But just how much does it really test? We are graded on percentiles in comparison to others in our year taking the RISE. But there are those rumors we all have heard, of programs that have remembrance databases or have year-long RISE specific lectures or students taking the test at home unproctored…so how much of our scores are true measures versus our peers if we don’t have access to these things?

Additionally, there is the other question: just how relevant is the RISE? Rinder et al. published the article “Senior RISE Scores Correlate with Outcomes of the American Board of Pathology Certifying Exams” in 2011, but how much of their findings truly are tied to our RISE scores and how much to our inherent study habits and test taking skills? These questions may be even more apropos at this time when there is a stronger recognition that we need to develop curricula to teach true competency and not just the ability to pass standardized tests. So yet again we are confronted with this question of just what does true competency mean and how do obtain it?

For me, the competency that I want to gain means possessing the ability and confidence to practice with very little supervision the day after I finish my studies and get a job. For some, this may be directly after residency and for the majority of us, after fellowship(s). So do you believe that the RISE helps us to pinpoint our weaknesses or really doesn’t help us much on our journey to competency? Does it help predict whether we will pass the AP and/or CP boards or is just a meter of our test taking ability? As a second year resident at a program where we do not, to my knowledge, have any of those aforementioned aids, I’m not so sure that I can answer these questions. All I know is that I’m still not done taking day-long standardized tests.

So, do you feel that the RISE is useful? Why or why not?

 

Chung

Betty Chung, DO, MPH, MA is a second year resident physician at the University of Illinois Hospital and Health Sciences System in Chicago, IL.