Proficiency Testing (PT) Part 2: Investigating Failures

Last month we discussed the rules and requirements for how to properly perform proficiency testing (PT) within your laboratory. In part 2 of this 3-part series we’ll review the rules associated with evaluating your results, and how to investigate any unsuccessful surveys. Still to come in part 3 we will look into how to utilize your PT results to monitor for trends and shifts in your values.

The rules:

• Performance Review: Laboratories must initiate and document a review of their PT performance evaluations within 2 weeks of notification that results are available. This includes a review of both graded and non-graded/educational analytes and events as well.

Key things to note: Even though educational samples are not formally graded, you should still verify the accuracy of your results, with appropriate follow-up for any failures. CAP specifically requires you to evaluate these educational challenges as well. Whether the sample is graded or not does not change the fact that you had an incorrect result.

• Unsatisfactory Performance: For any unsatisfactory results, you are required to perform a root cause analysis to determine why (see below for guidance). This also includes any clerical errors – you need to evaluate your process and find ways to prevent these simple errors from happening again. If they are happening with PT samples, it is possible they are happening with patient samples as well.

• Cessation of Patient Testing: Unsatisfactory events indicate that there was a problem with that particular survey; whereas unsuccessful events indicate there has been a pattern of unsatisfactory events/samples and a larger problem exists. If a pattern of poor performance is detected, you may be asked by your local state department of health to cease all testing for a particular analyte.

Key things to note: This also applies to clerical errors. Even if there was no technical problem with the accuracy of your results, failure to submit results on time or clerical errors made while submitting can also have severe impacts on your ability to continue offering that test.

• Remedial Action: If you’ve been notified by your PT provider or state DOH to cease testing, there are extensive steps that must be completed to prove that the problem was correctly identified and corrected. You must also identify where samples will be referred to for tests you are unable to perform in-house.

Key things to note: If testing has been removed from your laboratory, you will be required to demonstrate successful performance in 2 consecutive PT survey events for the analyte(s) in question before being granted permission to resume patient testing. This can cause significant delays and financial impact for your organization.

Root Cause Analysis: Investigate to determine who, what, why, when, and how the event occurred. Be sure to evaluate all phases of testing to ensure you identify all potential causes.

• Pre-Examination:
  • Human Resources – evaluate the training and competency records for staff involved in the handling and testing of samples.
  • Facilities – reagent inventory control & storage temperatures, equipment maintenance and function checks
  • Standard Operating Procedures (SOPs) – staff compliance with written policies, bench excerpts are current and valid, document version control up to date
  • Specimen –test requisition/order entry (was the correct test code ordered/performed?), labeling (were aliquot/pour off tubes properly labeled?), transport (was appropriate temperature requirements maintained until testing performed), quality (was there visible deterioration with the sample prior to testing or cracked/damaged tubes received?), quantity (was the original sample spilled or leaking causing an incomplete aspiration of sample by your instrument?)

• Examination:
  • Method Validations – were instruments current with calibration requirements, any bias noted during instrument correlation studies, values being reported within the verified AMR
  • Environmental Controls – temperatures/humidity within tolerance limits, for light sensitive studies (bilirubin) was there excessive exposure of the samples to light prior to testing, excessive vibrations occurring that may have affected results (nearby construction or a running centrifuge on a shared work bench)
  • Quality Control – did QC pass on the day of testing, was QC trending or shifts noted that month
  • Analytical Records (worksheets) – were sample results transcribed correctly between the analyzer and worksheet, between the worksheet and LIS
  • Instrument Errors – were any corrective actions or problems noted for the days before, during, or immediately after testing of PT occurred
  • Testing Delay, Testing Errors – were samples prepared and not tested immediately leaving them exposed to light or air which may affect results (blood gas samples), any errors or problems noted during testing that may have caused a delay or affected accuracy of results

• Post-Examination:
  • Data & Results Review – check for clerical errors, was data trasmitted correctly from the instrument into LIS, was data entered correctly on your PT provider entry submission forms
  • Verification of Transmission – did your results correctly upload to the PT provider website, was there an error or failure with submission
  • Review of LIS – are your autoverification rules set up correctly, is the autoverification validation current with no known issues
  • Patient Impact – perhaps the most important step to take when reviewing PT failures, you need to determine what impact your failure had on your patient results. Depending upon the identified root cause and how different your values were from the intended response, this can potentially pose a severe impact on your patient values tested at the same time as the PT samples.

Involve your medical director to determine if the discrepancy in results is clinically significant. Perform a patient look-back to review patient values for the same analyte with the failure during the time period in question. Evaluate the bias that was present, and if deemed to be clinically significant then corrected patient reports will need to be issued with a letter from the medical director explaining why. If it was decided that the discrepancy is not clinically significant, document this in writing and keep on record with your complete investigation response.

Corrective Actions/Preventative Actions – use the following set of questions to help guide you in ensuring that the problem identified during your root cause analysis will not occur again:

• What changes to policies, procedures, and/or processes will you implement to ensure there will not be a repeat of this problem?
• Do any processes need to be simplified or standardized?
• Is additional training or competency assessment needed? If so, identify specific team members to be trained, and who will be accountable for performing and documenting this training.
• Is additional supervisory oversight needed for a particular area or step?
• Are current staffing levels adequate to handle testing volumes?
• Would revision or additional verification of the LIS rules address or prevent this problem?
• How can the communication between laboratory, nursing, and medical staff be improved to reduce errors in the future?

Continuous Process Improvement – after identifying the true root cause(s) for the failure and implementing corrective/preventative actions, you need to evaluate the effectiveness of those improvements. Have they been sustained? Are they working to correct the original problem? Have you created new problems by changing the previous process?

• Quality Management Meetings – if necessary, increase the frequency of these meetings during the evaluation period for timely feedback to management and staff
• Implement internal audits and quality indicators to check for potential issues
• Access the specimen transport conditions to ensure they meet test requirements
• Evaluate and monitor your turnaround time metrics to track problem specimens and impact of testing delays
• If necessary, increase the frequency when QC is performed or calibration frequency if stability issues are identified

Performing a thorough root cause analysis for any failures will allow you to implement appropriate corrective actions that will address the true issues. Having a robust quality management program will help ensure these issues are identified and corrected in a timely manner, and reduce the potential for the dreaded Cessation of Patient Testing letter from your local DOH.

Coming up in the final installment of this series on PT testing, we’ll review all of the quality indicators and data that can be found in your PT evaluation reports to help ensure you’re on track for accurate patient values.

-Kyle Nevins, MS, MLS(ASCP)^CM is one of ASCP’s 2018 Top 5 in the 40 Under Forty recognition program. She has worked in the medical laboratory profession for over 18 years. In her current position, she transitions between performing laboratory audits across the entire Northwell Health System on Long Island, NY, consulting for at-risk laboratories outside of Northwell Health, bringing laboratories up to regulatory standards, and acting as supervisor and mentor in labs with management gaps.