Tag: proficiency testing

Last month we discussed the rules and requirements for how to properly perform proficiency testing (PT) within your laboratory. In part 2 of this 3-part series we’ll review the rules associated with evaluating your results, and how to investigate any unsuccessful surveys. Still to come in part 3 we will look into how to utilize your PT results to monitor for trends and shifts in your values.

The rules:

• Performance Review: Laboratories must initiate and document a review of their PT performance evaluations within 2 weeks of notification that results are available. This includes a review of both graded and non-graded/educational analytes and events as well.

Key things to note: Even though educational samples are not formally graded, you should still verify the accuracy of your results, with appropriate follow-up for any failures. CAP specifically requires you to evaluate these educational challenges as well. Whether the sample is graded or not does not change the fact that you had an incorrect result.

• Unsatisfactory Performance: For any unsatisfactory results, you are required to perform a root cause analysis to determine why (see below for guidance). This also includes any clerical errors – you need to evaluate your process and find ways to prevent these simple errors from happening again. If they are happening with PT samples, it is possible they are happening with patient samples as well.

• Cessation of Patient Testing: Unsatisfactory events indicate that there was a problem with that particular survey; whereas unsuccessful events indicate there has been a pattern of unsatisfactory events/samples and a larger problem exists. If a pattern of poor performance is detected, you may be asked by your local state department of health to cease all testing for a particular analyte.

Key things to note: This also applies to clerical errors. Even if there was no technical problem with the accuracy of your results, failure to submit results on time or clerical errors made while submitting can also have severe impacts on your ability to continue offering that test.

• Remedial Action: If you’ve been notified by your PT provider or state DOH to cease testing, there are extensive steps that must be completed to prove that the problem was correctly identified and corrected. You must also identify where samples will be referred to for tests you are unable to perform in-house.

Key things to note: If testing has been removed from your laboratory, you will be required to demonstrate successful performance in 2 consecutive PT survey events for the analyte(s) in question before being granted permission to resume patient testing. This can cause significant delays and financial impact for your organization.

Root Cause Analysis: Investigate to determine who, what, why, when, and how the event occurred. Be sure to evaluate all phases of testing to ensure you identify all potential causes.

• Pre-Examination:
  • Human Resources – evaluate the training and competency records for staff involved in the handling and testing of samples.
  • Facilities – reagent inventory control & storage temperatures, equipment maintenance and function checks
  • Standard Operating Procedures (SOPs) – staff compliance with written policies, bench excerpts are current and valid, document version control up to date
  • Specimen –test requisition/order entry (was the correct test code ordered/performed?), labeling (were aliquot/pour off tubes properly labeled?), transport (was appropriate temperature requirements maintained until testing performed), quality (was there visible deterioration with the sample prior to testing or cracked/damaged tubes received?), quantity (was the original sample spilled or leaking causing an incomplete aspiration of sample by your instrument?)

• Examination:
  • Method Validations – were instruments current with calibration requirements, any bias noted during instrument correlation studies, values being reported within the verified AMR
  • Environmental Controls – temperatures/humidity within tolerance limits, for light sensitive studies (bilirubin) was there excessive exposure of the samples to light prior to testing, excessive vibrations occurring that may have affected results (nearby construction or a running centrifuge on a shared work bench)
  • Quality Control – did QC pass on the day of testing, was QC trending or shifts noted that month
  • Analytical Records (worksheets) – were sample results transcribed correctly between the analyzer and worksheet, between the worksheet and LIS
  • Instrument Errors – were any corrective actions or problems noted for the days before, during, or immediately after testing of PT occurred
  • Testing Delay, Testing Errors – were samples prepared and not tested immediately leaving them exposed to light or air which may affect results (blood gas samples), any errors or problems noted during testing that may have caused a delay or affected accuracy of results

• Post-Examination:
  • Data & Results Review – check for clerical errors, was data trasmitted correctly from the instrument into LIS, was data entered correctly on your PT provider entry submission forms
  • Verification of Transmission – did your results correctly upload to the PT provider website, was there an error or failure with submission
  • Review of LIS – are your autoverification rules set up correctly, is the autoverification validation current with no known issues
  • Patient Impact – perhaps the most important step to take when reviewing PT failures, you need to determine what impact your failure had on your patient results. Depending upon the identified root cause and how different your values were from the intended response, this can potentially pose a severe impact on your patient values tested at the same time as the PT samples.

Involve your medical director to determine if the discrepancy in results is clinically significant. Perform a patient look-back to review patient values for the same analyte with the failure during the time period in question. Evaluate the bias that was present, and if deemed to be clinically significant then corrected patient reports will need to be issued with a letter from the medical director explaining why. If it was decided that the discrepancy is not clinically significant, document this in writing and keep on record with your complete investigation response.

Corrective Actions/Preventative Actions – use the following set of questions to help guide you in ensuring that the problem identified during your root cause analysis will not occur again:

• What changes to policies, procedures, and/or processes will you implement to ensure there will not be a repeat of this problem?
• Do any processes need to be simplified or standardized?
• Is additional training or competency assessment needed? If so, identify specific team members to be trained, and who will be accountable for performing and documenting this training.
• Is additional supervisory oversight needed for a particular area or step?
• Are current staffing levels adequate to handle testing volumes?
• Would revision or additional verification of the LIS rules address or prevent this problem?
• How can the communication between laboratory, nursing, and medical staff be improved to reduce errors in the future?

Continuous Process Improvement – after identifying the true root cause(s) for the failure and implementing corrective/preventative actions, you need to evaluate the effectiveness of those improvements. Have they been sustained? Are they working to correct the original problem? Have you created new problems by changing the previous process?

• Quality Management Meetings – if necessary, increase the frequency of these meetings during the evaluation period for timely feedback to management and staff
• Implement internal audits and quality indicators to check for potential issues
• Access the specimen transport conditions to ensure they meet test requirements
• Evaluate and monitor your turnaround time metrics to track problem specimens and impact of testing delays
• If necessary, increase the frequency when QC is performed or calibration frequency if stability issues are identified

Performing a thorough root cause analysis for any failures will allow you to implement appropriate corrective actions that will address the true issues. Having a robust quality management program will help ensure these issues are identified and corrected in a timely manner, and reduce the potential for the dreaded Cessation of Patient Testing letter from your local DOH.

Coming up in the final installment of this series on PT testing, we’ll review all of the quality indicators and data that can be found in your PT evaluation reports to help ensure you’re on track for accurate patient values.

-Kyle Nevins, MS, MLS(ASCP)^CM is one of ASCP’s 2018 Top 5 in the 40 Under Forty recognition program. She has worked in the medical laboratory profession for over 18 years. In her current position, she transitions between performing laboratory audits across the entire Northwell Health System on Long Island, NY, consulting for at-risk laboratories outside of Northwell Health, bringing laboratories up to regulatory standards, and acting as supervisor and mentor in labs with management gaps.

Every laboratory knows that they must participate in proficiency testing (PT) for all of the regulated analytes they report. But did you know that there is more to it than simply checking your overall score in each survey you participate in? Whether you utilize samples from the CAP, API, or have developed your own in-house blind sample testing algorithm, there is a lot of data available to help you assess the quality of your laboratory program. In the first of this 3-part series, we’ll review why PT testing is important and the rules that must be followed. In part 2 we’ll discuss how to properly perform an investigation when scores are <100%. Lastly, in part 3 we’ll look at how to review your results so that you get the most out of them for a successful quality laboratory.

Why participate? Well frankly, because you have to. It is a CLIA/CMS requirement, and if your lab has additional accreditations, those agencies will have their own rules and requirements as well (we’ll get to the rules in a little bit). But outside of the regulations stating you must participate; all labs should want to participate. It’s an opportunity to check your accuracy against peers who are using the same instrumentation as you. Similar to utilizing an affiliated QC report, this is a way to see what the “real” value is supposed to be (despite what a manufacturer may claim it to be), and how close/far off your lab is to that true value. It can help you identify potential problems before they become huge problems with patient values being affected, and it’s also a great way to satisfy competency requirements for your staff.

The rules:

• Participation: For every regulated analyte being tested under your laboratory permit¹, you must participate in a CMS-approved PT program².

Key things to note: This only applies to testing performed using non-waived methodologies. Waived testing is exempt from PT requirements; although it is still recommended that participation occur if an evaluation program is available. Additionally, this only applies to your primary instrumentation. For example, if you have an automated urinalysis reader and your backup methodology is to read dipsticks manually, you are only required to participate in PT for the primary methodology. (Your backup method would then be evaluated for accuracy through semi-annual correlation studies.)

• Routine Analysis: Unless otherwise instructed by the provider of your PT samples, PT samples are to be treated the same as patient samples. Meaning they are handled, prepared, processed, examined, tested and reported the same way you would perform patient testing; AND by the same staff who would handle patient testing.

Key things to note: If nursing staff perform a particular test within their unit (for example, ACT testing in the cardiac cath lab), it is those nursing staff members who must run the PT samples. You cannot have the laboratory perform PT testing unless the laboratory also performs the patient testing. Additionally, PT samples should be rotated among all staff members who perform patient testing. Meaning all shifts, and all days of the week that the test is performed – don’t let the day shift get all the fun.

• Repeated Analysis: Similar to rule #2, unless you routinely perform duplicate testing on your patient samples, you cannot perform duplicate or repeat testing on your PT samples. You cannot run a PT sample in duplicate “just to make sure.” Patient samples are just as important to be accurate as a PT sample, which is why we participate in a PT program in the first place.

Key things to note: After the date that laboratories are required to report results back to the PT provider, you are then allowed to use the samples for repeat testing. This can be used to check for uniformity in grading of reactions among staff members, and to assess annual competency. But only after the submission date has passed.

• Interlaboratory Communication: You cannot discuss the results or samples from a PT survey with any other laboratory (or Facebook user group) until after the results submission deadline has passed. Doing so before that time would be considered cheating. The point of PT testing is not to see how good your networking skills are, but to ensure accuracy of your own results. Plus, the other lab may not be as good as you think they are.

Key things to note: If your laboratory is part of a larger integrated health system, be careful that you have separate designated staff assigned to enter results from each location. Entering results for more than one permit number by the same person would be considered a violation of the interlaboratory communication rule as they could compare results from Lab A to Lab B prior to submitting. Also, be mindful of what you put on social media. User groups are a great networking resource and learning tool, but you still need to follow the rules. Violating them in a public arena such as Facebook for all the world to see would put yourself and your organization in great jeopardy if you were caught. 

• Referral of Samples: You are not permitted to forward or share your PT samples with any other laboratory until after the result submission deadline has passed. Similarly, if your laboratory has received PT samples from another lab, state regulations may require you to notify your local Department of Health to inform them of the violation.

Key things to note: The intended purpose of performing PT testing is to verify the accuracy of your own laboratory testing. If you would routinely send a positive sample to a reference lab for additional confirmation testing, you would not do so in this case. Simply report out the values for the tests that your laboratory performs only. The reference laboratory will have their own PT samples to check accuracy for the confirmation testing they perform for you. Ensure your testing menu is up to date and accurate so that your PT provider is not expecting values for a confirmatory test if you do not physically perform it in-house.

• Records Retention: Ensure that all records and documents related to the testing of PT samples are saved for the amount of time required by your regulatory agencies (typically 2-5 years). This includes instrument print outs, LIS chart copies of the filed results, QC records for the day of testing, and any associated worksheets used to document your results.

Key things to note: Retaining a copy of the instrument maintenance logs and QC records along with the actual PT results will help you investigate any scores that are less than 100%.

• Attestation: Both the laboratory director and all personnel performing testing must sign the included attestation statement. This is not just a way to track who performed the test, but is a legal binding document assuring that testing was carried out appropriately as per the rules defined above.

The penalties for labs that are caught violating the rules (whether intentionally or not) can be quite severe. These penalties can include the revocation of your CLIA permit; a ban for the laboratory owner and laboratory director; as well as possible financial penalties and fines.

Coming up in the next blog we’ll review the rest of the rules related to evaluation of your scored PT results, and how to perform a thorough investigation into any unsuccessful survey events.

1: https://www.cms.gov/regulations-and-guidance/legislation/clia/downloads/cliabrochure8.pdf

2: https://www.cms.gov/Regulations-and-Guidance/Legislation/CLIA/Downloads/ptlist.pdf

-Kyle Nevins, MS, MLS(ASCP)^CM is one of ASCP’s 2018 Top 5 in the 40 Under Forty recognition program. She has worked in the medical laboratory profession for over 18 years. In her current position, she transitions between performing laboratory audits across the entire Northwell Health System on Long Island, NY, consulting for at-risk laboratories outside of Northwell Health, bringing laboratories up to regulatory standards, and acting as supervisor and mentor in labs with management gaps.