Credibility
While browsing Redica today, I came across an interesting FDA Form 483 Observation citing “Second Stage Verification” as part of the AVI workflow, where vials classified as “undetermined” were fed back through a second time to reduce false rejects. This might be considered “testing into compliance”, a big GMP no-go. But why is this not allowed? As I read through the observation several times, I found the answer is not as clear as traditional old-school QC examples. Understanding the why behind regulatory observations is important, so I’ll give it a try in this blog entry!
We typically attribute “testing into compliance” with QC testing, where re-testing without justification is a risk due to inherent intra-batch variability. This is an age-old problem as is likely here to stay, as we humans struggle balancing objectivity and bias, especially when a multi-million dollar batch is in quarantine awaiting QC sign-off.
This AVI situation is different, however, more closely related to re-analysis of an original HPLC vial as part of a phase I OOS investigation. In this QC case, however, we have an OOS investigation where the obvious root cause (typically a hardware error) is documented and justified. The method is considered validated, and the error is part of natural variability (risk). Patient risk is under control.
In the AVI example, we have no such investigation (documentation and justification) and no obvious root cause - this is the reason for regulatory concern and why this practice is prohibited. The FDA guidance for AI in manufacturing came to mind, where the term “credibility” is king. The AVI example in the 483 may or may not be AI-driven, but the concept remains valid. If the root causes for high reject units is unknown, and the firm has simply resorted to second pass inspection to supplement lack of understanding/risk management, is the computerized system (AI or not) credible? Considering the criticality of visual inspection, a solid understanding of failure modes and control of those risks is the foundation to demonstrate credibility/validation. Just stating that the system is ultra sensitive is not enough, considering the criticality of visual inspection. How could the same unit be rejected on the first pass, then classified as acceptable during the second pass? Air bubbles, failure of the camera to pick up the particle, there may be more causes, including unknown unknowns… To feed unclassified rejects through a second time is quite risky - as no system is 100% perfect. This confidence in the system would be considered an “unjustified assumption” (see PIC/s QRM Guidance, 2021). The main risk to patient here is that the second pass results in a false negative, as again, no system is 100% accurate. This would be a tragedy if the unit had already been flagged as uncategorized and rejected on the first pass. We can do better!
In summary, rather than implement a “second pass inspection”, the firm should have put efforts into understanding and mitigating the risks of false rejects prior to going live, gaining more knowledge of the system prior to putting into service within a critical GMP workflow. The “system” was not validated/credible (whichever term you prefer) - and the 483 is very well written!
