Missed Opportunity (AI/ML)
On or about April 02, 2026 - I logged into the internet and encountered big hype due to the issuance of the first FDA Warning Letter related to the inappropriate use of AI in GMP. I was also hyped! But after reading the Letter, I felt a pretty deep lump in my stomach… I often feel ill after reading these Letters, but generally due to grave safety concerns. Not in this case. My sick feeling was due to the oversimplification of the Letter’s content and missed opportunity.
FDA had the opportunity here to outline the firm’s failures at each step of the AI lifecycle, from development to deployment to monitoring, but chose to take the easy route and simply cite lack of QA oversight (211.22c). The citation is obviously not incorrect, QA must oversee the use of AI in GXP, but the oversimplification will do industry no favors in the long-term.
“If you use AI as an aid in document creation, you must review the AI generated documents to ensure they were accurate and actually compliant with CGMP. Your failure to do so is a violation of 21 CFR 211.22(c).”
That’s it? Really? We can simply just ask QA to “review” the documents and we are free to use AI as we like? This is what caused my sinking feeling… I feel sorry for the QA folks around the world who are going to be placed in an impossible situation - reviewing for “accuracy” when the governance mechanisms are not established. I can imagine sites around the world reading this Letter and greatly underestimating what FDA really means by “ensure they were accurate” - which (depending on the COU) can be a great deal of background work related to development/validation, documentation, personnel training and ongoing monitoring. The failure to include a section in the Letter expanding on the expectation for “accuracy” is a big mistake by FDA.
It appears this letter was rushed out the door without fully understanding the gravity of releasing the first AI-related Warning Letter. This is coupled by the fact that the existing guidance for use of AI in GXP is thin, with vague pseudo-expectations and in the case of the EU, a published draft regulation that does not apply to LLMs (wild!!)!
The use of AI at this firm was cynical to begin with - responding to an FDA query regarding process validation: “You replied that you were not aware of the legal requirement, as the AI agent you used (b)(4), never told you it was required.” This was not a serious manufacturer, and apparently stopped manufacturing drug products shortly after the FDA inspection concluded. So, we wait in anticipation for the first serious AI-related Warning Letter, and we wait hoping that the FDA will provide QA with a clear pathway for compliance with 211.22c.
How long will we wait, I don’t think much longer…
