The FDA Asked How to Regulate Generative AI Devices. Physicians Have Until October 19.
On August 18, 2026, the FDA's Center for Devices and Radiological Health published a discussion paper on generative AI medical devices. It is not draft guidance. It is not final policy. It is a set of questions, and the comment window closes on October 19, 2026.
That distinction matters. A discussion paper is the moment when a physician's comment can still change the questions the agency treats as serious. After a draft guidance, the frame is already set.
The paper is Considerations for the Regulation of Generative AI-Enabled Medical Devices. Comments go to docket FDA-2026-N-7874.
The FDA is not asking whether generative AI belongs in medicine. It is asking how to tell a competent device from a fluent one.What the paper is actually proposing
CDRH starts from a fact clinicians already live with. A generative model does not behave like a calculator. The same question can produce different answers. The input can be an open-ended conversation. The model underneath the product may belong to a third party, and that third party can change it after the hospital has gone live.
The paper organizes the response in three moves.
Risk, on two axes. One axis is what the software does: offer information, direct an action, or take the action. The other axis is the harm if someone relies on a wrong output. A note that lisinopril is sometimes increased when blood pressure stays high is not the same object as an instruction to raise the dose from 10 mg to 20 mg. A suggestion to use over-the-counter hydrocortisone for poison ivy is not the same object as an instruction to change a basal insulin dose. Those examples are the FDA's, and they are the right clinical instinct.
A competency exam, not an endless test of every possible input. CDRH is considering a premarket approach modeled, loosely, on how physicians are credentialed: structured tests of knowledge and judgment, then supervised performance in realistic conditions. In the paper this is device benchmarking, followed by clinical confirmation. The thing being tested is the product a user will actually see, not the foundation model sitting alone in a lab.
Watching the device after it ships. Because these products keep changing, CDRH is considering whether some uncertainty at approval can be traded for harder postmarket monitoring: repeat the benchmark, have independent clinicians sample real outputs, and watch for drift when the patients, the data, or the underlying model shift.
None of this is a rule yet. The paper says so in the first paragraph.
Examples are the FDA's, shown here in everyday language. Darker boxes are the greater concern.
The risk that a disclaimer does not cancel
The part of the paper that matters most is about words that tell a person what to do. The FDA is not treating this as a simple on-off switch. The concern grows as the answer becomes more personal, and more like an instruction:
- A general fact. "Blood-pressure pills are sometimes increased when the reading stays high."
- A comment about this person. "Your blood pressure is still high on this medicine."
- A recommendation. "You should raise the dose from 10 mg to 20 mg."
- An order. "Take 20 mg tonight."
A warning at the end does not undo the instruction. If the software has already told the patient what to do, adding "talk to your doctor" or "I am not a medical professional" does not make that answer safe.
The dangerous answer is usually not a wild invention. It is a confident next step, written as if it belongs to the person in front of you.
Two more points are easy to miss:
- A chat can start with a fact and end with an order. Judge the whole conversation, not only the first reply.
- When the question is "should I go to the emergency room?" both mistakes matter. Missing a real emergency delays care. Sending everyone with chest discomfort to the emergency department causes its own harm: worry, extra tests, crowded departments, and people who stop trusting the tool the next time.
A written test is not the same as real use
The FDA is considering an exam before a product is allowed on the market. Passing a quiz is not the same as being safe with a real patient. The exam would look at four kinds of skill:
- Safety. Does it know when to stop and hand the person to a clinician? Does it stay inside the job it was built for? Does it say when it is unsure?
- Skill at the job. Does it know the subject, ask for the right information, handle numbers, and give an answer a person can actually understand?
- Whether the result holds up. Does the answer stay steady if you ask again? Does it work for different kinds of patients, not only the ones in the practice test?
- A harder exam, if the software acts on its own. A tool that takes the next step needs more than a tool that only answers a question.
A thorough quiz can still miss what happens in a real clinic. The paper also asks for a check in realistic conditions. A large clinical trial is the strictest check. It is not the only one. From lighter to stricter, the checks include:
- Looking back through real medical records
- Shadow mode: the software runs in a live clinic, but the care team does not see its answer
- Actors playing patients
- An outside doctor reading the answers
- A study that follows real patients going forward, when the risk calls for it
The paper is also honest about a trap:
- A famous practice test gets memorized. A product can score well on that quiz and still fail the patients a hospital actually sees.
- A company can write its own test and keep adjusting it until the product passes.
- If another AI grades the answers, that grader has to stand apart from the company that built the device and from the company that built the model. Independence is a setup, not a slogan.
A correct sentence is not strong evidence
In July I wrote on LinkedIn about a related failure. Clinical AI had learned to cite papers and still treat them as the same kind of proof. A large randomized trial and a chart review of 47 patients could sit under the same confident paragraph. A doctor does not need a faster footnote. They need to see how much the evidence can be trusted for the question in front of them.
That trust belongs to one question and one result. It does not belong to a paper by itself, and it does not belong to how sure the software sounds. That is the GRADE idea behind CertaintyGrade.
The FDA paper is the regulation version of the same argument. Its first question asks whether an answer should be traceable back to the original source. It should.
A device can pass a knowledge quiz and still do what I described in July:
- It can pull real citations.
- It can arrange them into smooth sentences.
- It can hide the fact that the evidence for this patient is weak.
A correct sentence and strong evidence are different things. An exam that measures only the first will approve tools that sound finished.
I have written separately about how to catch that kind of error before it reaches a patient, in AI hallucinations in medicine. The regulatory question is what the maker has to prove before the doctor is left to catch it alone.
What is worth saying before October 19
Comments are open until October 19. If you write one, or if your hospital's digital health group writes one, these are the points I would not leave to the device lawyers alone.
-
Show where the answer came from. If a clinician cannot check the answer against an original source, that answer is riskier, even when it only "gives information." The paper already treats a measurement this way, because the user cannot see how the number was made. A smooth paragraph with decorative citations deserves the same doubt.
-
Saying "I am unsure" is a safety feature. The paper's own exam list includes telling the user when the device is unsure, and handing the decision back. A product that sounds equally sure about a settled question and about thin evidence has failed a safety test. Say that in public comments, so this does not get watered down to a small disclaimer.
-
Watch it on real patients before it is allowed to give orders. Looking back at made-up cases is a start. It is not a substitute for watching the device on real patients while a person is still the only one allowed to act.
-
Say who it is being compared with. "As good as a clinician" can mean a panel of specialists, a typical general doctor, the human and the software together, or the software working alone. Those are four different products. Judge the tool against the person who will actually use it. A tool sold to general doctors should not be graded only against specialists who would never have missed the trap.
-
If the hidden model changes, that is the maker's problem, on a clock. If the model underneath updates on a Tuesday, the product on top of it is a different product on Wednesday. Ask for three things: notice when it changes, give the exam again, and pause any feature that tells people what to do until that re-check is done. That is a patient-safety comment.
-
The maker stays responsible after it ships. The paper invites hospitals, medical societies, and insurers to help watch these products. That help is useful. It becomes dangerous if "the hospital will watch it" replaces a duty the company can be held to.
The companies will write. The comments this docket is short on are from people who have had to decide, with a patient waiting, whether to trust an AI sentence. Physicians should write too.
Frequently Asked Questions
Q: Is this new FDA guidance I have to follow?
A: No. The August 18, 2026 paper is a discussion document. CDRH says it is not draft guidance, not final guidance, and not a statement of what evidence a future submission must contain. The actionable date is October 19, 2026, when comments on docket FDA-2026-N-7874 are due.
Q: Does the FDA regulate ChatGPT itself?
A: The paper draws the line the agency already uses for software. FDA does not regulate generative AI as a technology. It regulates products that meet the definition of a medical device. A general chatbot is not automatically a device. A product that uses generative AI to direct or take a clinical action may be.
Q: What is the "competency" approach in plain language?
A: Test the finished product the way we test a trainee. First, a structured exam of knowledge, safety behavior, communication, and whether the result holds up across patients. Then confirmation in realistic clinical use, which might be a chart review, a silent shadow period, standardized patients, independent clinicians, or a prospective study. The rigor is supposed to rise with risk.
Q: How does this relate to evidence grading and CertaintyGrade?
A: A device can be factually fluent and still rest on weak evidence. GRADE rates certainty for a defined question and outcome. CertaintyGrade applies that standard to a question you specify, instead of treating every cited paper as equally trustworthy. The FDA is asking how to test device competence. Evidence grading asks a second question the exam can miss: even if the words are competent, how strong is the evidence a clinician is being asked to act on?
Q: What should a physician do with a generative AI tool already in the workflow?
A: Use it as a draft. Check one primary source before you act. Notice when the tool stops informing you and starts instructing you. If your hospital is deploying a generative feature that writes orders, patient advice, or triage, ask whether it has been watched in shadow mode on your patients, and what happens when the underlying model changes.
Dr. Adarsh K. Gupta, DO, MS is a clinical AI strategist and the author of The Science of Habits. He is a Cochrane Associate Editor and the founder of CertaintyGrade, a GRADE-based platform that rates the certainty of clinical evidence for a specific question. Follow him at adarshgupta.com.