The following is a transcript of a conversation between Ayesha Rascoe, NPR Host, and Robert Califf, FDA Commissioner, about how AI is affecting medicine:
AYESHA RASCOE, HOST:
Artificial intelligence is used in everything from creating funny images on the Internet to organizing office records. And increasingly, it's also an integral component to health care. However, some patient advocacy groups are worried. They want more AI regulation in the healthcare industry. Last month, U.S. Food and Drug Administration experts agreed in an article they wrote for JAMA, the Journal of the American Medical Association. To tell us more, we're joined by the paper's senior author, FDA Commissioner Dr. Robert Califf. Thank you for being here.
ROBERT CALIFF: Ayesha, it's great to be with you.
RASCOE: You write that over the past 30 years, the FDA has authorized nearly 1,000 medical devices that use artificial intelligence. What do these devices do?
CALIFF: You know, I think of it in terms of category. So there's definitely a category where most of the action has been is in imaging, a CT scan or an X ray, and you want to make sense of it. The radiologist looks at it as a human being, and we all have limited processing capability. A computer never gets tired, doesn't need a cup of coffee and can analyze the data broadly.
But the much, much larger area that has implications for all of us is what's called decision support. And that is, either a consumer looking on the Internet or your doctor or a nurse dealing with a patient. You're inputting information. That information is processed, and it gives you a recommendation.
RASCOE: How does the FDA right now evaluate AI-enabled devices? Can you sketch out the process just generally for a layperson?
CALIFF: There are actually a set of laws that govern this. So if it's an AI algorithm which is embedded in a high-risk device - let's say a cardiac defibrillator. That's a really good example. If your heart suddenly stops beating, you want a shock to get your heart started again. You want to make sure that algorithm is right. That is very carefully regulated, and you got to do clinical studies, typically in people, that show that the algorithm actually really works. You have administrative uses of AI. Those aren't regulated at all. That's administration and finance.
In the middle, we have this vast area of decision support that I described. It ranges from, "I'm working out, and I want to know what my heart rate is," to "I've got heart failure. I could die." I want to make sure when I work out, I've got careful control of the parameters. And that would be more regulated, so there's a spectrum.
The bottom line is that this field will be so big that we couldn't hire enough people at the FDA to regulate it all on our own. So we must get the clinical community—the health systems, the professions, the people that you see for your health care—very involved in developing what we would call an ecosystem of regulation where these algorithms can be constantly evaluated.
RASCOE: So some advocates think that the products that help with, say, decision support - they should also go through clinical trials. Do you agree with that? Should they go through clinical trials?
CALIFF: Well, you know, besides being a cardiologist, I was a clinical trialist. That's what I did for a living. So I've never seen a clinical trial I didn't like. And of course, the more clinical trials we can do on these algorithms, the better. But there's an element of this which is different. When you produce a drug or a traditional device, it's the same thing for the rest of its existence. Here, the decision support the AI algorithms are changing every day. And so the real key here is ensuring they're safe at the beginning and then monitoring them. So I think the consumer advocacy groups are onto something here, and we'll have to work out exactly where to draw a line in the premarket phase for these algorithms, what it takes for them to get on the market versus what happens afterward.
RASCOE: I think one concern that people will have and one example that's come up, say, if AI product is using billing records to predict someone's future health care needs, the AI may falsely presume that low income people have fewer needs than wealthier people. But in reality, it's just that they have received fewer services because they don't have the resources to get them. Do your review procedures take things like that into account, or is that something that the FDA would have any insight on?
CALIFF: Well, you're asking a really important question there. And yes, we have a lot of insight into this. And you're completely right in your example, that if you have an AI algorithm, it's only as good as the data that goes in. But we're hyperaware of what you said, which is bias in formulating an algorithm leads to biased output and can make our inequities even worse than they already are, and they're pretty bad right now.
RASCOE: What do you say to those people who are concerned that, you know, a computer may be deciding whether they have some condition or not?
CALIFF: One of the most important things is, for the most part, for important decisions, AI should inform the clinician with whom you're working, not make the decision for that person. The decision should be made by a human being that's interacting with you, can understand dimensions of you that are not necessarily easily picked up by a computer.
On the other hand, I'd also point out that your doctor or nurse, as I mentioned at the beginning of this conversation, could be tired, might have had a spousal fight in the morning before going to work, or could be overwhelmed by other issues. A computer doesn't get tired, and it's not distracted by other things in the environment. In my 40 years of working on computer-human interfaces, the combination is always better if the rules of the road are right.
RASCOE: That's FDA Commissioner Dr. Robert Califf. Thank you so much for talking with me.
CALIFF: It's an absolute pleasure. Thanks for giving me a chance to go over this.
Copyright © 2024 NPR. 11/03/2024 --- All rights reserved. Visit our website terms of use and permissions pages at www.npr.org for further information.
NPR transcripts are created by an NPR contractor on a rush deadline. This text may not be in its final form and may be updated or revised. Accuracy and availability may vary. The authoritative record of NPR’s programming is the audio record.
Comments