Can Laws Keep Up With AI Therapists?

[00:00:02] Hey, TMB listeners. AI chatbots offering mental health therapy, or something like it, are getting the attention of regulators. Last month, my colleagues on our daily What's News podcast looked into how authorities are reining in a growing industry on the bleeding edge of AI. Today, we've got the final episode in their special series, The AI Therapist. You can hear the other installments in the TMB feed, or jump right into part three now.

[00:00:34] I think I was in Chicago, and I had a three-hour layover, and so I decided, hey, this is a good time. That's Tyler Lord. He's 36, and he's been dealing with depression. And he says what it was a good time for was using an AI mental health chatbot called ASH. He figured he could get a quick session in on his layover. I probably couldn't talk in the airport, but I could type out whatever I needed. But it turns out he couldn't.

[00:01:00] As soon as I opened it, you'd get a pop-up saying, hey, the legislators in this state have banned AI therapy, so unfortunately you cannot use it here. I was like, oh man, that sucks. He did a quick Google. Yup, AI tools for therapy were restricted in Illinois. And for Lord, who'd been considering moving to Chicago, not having ASH was a deal-breaker. The fact that I wouldn't be able to use it if I was in a state like Illinois, that could have profound consequences.

[00:01:27] And maybe my dream job is in Chicago, and I don't know it. And I'll never know it because I'll never move there. Because I like having the safety net that ASH provides, even if it's not a requirement. It just makes dealing with those tough moments so much easier. And so it's like, why am I going to make life harder for me by moving to a place where I don't have it anymore? Actually, the amount of time that ASH wasn't available in Illinois was pretty brief. We'll get into why it went away and why it came back a little later.

[00:01:58] But Illinois isn't the only state that's setting limits on AI therapy. Lawmakers are worried about these chatbots. So what do these laws do? And how are they changing the industry? I'm Julie Chang for The Wall Street Journal. This is What's New Sunday, the third and final episode of The AI Therapist. So far, we've looked at why AI tools for mental health are catching on, the risks that may come with using them,

[00:02:25] and why the companies behind the chatbots say they're safe. You should go back and listen to those two episodes if you haven't yet. Today, we'll be talking about what the law says about how people should be using these tools, where some say it falls short, and the future of AI tools for mental health. Let's travel back in time for a second to July 2025.

[00:02:54] Okay, that was not a very long journey. It was only about a year ago. A startup called Slingshot AI was launching its AI chatbot, Ash. And Slingshot CEO Daniel Kahn was busy promoting it with a video. I'm happy to share that today, we're announcing the launch of Ash, the first AI designed for therapy. We built a world-class... AI designed for therapy. That turned out to be a significant word choice. If you go to Ash's website today, the tagline says,

[00:03:24] AI designed for mental health. It doesn't say therapy. I asked Slingshot co-founder Neil Parikh about it. He said the company changed the wording on its website last fall, and they did it because of state laws that say something can't be called therapy unless a licensed professional is doing it. But he says that doesn't change Ash's mission. We don't call it AI therapy, right? But when we think about the value proposition to people is similar, which is like you have a problem, you can show up, and Ash is there to help you think through that.

[00:03:54] So Ash might be therapy in everything but name, but some companies embrace the label. On its website, one startup, Harmony, calls itself the first emotionally intelligent AI therapist. And still others say that actually they're not therapy at all. For example, Talkspace. It's a publicly traded company that connects people to mental health providers online. Earlier this summer, it launched an AI tool called T.

[00:04:20] And John Cohen, the CEO of Talkspace, made it very clear to me, T is not therapy. So it's a different population of people seeking a different type of conversation than a person who's actually seeking Talkspace for therapy. It's a different leap. It's a, I've recognized I have a problem, whatever it is, I really want professional help. These are people who haven't said they want professional help. They just want to talk to somebody. It's different.

[00:04:47] So why are some of these companies touchy about the word therapy? One big reason, as Slingshot told us, is that several U.S. states are passing laws to restrict the use of AI therapy. That's what Tyler Lord ran into in Illinois. Last year, the state's governor, J.B. Pritzker, signed a law prohibiting AI therapy. There wasn't much disagreement about this among legislators. The state House and Senate both passed the bill unanimously.

[00:05:14] Simply put, the law requires that mental health services be provided by humans. The man in charge of enforcing this law is Mario Tredo Jr. He's the secretary of the Illinois Department of Financial and Professional Regulation. The purpose of it was to ensure that we're protecting consumers and setting the appropriate guardrails in a rapidly evolving industry.

[00:05:40] There's a reason why we have licensed therapists, licensed clinical social workers that provide this critical care. Really, what we're wanting to make sure is that we have a licensed individual providing the mental health services. We want to make sure that there's a human in the loop. Other states have recently passed laws limiting AI for therapy, such as Nevada, Tennessee, Maine, Colorado, Vermont, and Rhode Island. Those last three just this summer.

[00:06:09] And lawmakers in other states, including New York and California, are considering similar legislation. Michelle Mello is a professor of health policy at Stanford's medical school. She's also a law professor, and she's been following the rollout of this legislation. Broadly speaking, she says it's early days for laws like these. It looks like a lot of early experimentation among people who are trying hard to be helpful, but still figuring their way.

[00:06:35] To her, these laws fall into a few different categories, starting with the kind of AI they address. And there I would distinguish between the three types, the therapy bots that are built for the purpose of providing mental health care, general stuff like ChatGPT, and then the companion bots that are built to provide social and emotional support, but not mental health care. Most of the laws target the first thing, the therapy bots. And then the second thing is, what is it doing? And for whom?

[00:07:02] And there we see a lot of states and the Congress focusing on protection of children and teenagers. And then there's the question of enforcement. The way it works in several of these states is that someone has to complain that a company is violating the law before regulators get involved. Mello says that's pretty much the only way for governments to know what's going on. Illinois says it has received consumer complaints related to the AI therapy law, but it hasn't issued any fines, at least not yet.

[00:07:31] We asked other states with similar laws about this too, and none of them, from what we found, have fined any companies. Still, chatbot makers are changing their approach. We'll get into how after the break. So, a growing number of states are implementing laws to limit how AI can be used for mental health treatment.

[00:08:01] Not everyone's a fan, particularly some of the scientists and companies behind those AI mental health tools. Take Nick Jacobson. He's an associate professor of biomedical data science, psychiatry, and computer science at Dartmouth. We started working on trying to use generative AI to provide evidence-based mental health care starting in 2019. He's been developing his own AI therapy tool called Therabot, but it's still being tested and is not available to download for just the average person.

[00:08:30] Jacobson says that it's the general-use AI chatbots like ChatGPT that are linked to psychosis. You heard about that on our earlier episodes. But he thinks that the state laws target specialized mental health apps more, which means, he says, that these state bans don't actually do what they set out to. They don't often have exemptions for these clinical products that are effective and safe. Illinois's regulator, however, did tell us that the law applies to any company that provides therapy.

[00:08:59] Michelle Mello, the Stanford law professor, says chatbots designed for therapy do tend to be the ones most subject to state regulation. Some, not all, but some of the therapy bots have worked really hard to share out evidence through a rigorous study that they produce better patient outcomes or comparable patient outcomes to human care. They're the ones that are easy to regulate because they're tied into structures that we already have for regulation.

[00:09:26] So we told you about Tyler Lord at the top of the show. He's the one who couldn't use ash during a layover in Chicago last year. But earlier this month, we asked one of our Wall Street Journal colleagues in Illinois to see if he could use ash. And it turns out he could. I asked Neil Parikh, the co-founder of Slingshot, why ash is now available in Illinois. He said that ash went dark for a few months in Illinois and Nevada, while Slingshot figured out how these new laws applied.

[00:09:54] But remember what I mentioned earlier about how Slingshot did away with the word therapy on its website? After making that change, the company restored access to its services. Some markets have proven trickier to navigate. In January, Slingshot pulled ash from the United Kingdom, citing concerns about how regulation works there. The country doesn't have a ban on AI therapy tools, but it does have rules around how AI can be used for medical purposes.

[00:10:21] Other companies say they'd like to see more federal regulation here in the U.S. John Cohen, the CEO of Talkspace, says its tool, T, complies with the laws in all 50 states. In fact, he says federal regulation would actually help T. I would love to see a federal solution so that we don't have to deal with 50 different laws and states and regulatory oversight that's different from state to state. I'm not worried about what we built.

[00:10:48] I think other people are worried about what they built because they don't have the safeguards that we put in place, quite honestly. Coming up, if you confide in an AI therapist, are your chats private? That's after the break.

[00:11:03] So AI therapy restrictions are becoming more common, but that still leaves a lot of gray areas. Like, what about apps that say they're wellness tools or coaching tools? For apps like these, which is most of them, there are very few rules in place.

[00:11:31] They don't have to work with clinicians to come up with guardrails, and they don't have to have a human in the loop for potentially risky conversations. In our reporting about the regulatory landscape, there was one phrase that kept coming up. It, on the regulation front, is quite the Wild West. There's no real regulation in this space because it's kind of the Wild West. It gets oversaid a little bit of the Wild West. One area where experts said more regulation is needed is privacy.

[00:11:58] Vail Wright, Senior Director of Healthcare Innovation at the American Psychological Association, told me it's definitely something users should be thinking about. It's important to remember that these chats are not private, and data breaches are 100% not preventable. So if you don't want somebody to know what you're chatting, if it's something that you feel too sensitive for, say, your family or your boss or somebody to know about, you may want to think twice about putting it into these chatbots.

[00:12:26] John Cohen, the CEO of Talkspace, talked a lot in our interview about how T protects user privacy. But on one key topic, he kept correcting himself. We built T with HIPAA-type protections on it. HIPAA is the longstanding federal law that safeguards patient medical data. I asked him about the distinction he was making between HIPAA and HIPAA-like.

[00:12:50] The law doesn't provide us to say that this is a HIPAA-protected delivery because it's not a medical device or a medical person. So what we did is we used all of the issues around how you make something HIPAA-protected and apply it to that. But I can't say it's HIPAA-protected because HIPAA only applies to live human interaction. So T promises basically the same kind of privacy protections you would get if you went to your doctor.

[00:13:20] Among other things, not selling your data to another company without your permission. Many of these tools say user data is protected via encryption. But they also have optional settings where they save user conversations to train their models. John Torres, a psychiatrist at Boston's Beth Israel Deaconess Medical Center, says this presents a risk that users might not fully realize. You're probably not being given clear, direct, informed consent where you are explained the risks and benefits and what happens to your data.

[00:13:49] And that is not going to be a bright spot just for the field of how these companies built these models. But for many of us, privacy is just one of those worries that tends to fall by the wayside. You could say it's a casualty of life spent interacting with the internet. That's how Tyler Lord thinks about it. He's the man from Kentucky that you heard at the beginning of the episode. He says what he gets out of talking to Ash is worth the risk of a potential leak of his data. Every couple of months you hear about some big company that had some data breach and a ton

[00:14:19] of stuff is leaked. It's always going to be a risk. But to me, it's worth the risk because the benefits would outweigh the possibility of that risk. Because I know it's not a guarantee thing, but I'm kind of guaranteed, at least from my experience, to get help from the app. I asked another Ash user, Jennifer Millie, a 45-year-old attorney living in Los Angeles, whether privacy was something she worried about when using the app.

[00:14:47] As a lawyer, I should say yes. As a person with an iPhone and a TV and a laptop and Alexa and all of the things, I figure all of my personal information is already out there, so I'm pretty open about it. So for now, regulation may be a Wild West. But the federal government has started making some early moves. Good morning. I would like to call this meeting of the FDA's Digital Health Advisory Committee to order on November 6, 2025.

[00:15:15] In November, the FDA's Advisory Committee for Digital Health held a meeting about generative AI-enabled tools used for mental health. The FDA makes a distinction between general-use AI tools like ChatGPT or CLODE and those that use AI to do things like diagnose and treat mental health conditions. Here's how Michelle Tarver, the director of the FDA's Center for Devices and Radiological Health, described the goal of the meeting in her opening remarks.

[00:15:41] And yes, for these purposes, the FDA considers ChatBots medical devices. The FDA has authorized over 1,200 AI-enabled medical devices. These innovations help strengthen health care and provide diagnosis, cures, and treatment to patients for a range of diseases and conditions. However, to date, there are no generative AI-enabled medical devices that have been authorized for use in mental health conditions.

[00:16:08] Over the course of the eight-hour meeting, participants hashed out many of the issues we've talked about in this series, like AI sycophancy, bots failing to escalate serious issues, and data privacy. Still, though, the FDA is allowing some experimentation in small ways. Last month, it announced the participants in a new pilot program for medical devices. One of them is called Unpacked from a company called Limbic. It's an AI tool for people with depression or anxiety.

[00:16:36] Patients get calls from an AI voice agent, and it's overseen by clinicians. If the trial goes well, Limbic says it would be allowed to make its product widely available on the market. So, the jury's still out on federal rules for these AI mental health ChatBots. But the companies that make them aren't waiting around. They want their apps to be woven deeper into their users' lives. Talkspace and Slingshot are both planning to find ways for users to send their sessions from

[00:17:05] the AI chatbot to their human therapists so the counselors can see what their patients were talking about between sessions. Because, Talkspace's John Cohen says, patients are doing this anyway. We hear it from psychiatrists all the time. They say, I see a patient on Tuesday. And they come into the office on Friday and say, oh, by the way, I had an issue on Thursday and I discussed it with X. Meanwhile, Slingshot wants Ash to feel even more real to its users. Like, by giving Ash an avatar.

[00:17:34] You can imagine a world in which you're going to FaceTime with Ash. And then you have interesting questions of, is it a human? Is it a character? Is it more like an animated character? Parikh says that could add a layer of communication that users currently aren't getting with voice and text. Body language could be an important part of helping a person see, like, okay, what is Ash thinking? Like, is Ash pausing or ruminating or whatever?

[00:17:59] And while Talkspace might have built a conversational AI chatbot, CEO John Cohen still says AI isn't therapy. And he's not sure AI tools will ever take its place. Therapy is not just a Q&A. You're going to ask Google or ChatGPT or whatever. You're going to ask them a question and you get an answer. You know, how far is the sun? How fast is the speed of light? Right? Question, answer. Therapy is not that way.

[00:18:26] There's a significant nuance to therapy that is not just a Q&A. That's why it's so complicated. Today's show was produced and mixed by Pierre Bien-Aimé with supervising producer Tali Arbel. Additional editorial support from Chris Zinsley. I'm Alex Osalev for The Wall Street Journal. We'll be back with our regular show on Monday morning. Thanks for listening.