Inside the New York City Council’s Marathon AI Hearing

[00:00:03] Welcome to Tech News Briefing. It's October 7th. I'm Danny Lewis for The Wall Street Journal. For months, the drumbeat of worries about the rapid advancement of AI technology has steadily increased, especially since former anthropic researcher Jacob Coxson publicly left the clawed maker over safety concerns amid news stories about AI agents hacking other companies on their own. The AI's had developed goals no one intended and carried out what would have been a federal felony if a human had done it.

[00:00:33] Coxson spoke this week during a marathon hearing held by the New York City Council as lawmakers are considering a spate of bills to rein in the technology. The hearing lasted all day as representatives from major companies like OpenAI, Anthropic, Meta and Google were grilled about their approaches to the risk posed by AI. Wall Street Journal reporter Kevin Dugan was there and he joins us now. And before we get going, we should note that News Corp, owner of The Wall Street Journal, has a content licensing partnership with OpenAI.

[00:01:02] Okay, Kevin, let's talk about this hearing now. What was the purpose? The purpose of this hearing was really to get a handle on how AI can affect New York City. Remember, even though it's not San Francisco, New York is a major tech hub. There are about 400,000 tech jobs here in the city and the city is courting companies like OpenAI and Anthropic who have both opened up offices here. They're trying to bring in more small tech startups.

[00:01:29] So really, this was a way for lawmakers to get a sense of what are the dangers for New Yorkers? What are the dangers for New York companies? And what are the dangers for New York's economy? So granted, this hearing was, you know, more or less the length of a work day. What was the vibe like? This was a packed hearing.

[00:01:49] And that really speaks to, like, how much people are paying attention to the possible dangers of AI in New York and how high stakes this really is. The overall vibe was there was a lot of skepticism that whatever benefits AI can give are good enough to outweigh the problems that AI could also wreak. And this could be through weapons, hacking swarms, things like that.

[00:02:17] Yeah, as we mentioned, former Anthropic researcher Jacob Coxson was one of the big names speaking out of this hearing. Coxson told the city council that he thinks the large AI firms are being reckless. And one reason is their company culture. The companies run on a startup mindset. Move fast, break things, fix them later. That works for a photo sharing app. It does not work for building the most powerful technology ever built.

[00:02:44] A few other researchers who used to work for various big AI companies also testified. Who are they and what did we learn from them? Right. So one of them was Alex Turner. He is a former alignment researcher for Google DeepMind. If the hugging face hacking swarm had been significantly more intelligent, but similarly misbehaved and misaligned, it might have caused billions of dollars of damage or even cost lives. And the other is Daniel Kakatailo.

[00:03:12] He was also a researcher for OpenAI. Our ability to even notice misalignment problems is already quite poor and is set to get much worse in the near future.

[00:03:26] Although they have different views about the dangers that AI can pose on the general public, they both quit their jobs in recent years because they were warning that the executives at their respective companies were not taking these risks seriously. How about the representatives of the companies themselves? What did they have to say?

[00:03:46] The responses that they gave were that they were taking safety very seriously, that the worst case scenarios, the extinction level events, this is not an acceptable risk for them. However, they did not really want to go too much into how much of a risk that they're viewing it. Really, they wanted to assure lawmakers that they were taking these problems seriously and that there are still lots of benefits to be had for regular people and for the economy here.

[00:04:15] And speaking to that, I mean, this hearing happened just a few days after OpenAI Chief Executive Sam Altman told Politico that, quote, the world should accept some bad things happening for the benefits of this technology. City Council Speaker Julie Menon brought this up during the hearing. No one is questioning that the incredible medical and scientific breakthroughs and innovation that AI has brought to this world.

[00:04:44] But why should the public accept that bad things should be happening? And why should AI companies get to decide what level of risk society must bear? What was the mood in the room like when she said that? The representatives from the companies did not seem thrilled to be getting this question. And especially the representative from OpenAI, who is their head of policy.

[00:05:08] We think AI has tremendous potential benefits to help solve some of humanity's hardest problems. But no technology is without risks. And there are a wide spectrum of risks. Clearly, some risks are unacceptable. But we also think it's important to get technology into the hands of as many people as possible so that they can benefit from it.

[00:05:32] You can imagine that it's an uncomfortable position to be asked about your boss's comments when you are supposed to be talking to the general public. We're just told by your boss that they have to accept some level of harm. The way that the companies responded was that no level of catastrophic risk is acceptable.

[00:05:54] But economic disruption, things like that, is something where, in their view, while there may be some short-term problems, the ultimate benefits for the economy are going to be so much bigger. So this was, it was a harder question for them. Would you like to see the AI industry regulated by the government? If you're a listener on Spotify, sound off in the comments.

[00:06:19] Coming up, if the New York City Council attempts to regulate AI, what might that look like? And could more cities follow? That's after the break. Welcome back. As we've been talking about, the New York City Council has been considering bills to regulate AI companies.

[00:06:47] But, Kevin, as important as New York City is to the country's economy, this is still just one city in one state that's reviewing blazingly fast developing tech made by giant international firms. Can the City Council actually influence these companies? Yes, but to what extent? I think that's the question. So right now, New York has a lot of capabilities to make companies do what they want. People want to start their companies here.

[00:07:16] And that's been true for a long time. And that hasn't gone away. The city has the ability to create and enforce licenses for different companies and can use regulation to withhold or threaten those licenses, which would allow them to operate here. The city can also regulate how whistleblowers are treated here.

[00:07:38] What they're trying to do is implement a kind of kill switch in case there is some sort of doomsday scenario that AI brings about. Will all of these pass? I don't know. And whether they're effective, I think that's a bigger question. But right now, in the absence of a larger federal movement to regulate, New York wants to be a leader for other cities and other states to put in their own regulation.

[00:08:05] And this would create a kind of nightmare scenario for large companies because you have a lot of patchwork different rules. It does feel kind of odd to be talking about regulating this massive industry at the local level. Could these efforts put pressure on the federal government to make moves to rein in AI? We have an election in about a month. And pretty soon, if Democrats come in, they're going to be more likely to want to regulate these companies.

[00:08:30] Republicans are much more open to letting these companies regulate themselves. So from here on out, other cities and other states, they're going to be looking to New York. They're going to be seeing what works. They're going to be seeing what doesn't. I don't know if the problems around AI regulation are going to be fixed at the federal level because a lot of people talk about this AI arms race between the U.S.

[00:08:55] and countries like China, where we don't have an incentive at the highest level of government right now to slow down because there's worries that they could beat us in some kind of way. Although that has never really been made clear what winning actually looks like. There could be ultimately some international cooperation down the line, but that is really much more of a long-term scenario.

[00:09:19] You mentioned having a patchwork of regulations would be a nightmare for AI companies, but we do have a precedent there in how regulations came for the social media industry. Is there anything to learn from that? One of the lessons that lawmakers have picked up from the very long tale of the social media regulatory questions, all the hearings, all the lawsuits, is that there is a reflexive distrust of a lot of tech companies.

[00:09:49] Whether that's fair or not, that is a sea change in the way that lawmakers have approached these companies in the past. So it is possible that you could see stricter regulation right off the bat. You could see things move much more quickly, right? We could be talking about years instead of a decade plus. And that is really the lessons that lawmakers have taken who I've spoken with. They don't want to wait.

[00:10:14] They don't want to see how this is going to play out because for them, when they hear about the potential for catastrophe, for human extinction, for weapons-grade drones to be used on a civilian population, things like that, they're not going to sit around and wait to find out. This is an Instagram for them. This is much bigger. That was Wall Street Journal reporter Kevin Dugan. And that's it for Tech News Briefing. If you're a listener on Spotify, be sure to leave us a comment.

[00:10:42] Today's show was produced by me, with supervising producer Katie Ferguson. I'm Danny Lewis for The Wall Street Journal. We'll be back tomorrow with a new episode. Thanks for listening.