Your right to repair AI systems | Rumman Chowdhury

[00:00:00] TED Audio Collective At this point in time, we are deeply intertwined with AI. Our data, online behaviors, images and language are all utilized to inform, train and enhance this technology. In turn, it's transforming numerous critical aspects of our lives, and it can be pretty controversial.

[00:00:36] Consider for example when users discovered that the images and videos we share on platforms like Facebook and Instagram are being used by Meta to train its various AI tools. The only way to opt out? Submit a form in the US, opt out if you're in Europe,

[00:00:52] or deactivate your account entirely. The course of technologists taking without permission, let alone disregarding user feedback, has shaped a world where most people are leery about AI. Sometimes it feels like we're quickly moving into a permissionless, wild, wild west of tech.

[00:01:12] But instead, imagine a future where the relationship between users and creators is not one-sided but a continuous feedback loop. Could this be the key to shaping the future of AI? This is TED Tech, a podcast from the TED Audio Collective. I'm your host, Sherelle Dorsey.

[00:01:33] Today, we'll learn from data scientist and Humane Intelligence CEO Dr. Ruman Chowdhury about how to fully realize the potential of an ethical AI. In her vision, we must intentionally team together across disciplines to inform the kind of AI we want to see in the world.

[00:01:51] But before we dive into Dr. Chowdhury's recommendations, a quick break to hear from our sponsors. Imagine this, in 2030, the CFO of a Fortune 100 company is a bot. I'm Paul Michelman, and on Imagine This, we'll be exploring possible futures and the implications they hold for organizations.

[00:02:15] Joining me will be BCG's top experts, as well as my co-host, Jean, BCG's conversational Gen AI agent. Blending human creativity with AI innovation, this podcast promises an unmatched listening journey. Join us on Imagine This from BCG. And now, Dr. Ruman Chowdhury takes the TED stage.

[00:02:39] I want to tell you a story about artificial intelligence and farmers. Now, what a strange combination, right? Two topics could not sound more different from each other. But did you know that modern farming actually involves a lot of technology? So computer vision is used to predict crop yields,

[00:03:01] and artificial intelligence is used to find, identify, and get rid of insects. Predictive analytics helps figure out extreme weather conditions like drought or hurricanes. But this technology is also alienating to farmers, and this all came to a head in 2017 with the tractor company John Deere,

[00:03:24] when they introduced smart tractors. So before then, if a farmer's tractor broke, they could just repair it themselves or take it to a mechanic. Well, the company actually made it illegal for farmers to fix their own equipment. You had to use a licensed technician,

[00:03:42] and farmers would have to wait for weeks while their crops rot and pests took over. So they took matters into their own hands. Some of them learned to program, and they worked with hackers to create patches to repair their own systems.

[00:03:58] In 2022, at one of the largest hacker conferences in the world, DEF CON, a hacker named Sick Codes and his team showed everybody how to break into a John Deere tractor, showing that first of all, the technology was vulnerable,

[00:04:13] but also that you can and should own your own equipment. To be clear, this is illegal. But there are people trying to change that. Now, that movement is called the right to repair. The right to repair goes something like this. If you own a piece of technology,

[00:04:32] it could be a tractor, a smart toothbrush, a washing machine, you should have the right to repair it if it breaks. So why am I telling you this story? The right to repair needs to extend to artificial intelligence. Now, it seems like every week

[00:04:51] there's a new and mind-blowing innovation in AI, but did you know that public confidence is actually declining? A recent Pew poll showed that more Americans are concerned than they are excited about the technology. This is echoed throughout the world. The World Risk Poll shows that respondents

[00:05:11] from Central and South America and Africa all said that they felt AI would lead to more harm than good for their people. As a social scientist and an AI developer, this frustrates me. I'm a tech optimist because I truly believe this technology can lead to good.

[00:05:31] So what's the disconnect? Well, I've talked to hundreds of people over the last few years, architects and scientists, journalists and photographers, rideshare drivers and doctors, and they all say the same thing. People feel like an afterthought. They all know that their data is harvested, without their permission,

[00:05:56] to create these sophisticated systems. They know that these systems are determining their life opportunities. They also know that nobody ever bothered to ask them how the system should be built, and they certainly have no idea where to go if something goes wrong. We may not own AI systems,

[00:06:17] but they are slowly dominating our lives. We need a better feedback loop between the people who are making these systems and the people who are best determined to tell us how these AI systems should interact in their world. One step towards this is a process called red teaming.

[00:06:36] Now, red teaming is a practice that was started in the military, and it's used in cybersecurity. In a traditional red teaming exercise, external experts are brought in to break into a system, sort of like what SICK codes did with tractors, but legal.

[00:06:52] So red teaming acts as a way of testing your defenses, and when you can figure out where something will go wrong, you can figure out how to fix it. But when AI systems go rogue, it's more than just a hacker breaking in.

[00:07:08] The model could malfunction or misrepresent reality. So for example, not too long ago, we saw an AI system attempting diversity by showing historically inaccurate photos. Anybody with a basic understanding of Western history could have told you that neither the founding fathers nor Nazi-era soldiers would have been black.

[00:07:28] In that case, who qualifies as an expert? You. I'm working with thousands of people all around the world on large and small red teaming exercises, and through them, we have found and fixed mistakes in AI models. We also work with some of the biggest tech companies

[00:07:47] in the world, OpenAI, Meta, Anthropic, Google, and through this, we've made models work better for more people. Here's a bit of what we've learned. We partnered with the Royal Society in London to do a scientific misin-disinformation event with disease scientists. What these scientists found is that AI models

[00:08:09] actually had a lot of protections against COVID misinformation, but for other diseases like measles, mumps, and the flu, the same protections didn't apply. We reported these changes, they're fixed, and now we are all better protected against scientific mis-and-disinformation. We did a really similar exercise with architects

[00:08:30] at Autodesk University, and we asked them a simple question. Will AI put them out of a job? Or more specifically, could they imagine a modern AI system that would be able to design the specs of a modern art museum? The answer, resoundingly, was no. Here's why.

[00:08:50] Architects do more than just draw buildings. They have to understand physics and material science. They have to know building codes, and they have to do that while making something that evokes emotion. What the architects wanted was an AI system that interacted with them, that would give them feedback,

[00:09:08] maybe proactively offer design recommendations, and today's AI systems, not quite there yet. But those are technical problems. People building AI are incredibly smart, and maybe they could solve all that in a few years. But that wasn't their biggest concern. Their biggest concern was trust. Now, architects are liable

[00:09:28] if something goes wrong with their buildings. They could lose their license, they could be fined, they could even go to prison. And failures can happen in a million different ways. For example, exit doors that open the wrong way leading to people being crushed in an evacuation crisis.

[00:09:46] Or broken glass raining down onto pedestrians in the street because the wind blows too hard and shatters windows. So why would an architect trust an AI system with their job, with their literal freedom, if they couldn't go in and fix a mistake if they found it?

[00:10:05] So we need to figure out these problems today, and I'll tell you why. The next wave of artificial intelligence systems called agentic AI is a true tipping point between whether or not we retain human agency or whether or not AI systems make our decisions for us.

[00:10:24] Imagine an AI agent is kind of like a personal assistant. So, for example, a medical agent might determine whether or not your family needs doctor's appointments, it might refill prescription medications, or in case of an emergency, send medical records to the hospital.

[00:10:40] But AI agents can't and won't exist unless we have a true right to repair. What parent would trust their child's health to an AI system unless you could run some basic diagnostics? What professional would trust an AI system with job decisions

[00:10:57] unless you could retrain it the way you might a junior employee? Now, a right to repair might look something like this. You could have a diagnostics board where you run basic tests that you've designed, and if something's wrong, you could report it to the company

[00:11:13] and hear back when it's fixed. Or you could work with third parties like ethical hackers who make patches for systems like we do today. You can download them and use them to improve your system the way you want it to be improved.

[00:11:25] Or you could be like these intrepid farmers and learn to program and fine-tune your own systems. We won't achieve the promised benefits of artificial intelligence unless we figure out how to bring people into the development process. I've dedicated my career to responsible AI,

[00:11:46] and in that field, we ask the question, what can companies build to ensure that people trust AI? Now, through these Red Teaming exercises and by talking to you, I've come to realize that we've been asking the wrong question all along.

[00:12:05] What we should have been asking is what tools can we build so people can make AI beneficial for them? Technologists can't do it alone. We can only do it with you. Thank you. That was Dr. Rumin Chowdhury at TED 2024. And that's it for today.

[00:12:34] TED Tech is part of the TED Audio Collective. This episode was produced by Nita Byrd-Lawrence, edited by Alejandra Salazar, and fact-checked by Julia Dickerson. Special thanks to Maria Ladius, Farah DeGrunge, Daniela Bellarezzo, and Roxanne Hailech. I'm Sherelle Dorsey. Thanks for listening in.