The AI Computing Power Gold Rush

[00:00:03] Welcome to Tech News Briefing. It's Monday, October 12th. I'm Danny Lewis for The Wall Street Journal. In the AI industry, computing power is king. Several years into the AI boom, the chips and data centers required to power these tools are fueling a modern gold rush. Here now to discuss how this is reshaping relationships across Silicon Valley is WSJ reporter Kate Clark. Kate, AI companies have been scrambling for more compute for years now. Why is access to it still so tight?

[00:00:32] In short, it's because the infrastructure just can't be built fast enough. I mean, there are data center projects across the globe in progress, but these companies, whether it's a small startup or a huge lab like OpenAI or Anthropic, they need a ton of computing power to power their products like ChatGPT. And what does the struggle to get enough computing resources mean, practically speaking, for these companies as they're developing more and more powerful AI models?

[00:00:57] It really varies. And I think what we learned in reporting this story is for the smaller companies, which, you know, there are thousands of AI startups in Silicon Valley, they are really challenged because the large hyperscalers who are providing a lot of this compute, they don't really see any need to work with these small companies because these small companies are not signing multi-year contracts. They only need a little bit of compute and they don't actually even know how long they need it for. And at the same time, we all know that Anthropic and OpenAI basically can't get enough.

[00:01:24] I mean, Anthropic this year dealt with some outages after the overnight success of Cloud Code. And since then, they've signed hundreds of billions of dollars of computing contracts to make sure that they do have that capacity as they keep growing. So how does that affect the types of projects and products these companies can work on? It's everything. I mean, if you don't have compute, you can't build anything. If this is a generative AI company, if you're building a large language model, if you are an AI lab, you absolutely need computing power. There's just no other option for you.

[00:01:53] A lot of people told us it's more important than capital. Of course, you need the money to buy the compute. The money is actually easier to get because the venture capital industry right now is very frothy. And investors are really excited about AI startups, and they are kind of throwing money at them. So the capital isn't the challenge, but finding the compute, finding enough is really what is keeping companies that are able to get it. It's keeping them alive. It's keeping them ahead in this race. You mentioned VC firms. Like, how are they playing into this whole moment, too?

[00:02:21] They're increasingly realizing that capital is a commodity in this environment, and compute is incredibly important. Some firms, and it seems like more and more all the time, are playing chip broker, meaning the venture capitalist is developing relationships at neoclouds, at hyperscalers, networking, making sure they know the person who has control of the GPUs so that they can actually make calls and connect to their companies. Some investors have realized that they can actually help facilitate this process, and some have even gone as far as setting up their own cluster.

[00:02:51] Y Combinator, which is the biggest Silicon Valley startup accelerator, they have done this. And from what I've been told, it's been very successful, meaning the companies are clamoring for it, and the compute is fully utilized. And I think that we are going to see a lot more projects like that in the next couple of years. With this compute crunch and many new data center projects facing local opposition, how are the large AI companies adapting to this? I think it's a big question mark. I mean, there's a lot of regulation. There's a lot of political backlash. It is slowing down a lot of data center projects.

[00:03:22] And this is why you see some companies actually overcommit to computing contracts, meaning they're buying more than they need because they know that a lot of these are not going to come online. Or they're not going to come online in the time frame that is currently being promised. It is interesting and I think a little bit concerning just that a lot of companies are choosing to overcommit because there may come a time in which those data centers are online and there isn't as much appetite then to resell it or rent it out.

[00:03:46] Is competition the only way that the large companies are approaching this, or are they collaborating in any way to share resources? They are collaborating. I think that's also been somewhat of a surprise to us. As we reported in the story, Anthropic purchased compute from SpaceX, which had an excess of compute. And that involved Dario Amode, the CEO of Anthropic, personally meeting with Elon Musk and Elon then meeting with some others at Anthropic and kind of understanding more about the company, more about what they needed.

[00:04:13] And so you saw an alliance form that was earlier this year and Anthropic signed a massive, massive, massive deal to get compute from SpaceX. We also report that Dario has spoken on the phone with meta executives about compute and that did not result in a deal yet. But if some companies are overcommitted and actually have an excess of capacity, you will see more of these alliances form over time. What do you make of the AI build out? If you're a listener on Spotify, leave us a comment with your thoughts.

[00:04:43] Coming up, compute is like gold for AI companies and people are traveling the world to find it. That's after the break. Welcome back. We've been talking about the strategies AI companies are using to secure access to computing power amid ever-growing demand.

[00:05:10] But Kate, if this is a gold rush and data centers are gold, where are they going? There are some companies who are traveling the globe looking for new data centers. They're vetting those data centers, trying to get that capacity online and then renting it out to customers, big and small. We spoke to an executive at a company called San Francisco Compute that has visited many continents this year. He went to vet a data center and found that it was actually just a converted chicken coop. He said there was like a single armed guard and a chain link fence.

[00:05:40] And you need to have proper security for these kinds of things. You also need proper fire protection and cooling methods. These are like very complicated technical setups. So that did not meet their bar. He also visited one of the Philippines where he ended up stuck in a monsoon flood and had to escape his vehicle and wade to safety, where he then waded at someone's home and flagged down a truck to take him to the airport.

[00:06:01] He did say that many of those projects were not up to snuff, meaning like these were not going to be data centers which you could reliably be renting capacity to AI companies. So he has to effectively turn them down. But at the same time, they found many and have been able to get that capacity online and have serviced companies as large as NVIDIA to the little guys in Silicon Valley. It really paints a picture of where these data centers are, like the level of quality that there is, which I think has surprised me.

[00:06:27] And it, again, shows just how desperate and global this race for power is right now. How is this moment shaping the AI industry's future, both for these large giants like, you know, your Googles, your Open AIs, your Anthropics, but also these small startups? The concern is that the largest labs really will just continue to win and there won't be as much opportunity to compete, which is really important in Silicon Valley and beyond. And how much compute you're able to have is really determining who's winning right now.

[00:06:55] Meta, for example, owns and operates their own data centers. That gives them an advantage if they're competing over a smaller startup. We gave an example in the story of a very viral new AI assistant called Instinct, which is a small startup, has raised a lot of money, but is now competing with Meta's also viral AI assistant, Muse. Meta has a lot of compute resources. Instinct is raising money to then buy compute, of course, but it's very clear who has an advantage in that situation. So you wonder how in situations like that startups can remain competitive.

[00:07:24] And I think that is a big question and a big concern across the startup world right now. So it sounds like not all data centers are created equal. What does a big one offer over a couple of small server racks? I think it really depends. The issue with like a converted chicken coop or something is it can burn to the ground. It may not have proper cooling. That may not be there long term. And so there is like different levels of quality. And I think we're learning in real time what that can look like. But working with a hyperscaler is more reliable. Many new AI chips are coming to market.

[00:07:53] OpenAI is developing their own AI chip. The field is rapidly changing and there will be other options. And I think we're going to start to see the entire landscape shift, which will be really, really interesting. But I think it's fair to say like most people don't want to be relying on a chicken coop. They want to imagine a very nice fancy data center that is funded by Amazon or any of these much larger companies who are going to be sticking around a long time. That was WSJ reporter Kate Clark. And that's it for Tech News Briefing.

[00:08:23] If you are a listener on Spotify, be sure to leave us a comment. Today's show was produced by me with supervising producer Katie Ferguson. I'm Danny Lewis for The Wall Street Journal. We'll be back tomorrow with a new episode. Thanks for listening.