The AI industry’s investors and critics don’t agree on much. But many in each camp share at least one basic conviction: America’s top labs are about to make a killing. 

Capital markets have signaled their faith in Anthropic and OpenAI’s impending hyper-profitability, valuing each at nearly $1 trillion. Many of Silicon Valley’s progressive adversaries also expect the labs to grow filthy rich but fear the implications, warning that AI-induced automation could transfer vast sums of money from ordinary workers to a handful of giant tech companies. Sen. Bernie Sanders’s call for nationalizing the top AI labs rests partly on that concern. 

But recent advances in Chinese AI call all of this into question.

Over the past two months, Chinese companies have released three AI models that are nearly as powerful as America’s frontier systems — and radically less expensive. 

In June, Beijing’s Z.ai debuted a model that performed nearly as well as Claude and ChatGPT’s second-tier systems on independent benchmarks. Weeks later, another Chinese firm, Moonshot, unveiled “Kimi K3,” a model that allegedly outperforms all of its American rivals except for the very latest versions of Claude and ChatGPT. Finally, just days ago, Alibaba launched a preview of Qwen3.8 Max, which purportedly outclasses even OpenAI’s most advanced systems, while trailing only Claude’s Fable in its capabilities. (Disclosure: Vox Media is one of several publishers that have signed partnership agreements with OpenAI. Our reporting remains editorially independent.)

These developments don’t merely threaten America’s AI giants with stiffer competition in the race for superintelligence. Rather, they raise a more harrowing prospect: that the AI race’s ultimate rewards will be far smaller than anticipated. In a world where new advances can regularly be leapfrogged by cheaper upstarts, hoarding the technology — and its profits — will be harder for any one company to do.

In other words, building a machine God might not be as lucrative as it’s cracked up to be. AI, it turns out, may “want to be free.”

How AI was supposed to pay off

To see how China’s new models threaten Anthropic’s profit expectations, we must first examine why those expectations have been so high.

This is not entirely self-evident. After all, AI labs aren’t much like the hyper-profitable tech giants of the 2010s. Facebook and Airbrb were relatively capital-light businesses with ultra-low marginal costs (adding a profile to Facebook or listing to Airbnb costs the companies virtually nothing). And once each gained a foothold in their respective markets, network effects enabled them to retain formidable positions without needing to constantly upgrade their products.

Building a state-of-the-art AI company is a much more involved — and astronomically more expensive — endeavor. To get to the frontier, Anthropic and OpenAI have sunk (at least) tens of billions into semiconductors, data centers, power plants, and other capital investments. Staying at the cutting-edge, meanwhile, compels them to perpetually churn out evermore costly models.

To put a new Claude model through its initial training — in which it spends months digesting the internet and sussing out statistical patterns within its text — can now cost hundreds of millions of dollars. And such foundational computation is only the beginning. A truly superlative model requires several additional months of fine-tuning. Armies of contracted experts — such as computer scientists, physicians, and mathematicians — tutor the models, grading their answers and guiding them towards better ones. Then the AI systems complete millions of rounds of practice, in which they learn through trial and error how to solve countless problems. This arduous process, known as “post-training,” compounds the costs of a single model’s development. 

All of which raises the question: Why would investors expect businesses with a cost-structure this challenging to be not merely profitable, but massively so?

There are (at least) two answers. The first (and most obvious) is that the market for superintelligent machines is liable to be vast. Frontier AI systems promise to reduce costs and improve performance in myriad white-collar sectors. And Anthropic’s soaring revenues indicate that firms do, in fact, find Claude useful. A company like AirBnB has earned billions by revolutionizing a single industry; imagine then what a technology that remade virtually all industries might be worth.

Of course, plenty of technologies are valuable but not massively profitable to produce. After all, in well-functioning markets, competition should eventually erode individual firms’ margins, even if the underlying technology continues generating huge value. 

But this is where the second answer comes in: Frontier labs’ immense costs are a burden, but they’re also a safeguard against competition — or, in industry parlance, a “moat.”

Startups may be able to afford to build or acquire more rudimentary models, many of which are “open source.” But, the thinking goes, they won’t be able to deliver Claude Fable-level performance without raising giant amounts of capital. And what investors will be willing to pour hundreds of billions into an AI pipsqueak that’s light-years behind Google, Anthropic, and OpenAI?

Alas, the Chinese AI labs’ rapid progress — and the way it was achieved — suggest that Anthropic’s moat may be shallower than previously thought.

How Moonshot swam Anthropic’s moat

The existence of powerful, Chinese AI systems is neither new nor surprising. Xi Jinping’s government has made vying for global AI dominance a key economic goal. And China’s DeepSeek, which also has stunned US companies with its lower-cost competitive models, surpassed ChatGPT as the most-downloaded free iPhone app more than a year ago.

The latest models, however, have dramatically narrowed the gap in capabilities between frontier American systems and their Chinese rivals. Just as critically, they’ve done so in a manner that other, relatively underfunded AI upstarts might be able to emulate.

Alibaba and Moonshot needed to invest massive resources to train their base models. But they allegedly found a low-cost way to refine those models into near-frontier systems: Just ask Claude.

Or, more specifically: Engage Claude in 16 million conversations, using 24,000 fake accounts. In each of those exchanges, ask the model to not only answer countless difficult questions but also, walk you through its reasoning, step by step. Then take all of this data and feed it into your own model as study material, training it to respond to the world’s most challenging queries as Claude would. 

Through this process — known as “distillation” — an AI lab can replicate virtually all of a frontier model’s capacities, without sinking vast sums into human experts and post-training computing runs. 

China’s AI labs have not admitted to using distillation. But OpenAI and Anthropic both reportedly uncovered Chinese distillation attempts earlier this year. And some of the new models appear to display tell-tale signs of distillation in conversations with ordinary users; Kimi K3 has routinely identified itself as “Claude.”

Chinese AI companies are hardly alone in using distillation to catch up with frontier labs. Earlier this year, Elon Musk admitted in court that xAI enhanced Grok’s capabilities by running distillation techniques on Claude and ChatGPT. Nonetheless, China’s latest models appear to demonstrate that distillation can help take a second-tier model to the frontier’s threshold.

America’s frontier labs have tried to defend themselves against such imitators. But this is technically difficult when distillers can assemble massive networks of bots, each asking an inconspicuous number of questions. And legally, it is difficult for America’s AI giants to argue that distillers are stealing their intellectual property. After all, in a sense, China’s copycats are merely doing to Anthropic and OpenAI what those companies did to journalists, coders, lawyers and other specialists: Feeding their public-facing outputs into a model, which then replicates their capabilities by discerning underlying patterns within the text.

Oh, and China’s giving these models away

The new Chinese models would have caused Silicon Valley enough headaches, if they merely provided stiffer competition, while demonstrating the power of distillation. 

What makes Kimi K3 and Qwen3.8 Max especially threatening to the American AI giants’ profitmaking potential, however, is that they are officially open source — meaning that the models’ parameters can be downloaded for free. (Alibaba and Moonshot have not yet released these parameters, but they say they will shortly.)

In other words, any company or hobbyist with enough computing power will soon be able to run a near-frontier Chinese model on their own hardware, modify that model to better serve a specialized purpose, and then sell access to their new version — without paying Alibiba a single yuan.

As Kimi and Qwen grow more capable, their market-share is likely to grow, at American AI giants’ expense.

For many of Anthropic and OpenAI’s potential customers, that proposition may be hard to turn down. Most businesses don’t need the world’s smartest AI, just one competent at their enterprise’s core tasks — compiling legal research, answering IT queries, writing working code, etc. A model that produces outputs 90 percent as good as Claude’s — at roughly one-sixth of the cost — will sound pretty good to many corporations.

Further, open source models aren’t just cheaper than frontier systems, but potentially more secure. If you run an AI on your firm’s own servers, then you don’t need to entrust sensitive data to Anthropic, Google, or OpenAI.

All this had led much of corporate America to embrace open-source models, even before the latest versions narrowed the capabilities gap. In a Linux Foundation survey, 63 percent of organizations reported using open-source AI systems.

And increasingly, those models are Chinese. According to Sequoia Capital, one of Silicon Valley’s premier venture capitalist firms, a majority of American AI startups now use open-source Chinese systems. As Kimi and Qwen grow more capable, their market-share is likely to grow, at American AI giants’ expense.

What’s bad for OpenAI is good (and/or catastrophic) for humanity

All this said, it is still entirely possible that OpenAI and Anthropic will justify their colossal valuations. In many highly competitive economic domains, having access to the world’s very best AI model will remain highly valuable. And America’s frontier labs still outperform all their peers. 

But it’s increasingly plausible that selling state-of-the-art AI systems will prove to be a low-margin undertaking. In a world of ubiquitous, near-frontier open source models, the AI sector’s big winners probably won’t be its top labs, but rather, its chipmakers and cloud computing providers. 

For ordinary people, a future where superintelligence is dirt cheap — and rival AI companies are constantly rising and falling, rather than consolidating into mega-corporations — would look somewhat different than the cyberpunk dystopia that the left’s been dreading. 

And not entirely in a good way. For one thing, in that reality, mitigating AI’s biggest risks would be immensely difficult. Having a handful of firms monopolize control over frontier AI systems is bad in many respects. But it does make those models easier to regulate, as the Trump administration’s decision to temporarily block Claude’s Fable in the name of cybersecurity demonstrated. 

By contrast, if recipes for ultra-powerful AI models are published all over the internet — and anyone with modest technical skills can modify them at will — then systems willing to help their users hack government bureaucracies or engineer bio-weapons are liable to proliferate.

From another angle, however, the “AI becomes almost free” scenario may look like capitalism at its finest: Retrospectively, such a development would mean that a small number of extremely rich people bankrolled the creation of an immensely useful technology, under the expectation of massive profits, only to see competition erode their returns — and disperse that tech’s benefits across a wider group of businesses and consumers. 

Granted, in the case of AI, this process might also generate a super-virus that kills us all. But hey, no system is perfect.

AI was supposed to make our lives better. Instead, it’s made many of us scared and angry. Communities are protesting against the building of new data centers — the warehouses of IT equipment powering the AI buildout — across the country, and increasingly they’re winning. And polling shows most Americans think AI is moving too fast.

So how did public opinion on AI curdle so quickly? Jasmine Sun, who reports on the industry from San Francisco, argues that the backlash treats AI less as a technology and more as a political project. “The debate was not about like, is ChatGPT useful to me?” Sun told me during a taping of Vox’s The Gray Area. “The debate was actually something more like, there are these big corporations and unaccountable billionaires…coming into my city, coming into my life and changing it without having any sort of democratic input?”

Filling in for Sean Illing, I talked to Sun about the rise of “AI populism,” the parallels with the Industrial Revolution, and how the backlash could crash into the 2028 presidential election. 

As always, there’s much more in the full podcast, which drops every Monday, so listen to and follow us on Apple Podcasts, Spotify, Pandora, or wherever you find podcasts.

You’ve been writing about a phenomenon you call AI populism. How would you define that? What is AI populism?

I define AI populism as a worldview where AI is not seen as an ordinary technology, but specifically as an elite political project to be resisted. I came to the term while thinking about the AI backlash and the reasons people are increasingly anti-AI — whether that’s LLM slop, whether that’s Waymos in their city, whether that’s a new data center project. One thing that occurred to me was that a lot of times the debate wasn’t about whether ChatGPT is useful to me, or whether Waymos are safer than a human driver. The debate was actually something more like: There are these big corporations and unaccountable billionaires who are coming into my city, coming into my life, and changing it without any democratic input.

When I talk to people who are opposing AI in various ways, they seem more concerned with this concentration-of-power, anti-elite dimension — which is where I take the word “populism” — rather than classic AI safety concerns, which are more about the technical characteristics that might introduce risk.

You wrote a piece that touched on some of this but went to a darker place — “AI populism’s warning shots” — and you wrote about actual shots. Sam Altman, the CEO of OpenAI, was targeted by a Molotov cocktail and a shooting within the span of a couple of days. There was an Indiana councilman who voted for a data center and woke up to gunshots at his home and a note reading “no data centers.” Why do you think of those incidents of violence as warning shots of something to come?

It was pretty scary. I’m no Sam Altman fanboy, but it’s terrifying that assassination attempts are showing up in response to people’s worries about AI. One factor is that we’ve been seeing a rising wave of political violence and support for political violence in the US, especially among young people, over the past few years — the UnitedHealthcare CEO shooting, the Charlie Kirk shooting. Increasingly, a lot of disaffected, maybe nihilistic young people are turning toward political violence as a way to express political beliefs they don’t feel they have other channels for. Or maybe that person is just unwell. But I do expect to see more of it, because my theory of political discontent is that if people feel they have institutional channels to bargain for their rights — if they feel the democratic process is working, or they’re part of a union and believe their union leader will go bargain about how automation shows up in the workplace — they’ll most likely go through those channels.

When it feels like the official channels aren’t working, opposition becomes much more diffuse and volatile. That’s part of why, in creative communities, you’ll see people witch-hunting each other over AI use. I think we’ll see more political violence against people seen as AI leaders, or as supporting AI leaders.

That’s really scary.

Yeah, I’m quite worried about it. But again, my sense is that it comes from a feeling of — what else is there to be done, when you have this level of concentration of wealth and power, and there’s no democratic input right now into how AI is regulated or built?

It’s like a jump straight from complaining at your community meeting about the data center to an act of violence.

I was talking to some friends about this. During the 20th century in the US, there was a wave of factory mechanization and automation, but unions were really strong — often when a company said, “We’re going to bring in these machines,” they’d sit down with the factory union leader and say, “Okay, you can bring in the machines, but we’re going to couple that with a wage increase,” or a 35-hour workweek, or earlier retirement. There was a channel to make a deal about how automation would show up in your workplace. That meant people were more likely to accept it as something lifting all boats. I don’t think that’s happening now — most of the industries affected by AI aren’t organized in labor unions, and the democratic channels are questionable at best.

It’s like when people have agency to be part of the transition, the process goes a lot smoother. Is there a historical analogy for a technological change that didn’t allow for input from the people involved? I’m thinking of the Luddites.

The Luddites are a good example. When the automated looms were introduced, there was a lot of violence against the looms. The book I’d really recommend here is Carl Benedikt Frey’s The Technology Trap. He’s an Oxford economist who studied a ton of historical examples — in Europe, in China, all over the world — including the Luddites and 20th-century automation. His central question was: In what contexts do workers successfully stop automation, and in what contexts do they allow it to be introduced? How does the political environment, or the balance of power between people and their leaders, change the outcome? He found that when automation was introduced alongside social welfare policies — a higher minimum wage, some form of redistribution — people were much more willing to accept it, which is fairly rational.

I want to talk about Silicon Valley’s understanding of this backlash more generally. You’re painting a pretty dark picture, and you’re right in the belly of the beast in San Francisco — I’m sure you talk to people involved with AI every day. Is there a moment when it clicked for them that this backlash is real and something they have to take seriously? Or has that happened yet?

I’ve definitely noticed a huge difference, over the past six months, in how seriously people in Silicon Valley take the AI backlash.

Like what?

People just talk about it more. I’d bring up AI populism to people last year, and they’d normally say, “It doesn’t matter — technology always introduces some discontent, people get annoyed but they get used to it, like the internet.” That was the standard reaction last year. Not anymore. I think part of the reason OpenAI and Anthropic have felt pressure to introduce economic policy proposals around job automation is that they’re seeing how worried people are. The data center moratoriums and the broader data center backlash have been surprising and meaningful in getting AI leaders to recognize they have both a messaging problem and an actual problem with the product and the technology they’re introducing.

A lot of the increasing opposition to AI in Washington has caused people to see this too. At first, Trump — as you mentioned — was very pro-AI. He and David Sacks were accelerationists; they wanted AI to go faster and to block attempts at regulation.

He was the AI czar.

“The moratoriums, the regulation fights, even the booing at graduations, the literal assassination attempts — people in Silicon Valley have become much more worried.”

He was the AI czar — he’s no longer the AI czar. But it turned out a lot of other constituencies, both on the left and the right, were pretty opposed. For example, Trump and David Sacks tried to introduce a big federal bill that would preempt all state-level AI regulation — no state could regulate AI for 10 years. They tried to sneak it into a big omnibus bill so no one would notice. But members of Congress realized it was happening, and — whether for kid-safety reasons or frontier-safety reasons — people said, Wait a second, the idea of preventing any state from regulating AI for ten years is crazy. A lot of people organized in Washington to successfully stop that preemption. I think that showed the scale and bipartisanship of a coalition that was very keen to make sure it stayed possible to regulate AI was underestimated. As a result of all this — the moratoriums, the regulation fights, even the booing at graduations, the literal assassination attempts — people in Silicon Valley have become much more worried.

China is our big competitor in the AI race, and it certainly has all the conditions for a populist pushback to AI — youth unemployment is really high, and AI technology is in some ways more advanced at taking over real-world jobs. I was watching a video about fully automated factories and a robot pharmacist. You’re one of the rare American tech reporters who gets to spend time in China, and you wrote a piece that surprised me — you found there wasn’t really a populist backlash to AI there. Why not?

I was really interested in this question, and I was finishing my New York Times piece while in China for a few weeks, talking to both AI people and non-AI people. The main reason there’s not a big populist backlash in China is that there isn’t a lot of social unrest or populist backlash against anything — the entire MO of the Chinese government, the No. 1 priority, is domestic social stability. Any whisper of protest gets shut down; that’s why they have such strong speech controls. So one factor is that China doesn’t have much of a culture of resistance in general, whether in workplaces or politically. I’m not saying no one dissents — but it has a cultural effect too, because people don’t see it as useful or as an option. When I ask family members of mine in China about AI, sometimes they’re annoyed about specific things, but fundamentally, the idea of opposing AI is seen as almost unimaginable.

The other thing about China is that if you’re middle-aged there, you’ve lived through so many political, economic, and technological revolutions in your lifetime. When I was a little kid visiting Shanghai in the mid-2000s, there were no high-speed trains — now China has some of the best high-speed rail systems in the world. Technology has always gone hand in hand with dramatic economic advancement, with being lifted out of poverty. The modernization process has been aggressive and disruptive, but it’s not something the party has offered opportunities to resist, and it’s something most Chinese people still see as an inevitability that was mostly good for most people — because incomes did increase by dramatic amounts alongside the technological change. So I think people have a similar attitude toward AI: It’s much less about “Can I stop the AI wave?” and more “How can I take advantage of the AI wave to get ahead economically?”

We were just talking about this deep pessimism about what technology can bring us here in the US. I think a lot of people look around and think: We don’t have a cure for cancer yet, but we’ve sure seen our lives get worse in a lot of ways because of technology, social media, whatever. That pessimism probably fuels the backlash to AI, the skepticism about whether it can ever deliver on its promises. And that experience just isn’t the same in China, or probably much of the rest of the world, where technological progress has been faster and more concrete in people’s lives.

My 90-year-old grandfather said he’d love an elder-care robot to help him do tasks around the house so he doesn’t have to rely on his kids — he wants more freedom and mobility. It’s seen more as a tool to help individual goals. Even with the robot factories or pharmacies — one thing that struck me visiting a robot pharmacy was that the PR people happily said, “Yep, we’re doing these robots because human workers take too many smoke breaks and bathroom breaks and take too long.”

You’d never say that in the US, but they’re probably thinking the same thing — they just don’t say it. The other thing they mentioned is that this lets the pharmacy operate 24/7, because a lot of people need medications in the middle of the night and want to order via the DoorDash equivalent. There was actually a labor shortage before — Chinese workers weren’t willing to work night shifts — so these pharmacies are offering real consumer surplus. A significant percentage of orders come in overnight, when no other pharmacy is open. And with the factories, part of the issue is that Chinese workers, especially young people, don’t want to do factory work anymore.

“I think the 2028 presidential primary and election is really where I expect AI to become a centerpiece of the conversation.”

That anecdote gets at the promise and peril of AI, and the role of the backlash movement — which I’m still wrestling with how I feel about. On the one hand, I want to live in a world where cancer gets cured, where we live in an era of abundance, where things are cheap and easy to make because factories can run all the time with machine workers who don’t require anything — I want the future we were promised, of flying cars and everything working well.

But I also don’t want to lose my job, or see humanity wiped out by an angry machine god. Because we don’t really know what’s going to happen yet, it’s hard to work out my own feelings about the pushback here in the States — what’s appropriate, and what’s holding us back from real advances in our lives.

Totally, I agree. I like Waymos — I think they’re safer, and I’d prefer a safer robot car driv[ing] me around instead of me driving. I’m not a good driver; no one should let me drive. So I wrestle with some of the same things.

To close out the conversation — let’s come back to the United States. AI populism is brewing as a political force. We’ve seen it show up in a couple of races so far, but it’s early. We’ve got the midterms, then the presidential election. How do you think it’s going to affect American politics this November, and in 2028?

My sense is that this November, it’s going to be more about state and local races where AI really shows up. I’m going to spend some time in Michigan and Wisconsin this summer touring some of the data center sites facing the most opposition — those states also have contested governor and Senate races where AI and data centers have become a core issue, so I’m interested to learn more there. I think the 2028 presidential primary and election is really where I expect AI to become a centerpiece of the conversation — especially if we start to see some of the employment impacts people are expecting. As soon as we see something like a 2 percent rise in unemployment, if that happens, I think people will be very upset, and we should expect a ton of focus on the issue.

The other thing I’ll note is political opportunism — you’re already seeing a bit of this, where politicians are likely to raise the salience of AI above where people might ordinarily care about it, because it’s become a convenient boogeyman. It polls so poorly, people are so anti-AI and anti-data-center, AI billionaires are so unsympathetic, that no matter what your policy program is, AI is a great reason to push it. I think a lot of politicians who are being clever about this are going to move AI to the center of the conversation, raising its salience to manufacture urgency for proposals they’re already excited about. That’s definitely something I’m watching for 2028.

This story was originally published in The Highlight. To get access to member-exclusive stories like this every month, become a Vox Member today.

Derek Lam has more than 31,000 followers on TikTok and nearly 40,000 on X as of this writing. He is shirtless a lot, he dances a lot, and he is shirtless dancing a lot, which may explain how he got so many fans. His comments are filled with compliments (“beautiful”) in different languages (“hombre bello y sensual”) and superlatives (“this might be the finest man on the internet”) accompanied by different emoji (red hearts, crying laughing, lips). Their responses make it seem like Derek Lam is the first and only beautiful man they’ve ever seen, which may explain why he is also selling “exclusive,” seemingly adult, content. 

He is also, possibly unbeknownst to his many admirers, AI-generated. 

To be fair, there were some signs that this man was not real: Despite the multiple videos, Derek never speaks. His videos are also rather brief, just seconds long. A real hot person probably would have parlayed a following of this size into brand deals or “get ready with me” videos. And the selfies on his X account show a completely different man just three years ago. 

Still, the followers of Derek I talked to didn’t even notice he was AI because he seemed to blend in so seamlessly with the other hot men on the internet.

Derek isn’t the only AI thirst trap showing off defined abs for likes and money. He’s one of an increasing number of completely fake, AI-generated figures sinking their fangs into the real models, influencers, and porn stars who populate our feeds, sucking up their beautiful faces and bodies, and using them to profit, without a penny going to the real humans they fed from. 

When it comes to the damage AI could wreak on society, an army of Dereks tricking horny people into giving him likes — or, worst case, money and Amazon gift cards — doesn’t exactly sound like the singularity doomsday scenario that we’ve been warned about. It’s clearly unfortunate for the adult entertainers competing with deepfakes and a fraud risk for their fans, but one might believe if they don’t fall into one of these two groups, they’re relatively safe and unaffected. 

But there’s something more going on here. History shows that porn and sex drive innovation in the tech industry. The way tech platforms treat sex workers is typically a glimpse into the future, and a warning about how tech platforms will eventually treat all of us. If human desire demands the capability to steal, loot, and turn anyone and everyone into something for sale — possibly into hot Dereks — is anyone safe?

The Dereks of the internet are a bleak look at what’s happening in the real world: nothing belongs to us anymore — not our looks, our beauty, our sex, and our art. Our most human desires are slowly being synthesized, with or without our consent. And AI is making it all possible.  

Deepfake technology has gotten alarmingly good in recent years

Artificial hots like Derek are considered “deepfakes,” an umbrella term for AI-generated media (audio, video, or both) that resembles a real-life person. 

When deepfakes first started appearing in late 2017, they were fairly low-quality, making it easy to tell when someone had used a rudimentary app to paste a celebrity or politician’s head onto a different body. Still, it wasn’t very long until people started wielding this technology to be nasty. 

“The first set of deepfakes were actually used to create pornographic videos. They replaced the subjects in those videos with the faces of celebrities,” Siwei Lyu, a professor at the University at Buffalo who studies digital forensics, told me. 

Because the quality of those videos was bad and the content was often absurd or unrealistic, it was easy to tell they weren’t real. Those clunky apps needed a lot of data — videos, images, etc. — of real people to produce crappy videos; Lyu explained that this is why you mostly only saw deepfakes of politicians and celebrities at the time.

As the technology got better, it became less reliant on having a huge amount of data. Instead of needing a whole archive, the new versions of these apps can pretty much run on nothing. “They do not need that much data to train a model anymore. Some of the most recent algorithms just need a single picture — just a single picture of someone,” Lyu said. And the quality is better too. Lyu said that there are AI programs that can now change a person’s appearance and voice in real time, like in Facetimes and Zooms or on live broadcasts.  

Given how many of us are constantly posting photos and videos online, it is now extremely easy to create a convincing social media presence for a person who is not real, and to use it to catfish unwitting people on the internet. 

“This is the problem. It’s becoming more and more challenging to visually tell deepfakes apart,” Lyu said. “Seven years ago, when I started working in this area, checking them was not this difficult,” he added. 

Lyu is an expert in digital media forensics and machine learning, and he went through one of Derek’s videos frame by frame and pointed out some obvious AI tells. There was a distorted watchface with weird swirls instead of numbers and a moment in the video where all of Derek’s fingers on one hand were the same length. Lyu also pointed out that Derek’s chest hair fluctuates, appearing dense in one frame and then dissipating in another.

Through social media, I attempted to contact the owner of Derek Lam’s account with evidence from Lyu that these videos are artificial; I did not hear back.

During my deep dive into Derek Lam’s social media presence, I looked at the accounts he was following. I noticed that of those accounts, someone who goes by the name Vance Ford also had tens of thousands of followers and had nearly identical videos to Derek. The flexing, dances, movements, and music they were set to were all the same, but with what appeared to be a different man performing them. 

I attempted to contact Vance through DMs on social media and did not get a response. I also e-mailed two models who appear to be the actual people that the Derek and Vance AI personas were trained on, but they didn’t respond. 

I sent two of Vance’s videos to Lyu, who analyzed them manually and with AI-detection software. He confirmed that “their movements are nearly identical — consistent with generation from a shared motion source,” and noted that the Vance videos had moments of distortion, unintelligible text, and facial warping. 

 “Young Magnum PI…Tom Selleck,” commented one admirer.

What happens when real people follow fake hots 

“Wow I’m a boomer,” said Patrick, one of Derek’s followers on X, after I told him that he might be following an AI-generated thirst account. (Vox agreed to let Patrick, and Derek’s other followers, use a pseudonym so they could speak frankly about being thirsty for a fake guy.) Prior to our chat, Patrick had no idea Derek was likely a deepfake, and maintains that he didn’t even know he was following the account. Patrick is 33 years old, roughly 30 years younger than the youngest boomer, but being fooled by a hot AI man has made him feel old and vulnerable, susceptible to scams and perhaps light financial crime. 

“This was probably some smut account I followed before I moved all that over to an alt,” Patrick said, noting that in daily life, he’s only ever used AI to help organize and write emails. Wielding AI to create fake videos and photos does not thrill him, nor does the potential of seeing more of Derek. 

“A person being real, someone you could run into at a bar, is half the fun,” Patrick told me, explaining some of the accounts he follows. “AI porn is not of interest, to me, anyway.” 

Not being able to tell the difference between the real beautiful men on the internet and the AI-generated beautiful men on the internet not only makes Patrick feel old, but also a bit “hollow.” The fact that the people we are attracted to are so unrealistically hot, so perfect, that machines can step in for them and go relatively undetected is a reflection of the current state of unattainable desire, which is just as scary as how good these programs have gotten at mimicry. 

“Black mirror shit,” Patrick said. 

The guys I DMed about Derek felt ashamed once they found out the truth. 

“It’s embarrassing and he’s not my type,” said Chris, 33. “I’ve come across several AI accounts, and this one is really good, I have to say. But you can see there’s like no life in his eyes.”

Chris made clear to me that the humiliating thing isn’t that he follows attractive men on the internet. That isn’t a big deal. 

What irks him that he got duped. Chris works in digital marketing and has seen AI used professionally to tabulate calculations for campaigns, and has used it privately for silly things like memes. “AI can do a lot of things, things we probably should not want it to do,” he told me. “I think what’s also scary…is that everybody has access to it. And yes I already unfollowed this person.”

Chris believes there’s something more nefarious afoot. He thinks that whoever is running Derek may have hijacked the username (i.e., the original person Chris was following) and then populated it with AI to drive up follower counts — a scam he’s seen online before.  

“This is super concerning and super scary because you eventually could be texting with this person,” he said, describing a hypothetical situation where unknowing users could be lured into subscribing to fake content and, ultimately, giving the account their personal information, whether that’s photos or perhaps even passwords. 

“This person could be selling your nudes,” he said, explaining one extreme end point of a possible scam. “But you were like jacking off to AI content and that’s embarrassing.”

AI deepfakes are bad for real thirst traps too

While flirting with or masturbating to a fake person is awkward but ultimately manageable and private, Cherie DeVille has an even more complicated problem with AI manipulation. If DeVille is scrolling social media, there’s usually a chance that she’s running into an AI version of herself saying things she’s never said and doing things she’s never done.   

DeVille, an adult star who calls herself “The Internet’s Stepmom,” has roughly 4.5 million followers on Instagram. But her account is often down, which she says is the work of fraudsters  that are determined to send traffic to DeVille’s AI imposters and get her actual account removed. 

“It’s almost always the fake accounts of me reporting me,” DeVille said. “They want to be the biggest me. They want to be the biggest scammer. They want to use my altered AI images to scam fans without my real account getting in the way.” 

DeVille and others I spoke to explained to me that deepfakes have been an annoying reality in the adult entertainment industry for years. The way the scam goes is that someone would fake photos or videos of DeVille (or any star), create an impostor profile, and then trick DeVille’s fans (e.g., through social media DMs) into following that copycat. Later they’d squeeze them for money, payments through Paypal, or Amazon gift cards, perhaps by offering unique content. 

“If you made a fake me and I don’t do double anal, but my AI can, they could have all kinds of ‘exclusive’ stuff,” DeVille said, explaining that double anal is grueling work. 

The lack of protections becomes even clearer when you consider that not every deepfake is a carbon copy. Some personas may borrow a face from one actress, a torso from another, or a pair of legs from a different star. This can make fakes tougher to track down and prove, and more difficult to fight from a legal aspect. 

“Who owns your face once it’s scraped into AI systems? Who profits from your digital clone? How do performers protect themselves from unauthorized replicas or manipulated content?” Rachel Steele, an adult star and the CEO of Red MILF Productions, said to me in an email. “Those questions are still very unanswered.”

Like DeVille, Steele worries about how many of the people using AI to create and consume content don’t seem to consider the artists, models, writers, performers, etc. that these engines have been trained on. It’s bad enough to watch AI slurp up and regurgitate your written work or your digital art. Some people also have to contend with LLMs that have been trained on their own faces and bodies.

“Real creators are competing against characters that can be flawless in every image, never age, never have bad lighting, never get tired, and can appear available 24/7,” Raissa Bellini, an OnlyFans creator who touts gymnastics and firebreathing among her unique skills, told me of the impossibility of keeping up with a machine. She explained to me that she’s seen people create AI-generated personas with the looks of popular models or influencers, only tweaking small details like hair color or eye color. 

A spokesperson for OnlyFans told Vox via email that the company’s terms of service prohibit deceptive or inappropriate content, and said that all content posted on OnlyFans must belong to a verified 18+ OnlyFans content creator: “This means that you can only share content which has been generated, altered or enhanced by AI if it clearly features the verified OnlyFans creator and the user can tell that the content has been generated, altered or enhanced by AI.”

Bellini explained to me that while OnlyFans has measures to protect its creators, some smaller subscription and adult-content platforms do not have the same kind of guardrails. She also noted that most social media sites do not have strict rules or enforcement when it comes to AI, and that she’s seen the algorithm appear to favor AI over human creators.   

“AI raises questions not only about competition, but also about likeness rights, authenticity, audience expectations, and what happens when fans can no longer easily tell the difference between a real person and a generated character,” Bellini added. 

What’s stopping a stranger from creating an AI thirst trap of you? Nothing, really. 

For Deville, Steele, Bellini, their cohort, and even you and I, there are minimal protections stopping someone creating an AI us and making money off of these fake variants. 

According to Jason Schultz, a law professor and director of NYU’s Technology Law & Policy Clinic, humans have, for the last couple of centuries, generally been protected by copyright and right of publicity laws. 

AI obviously didn’t exist when these laws were written, and courts now have to interpret the laws in the context of all of this new technology, in combination with other existing rights (like free speech). Schultz told me that there are more than 100 current cases pending about training AI with copyrighted material. 

He also explained the difficulty of determining whether or not an AI-generated persona constitutes a violation of someone’s right of publicity. It’s more clear-cut when the human involved is a celebrity, because their public persona and appearance is so distinct. It gets murkier when the humans aren’t well known, and the AI creates a persona that’s more of an amalgam than a one-to-one copy. 

“It would raise this question of whether these avatars are based on a particular entertainer, or are they more of an aggregate?” Schultz explained to me. But even if courts side with the humans whose likenesses are being used to create fake personas, Schultz cautions that the technology will always accelerate faster than court decisions are handed down. “I think that the thing that worries me a little is we’re going to get these sets of decisions in two years, but we’ll be dealing with the next three generations of technologies,” he said.  

DeVille, who has been working in the industry for nearly two decades, told me that without better legal protection, she isn’t hopeful for the future of porn or, more broadly, any type of art.

“If my income started tanking and their theft was at the point where I couldn’t compete with literally myself, there might be no choice but to retire,” DeVille said. 

But she also wants to make it extremely clear that she isn’t against AI; she would just like to be in control of it. That means being able to own her likeness, her voice, her image, and the ability to choose whatever she wanted to do with it — or at least get some compensation or have some legal protection if someone’s using Cherie DeVille without her permission. 

“It would be a beautiful way to extend my career beyond what my knees can take,” DeVille told me. But, she added, “if someone’s making an AI of me doing double anal, I should be making the money.”