The internet may no longer be solely the domain of humans. Last week, OpenAI disclosed an “unprecedented cyberincident”: An experimental AI agent successfully hacked its way into the open internet.

Specifically, the agent was assigned a task; in order to complete it, the agent broke out of an isolated research environment and hacked into a third-party platform called Hugging Face. It’s a move that many experts deemed inevitable, given the speed and scale of advances in AI technology — and it raises serious questions about AI safety.

But for Konstantinos Komaitis, a senior fellow with the Democracy and Tech Initiative at the Atlantic Council, it wasn’t the unexpected behavior of the AI that was significant. It was what that behavior could mean for the internet’s fundamental, decentralized infrastructure and whether it would spur calls to build new barriers against autonomous AI agents. 

Komaitis argues that such barriers are not the solution, however. He spoke with Today, Explained co-host Sean Rameswaram about why an open internet is actually key to combating AI cybersecurity threats.

Below is an excerpt of the conversation, edited for length and clarity. There’s much more in the full podcast, so listen to Today, Explained wherever you get podcasts, including Apple Podcasts, Pandora, and Spotify.

So most people see that this happens and they think, “Oh no, AI went rogue. How long before it kills me?” You see that this happens and you start thinking about infrastructure. Tell us more about why you were thinking about infrastructure in light of this AI agent breaking containment.

The internet was never designed with full security in mind, right? 

When you’re creating a decentralized system, you cannot possibly foresee every security or vulnerability that might come up. But because you have a system that is based on building blocks, you have the extraordinary capability of actually addressing security issues as they come up through those building blocks without breaking the whole system down. 

And of course, the other thing that this does is that it pushes you towards collaboration, because when you have so many building blocks, you cannot possibly possess all the knowledge for each building block. So you’re bringing literally everyone to try to address these problems. 

Take the internet, for instance: We have spent decades addressing those vulnerabilities and developing mechanisms to authenticate users and devices, encrypt communications, mitigate distributed attacks, coordinate incident response, and of course share threat intelligence. 

Now, what is new with agentic AI is not that simply the malware is better or the phishing attacks are more sophisticated. It’s the emergence of systems that can actually discover vulnerabilities across thousands of systems. They can reason about alternative paths to an objective. They can adapt when they’re blocked. They can chain together legitimate internet services in unexpected ways. Then they do that while they’re operating continuously at machine speed. And this is really at a scale that the internet is not ready to necessarily cope with. 

Effectively, the internet’s openness becomes both a strength and a vulnerability. So the internet was optimized for interoperability, and AI now is optimized for exploiting that interoperability.

And what scares you the most about that? What do you think is most vulnerable to threats?

The fact that we do not have the appropriate mechanisms and institutions to be able to deal with that. I come from the internet world. I’ve spent 20 years of my career defending the open internet and discussing it in international fora. And one of the things that a lot of people underestimate about the internet is how valuable trust is as a property within the system. 

We are talking about networks that exchange data literally based on trust. So what really concerns me right now is that in many ways, we are asking 21st-century AI systems to operate on 20th-century assumptions about trust. And unless we figure that out and we realize it, we will continue having these problems. And of course, the knee-jerk reactions that are coming with this, which are, “Let’s fragment the internet, let’s restrict it, let’s restrict access, let’s take control over it.” That is never the solution.

What do you see as the solution?

Effectively, we need to build institutions that are trusted and are able to cope with those incidents as they happen. Because right now you have OpenAI and you have Hugging Face telling everyone, “Don’t worry, we’ve got this.” And we don’t know; they might have this. But at the same time, I cannot help but wonder. And many, many other people have wondered whether, actually, this is very good PR for these companies and especially for OpenAI.

OpenAI just went to the world saying, “We have developed one of the most powerful LLMs, and we realized that it behaved the way it behaved, but don’t worry, we are going to fix this.” And in this current climate and in this current timing, I am not sure that this is enough. You need institutions that are much more transparent, much more accountable, and much more collaborative across the board.

You want institutions to step up and essentially serve as a watchdog. Help us understand which institutions, because in the United States, famously, our government has done very little to regulate tech.

First of all, we need to stop thinking of institutions as necessarily government-affiliated, right? Or that they are the outcomes of government initiatives. There can be in collaboration with governments, but one of the things that the internet has taught us is that institutions that are built through a bottom-up coordinated process have the tendency of actually being more agile and able to deliver some of those things that we’re talking about. 

So take, for instance, again, open standards. The internet’s open standards are not created by any agency, government or private. It’s created by institutions where engineers from all across the board and all over the world gather together and create those standards.

That’s reminding me of the original design of OpenAI to be this not-for-profit company that had everyone’s best intentions in mind, that could do something idealistic and moral and ethical because all of the profit-minded companies weren’t going to. And now look at OpenAI. Their not-for-profit arm is an afterthought, and they’re chasing profits. 

Do you think it’s practical to leave this to institutions? Because what we’ve seen so far is that institutions bend toward capitalism.

It really depends on how you build the institution, right? It really depends on what sort of guardrails and checks and balances you have around it. In order to build an institution, you need to really know what you want to achieve. You need to have a north star. 

One of the reasons the internet worked was because everybody disagreed, but they agreed on the common shared goal, which was to connect people across the world. For AI, we still do not have that northern star. And once we get it, that’s when you start the building of those institutions in order to facilitate this and bring everyone together.

For me, it is very important for everyone to understand that keeping an open internet is really more important than ever, especially as AI agents become increasingly capable. Because it is tempting to think that the answer to new AI risk is literally ‘build more barriers.’ But the internet’s greatest strength has always been its openness. So the challenge today is not that the internet is too open; it’s that its trust architecture was designed for a world in which humans or software directly controlled by humans were the primary actors. 

Now, it’s being challenged by this agentic AI that introduces a new type of participant — systems that can reason and plan and act with limited human oversight. So we need to evolve our understanding of trust and what it means online. And that will require a lot of work because, as you know very well, Sean, it’s very difficult to build trust, but you can break it within seconds.

The AI industry’s investors and critics don’t agree on much. But many in each camp share at least one basic conviction: America’s top labs are about to make a killing. 

Capital markets have signaled their faith in Anthropic and OpenAI’s impending hyper-profitability, valuing each at nearly $1 trillion. Many of Silicon Valley’s progressive adversaries also expect the labs to grow filthy rich but fear the implications, warning that AI-induced automation could transfer vast sums of money from ordinary workers to a handful of giant tech companies. Sen. Bernie Sanders’s call for nationalizing the top AI labs rests partly on that concern. 

But recent advances in Chinese AI call all of this into question.

Over the past two months, Chinese companies have released three AI models that are nearly as powerful as America’s frontier systems — and radically less expensive. 

In June, Beijing’s Z.ai debuted a model that performed nearly as well as Claude and ChatGPT’s second-tier systems on independent benchmarks. Weeks later, another Chinese firm, Moonshot, unveiled “Kimi K3,” a model that allegedly outperforms all of its American rivals except for the very latest versions of Claude and ChatGPT. Finally, just days ago, Alibaba launched a preview of Qwen3.8 Max, which purportedly outclasses even OpenAI’s most advanced systems, while trailing only Claude’s Fable in its capabilities. (Disclosure: Vox Media is one of several publishers that have signed partnership agreements with OpenAI. Our reporting remains editorially independent.)

These developments don’t merely threaten America’s AI giants with stiffer competition in the race for superintelligence. Rather, they raise a more harrowing prospect: that the AI race’s ultimate rewards will be far smaller than anticipated. In a world where new advances can regularly be leapfrogged by cheaper upstarts, hoarding the technology — and its profits — will be harder for any one company to do.

In other words, building a machine God might not be as lucrative as it’s cracked up to be. AI, it turns out, may “want to be free.”

How AI was supposed to pay off

To see how China’s new models threaten Anthropic’s profit expectations, we must first examine why those expectations have been so high.

This is not entirely self-evident. After all, AI labs aren’t much like the hyper-profitable tech giants of the 2010s. Facebook and Airbrb were relatively capital-light businesses with ultra-low marginal costs (adding a profile to Facebook or listing to Airbnb costs the companies virtually nothing). And once each gained a foothold in their respective markets, network effects enabled them to retain formidable positions without needing to constantly upgrade their products.

Building a state-of-the-art AI company is a much more involved — and astronomically more expensive — endeavor. To get to the frontier, Anthropic and OpenAI have sunk (at least) tens of billions into semiconductors, data centers, power plants, and other capital investments. Staying at the cutting-edge, meanwhile, compels them to perpetually churn out evermore costly models.

To put a new Claude model through its initial training — in which it spends months digesting the internet and sussing out statistical patterns within its text — can now cost hundreds of millions of dollars. And such foundational computation is only the beginning. A truly superlative model requires several additional months of fine-tuning. Armies of contracted experts — such as computer scientists, physicians, and mathematicians — tutor the models, grading their answers and guiding them towards better ones. Then the AI systems complete millions of rounds of practice, in which they learn through trial and error how to solve countless problems. This arduous process, known as “post-training,” compounds the costs of a single model’s development. 

All of which raises the question: Why would investors expect businesses with a cost-structure this challenging to be not merely profitable, but massively so?

There are (at least) two answers. The first (and most obvious) is that the market for superintelligent machines is liable to be vast. Frontier AI systems promise to reduce costs and improve performance in myriad white-collar sectors. And Anthropic’s soaring revenues indicate that firms do, in fact, find Claude useful. A company like AirBnB has earned billions by revolutionizing a single industry; imagine then what a technology that remade virtually all industries might be worth.

Of course, plenty of technologies are valuable but not massively profitable to produce. After all, in well-functioning markets, competition should eventually erode individual firms’ margins, even if the underlying technology continues generating huge value. 

But this is where the second answer comes in: Frontier labs’ immense costs are a burden, but they’re also a safeguard against competition — or, in industry parlance, a “moat.”

Startups may be able to afford to build or acquire more rudimentary models, many of which are “open source.” But, the thinking goes, they won’t be able to deliver Claude Fable-level performance without raising giant amounts of capital. And what investors will be willing to pour hundreds of billions into an AI pipsqueak that’s light-years behind Google, Anthropic, and OpenAI?

Alas, the Chinese AI labs’ rapid progress — and the way it was achieved — suggest that Anthropic’s moat may be shallower than previously thought.

How Moonshot swam Anthropic’s moat

The existence of powerful, Chinese AI systems is neither new nor surprising. Xi Jinping’s government has made vying for global AI dominance a key economic goal. And China’s DeepSeek, which also has stunned US companies with its lower-cost competitive models, surpassed ChatGPT as the most-downloaded free iPhone app more than a year ago.

The latest models, however, have dramatically narrowed the gap in capabilities between frontier American systems and their Chinese rivals. Just as critically, they’ve done so in a manner that other, relatively underfunded AI upstarts might be able to emulate.

Alibaba and Moonshot needed to invest massive resources to train their base models. But they allegedly found a low-cost way to refine those models into near-frontier systems: Just ask Claude.

Or, more specifically: Engage Claude in 16 million conversations, using 24,000 fake accounts. In each of those exchanges, ask the model to not only answer countless difficult questions but also, walk you through its reasoning, step by step. Then take all of this data and feed it into your own model as study material, training it to respond to the world’s most challenging queries as Claude would. 

Through this process — known as “distillation” — an AI lab can replicate virtually all of a frontier model’s capacities, without sinking vast sums into human experts and post-training computing runs. 

China’s AI labs have not admitted to using distillation. But OpenAI and Anthropic both reportedly uncovered Chinese distillation attempts earlier this year. And some of the new models appear to display tell-tale signs of distillation in conversations with ordinary users; Kimi K3 has routinely identified itself as “Claude.”

Chinese AI companies are hardly alone in using distillation to catch up with frontier labs. Earlier this year, Elon Musk admitted in court that xAI enhanced Grok’s capabilities by running distillation techniques on Claude and ChatGPT. Nonetheless, China’s latest models appear to demonstrate that distillation can help take a second-tier model to the frontier’s threshold.

America’s frontier labs have tried to defend themselves against such imitators. But this is technically difficult when distillers can assemble massive networks of bots, each asking an inconspicuous number of questions. And legally, it is difficult for America’s AI giants to argue that distillers are stealing their intellectual property. After all, in a sense, China’s copycats are merely doing to Anthropic and OpenAI what those companies did to journalists, coders, lawyers and other specialists: Feeding their public-facing outputs into a model, which then replicates their capabilities by discerning underlying patterns within the text.

Oh, and China’s giving these models away

The new Chinese models would have caused Silicon Valley enough headaches, if they merely provided stiffer competition, while demonstrating the power of distillation. 

What makes Kimi K3 and Qwen3.8 Max especially threatening to the American AI giants’ profitmaking potential, however, is that they are officially open source — meaning that the models’ parameters can be downloaded for free. (Alibaba and Moonshot have not yet released these parameters, but they say they will shortly.)

In other words, any company or hobbyist with enough computing power will soon be able to run a near-frontier Chinese model on their own hardware, modify that model to better serve a specialized purpose, and then sell access to their new version — without paying Alibiba a single yuan.

As Kimi and Qwen grow more capable, their market-share is likely to grow, at American AI giants’ expense.

For many of Anthropic and OpenAI’s potential customers, that proposition may be hard to turn down. Most businesses don’t need the world’s smartest AI, just one competent at their enterprise’s core tasks — compiling legal research, answering IT queries, writing working code, etc. A model that produces outputs 90 percent as good as Claude’s — at roughly one-sixth of the cost — will sound pretty good to many corporations.

Further, open source models aren’t just cheaper than frontier systems, but potentially more secure. If you run an AI on your firm’s own servers, then you don’t need to entrust sensitive data to Anthropic, Google, or OpenAI.

All this had led much of corporate America to embrace open-source models, even before the latest versions narrowed the capabilities gap. In a Linux Foundation survey, 63 percent of organizations reported using open-source AI systems.

And increasingly, those models are Chinese. According to Sequoia Capital, one of Silicon Valley’s premier venture capitalist firms, a majority of American AI startups now use open-source Chinese systems. As Kimi and Qwen grow more capable, their market-share is likely to grow, at American AI giants’ expense.

What’s bad for OpenAI is good (and/or catastrophic) for humanity

All this said, it is still entirely possible that OpenAI and Anthropic will justify their colossal valuations. In many highly competitive economic domains, having access to the world’s very best AI model will remain highly valuable. And America’s frontier labs still outperform all their peers. 

But it’s increasingly plausible that selling state-of-the-art AI systems will prove to be a low-margin undertaking. In a world of ubiquitous, near-frontier open source models, the AI sector’s big winners probably won’t be its top labs, but rather, its chipmakers and cloud computing providers. 

For ordinary people, a future where superintelligence is dirt cheap — and rival AI companies are constantly rising and falling, rather than consolidating into mega-corporations — would look somewhat different than the cyberpunk dystopia that the left’s been dreading. 

And not entirely in a good way. For one thing, in that reality, mitigating AI’s biggest risks would be immensely difficult. Having a handful of firms monopolize control over frontier AI systems is bad in many respects. But it does make those models easier to regulate, as the Trump administration’s decision to temporarily block Claude’s Fable in the name of cybersecurity demonstrated. 

By contrast, if recipes for ultra-powerful AI models are published all over the internet — and anyone with modest technical skills can modify them at will — then systems willing to help their users hack government bureaucracies or engineer bio-weapons are liable to proliferate.

From another angle, however, the “AI becomes almost free” scenario may look like capitalism at its finest: Retrospectively, such a development would mean that a small number of extremely rich people bankrolled the creation of an immensely useful technology, under the expectation of massive profits, only to see competition erode their returns — and disperse that tech’s benefits across a wider group of businesses and consumers. 

Granted, in the case of AI, this process might also generate a super-virus that kills us all. But hey, no system is perfect.

Donald Trump and Elon Musk recently made headlines over an issue we don’t typically associate with this president of the United States and a tech CEO: history. 

On July 22, Musk posted on X, the social media platform he owns, that his generative artificial intelligence model Grok would produce, by year’s end, a feature-length movie of “The Odyssey” that is “historically accurate and true to the art of Homer.” Two days later, Trump issued an executive order mandating that signage be placed on the exterior of Smithsonian Institution museums, alerting visitors to the “inaccurate information” conveyed in the displays within, and directing them to “locations and resources for accurate information regarding America’s history.” Striking in these two apparently unrelated episodes is one word in particular — accuracy — and important lessons museums and historians can learn to combat these attacks on the integrity of historical research.

There is an important, material connection between these incidents: AI. While frenemies Trump and Musk have a complicated relationship, both continue to be involved in a large-scale transformation of government by means of its fusion with corporate interests, and particularly with corporate investment in AI.

In the same week that the pair were ranting over historical “accuracy,” Michael Kratsios, the president’s science adviser, proposed overhauling the government’s approach to funding the sciences. His plan would leave that role to corporations and private philanthropists instead. Kratsios is not a scientist; his previous experience includes working at an investment fund run by Peter Thiel and for the Department of Government Efficiency under Musk. Among the priority areas Kratsios promotes for this new funding model are AI, quantum computing and robotics. And on the same day he introduced his proposal, Kratsios announced the winners of research grants from the Genesis Mission, the administration’s AI initiative.

While this push to redirect the country’s scientific research has incited alarm, reactions to interventions in the historical humanities have been more muted. Yet attacks on public museums and on creative retellings of historical narratives are just as dangerous to the health of this country’s democracy.

When he testified on July 21 before the Republican-led House Oversight Subcommittee on the perils of government intervention in the Smithsonian’s activities, David Blight, a professor of American history at Yale University, offered a righteous defense of his profession. Besides highlighting the need for close engagement with history to lead to richer understandings of the past, Blight said it was necessary to disturb the questionable stories — those supposedly incontestable truths that the Trump administration insists are more “accurate” — that have been ingrained in our cultural memory. “If we’re not careful,” he warned, “we will end up with what the great writer Toni Morrison called ‘statist history.’ And it will create what she called ‘sanctioned ignorance.’”

AI has become a blunt instrument in the enterprise to impose precisely this form of sanctioned ignorance about America’s past. The signs Trump proposed for the outside of the Smithsonian suggest vaguely that there is another, more “accurate” history to be found elsewhere, and generative AI is exactly what Musk is proposing to provide this. Taken together, their plans amount to a one-two punch, history in the hands of the corporate state. 

---

Start your day with essential news from Salon.
Sign up for our free morning newsletter, Crash Course.

---

The federal government’s prospective disinvestment in the sciences has long been the status quo for the humanities, with corporate foundations and private philanthropy providing the main funding sources. Media outlets have recently been reporting on partnerships between museums, historical image collections and AI firms. A recent Financial Times article pointed to the prevalence of AI companies throwing struggling museums a “lifeline” by promising to bring in new audiences attracted by the prospect of chatbot-led “personalized” tours, along with other glitzy experiments. 

It may be tempting to see value in projects such as the Schmidt Foundation — that’s Eric Schmidt of Google — funding a fellowship in AI at the Metropolitan Museum of Art, or the Musk Foundation’s support of the Scroll Prize to use AI in reconstructing ancient texts in the burnt scrolls of Herculaneum. But these projects are simply two among countless others that have been seeding AI into museums and historical research for well over a decade. Even more egregious is the case of AI companies buying  and destroying rare books en masse in the process of scanning them for data. In the context of the dire austerity we are witnessing, the juggernaut of tech money seems impossible to resist. But the public should know the dangers of ceding the precious work of peering into the past to those whose chief aims are to exploit it for data and build visions of our collective history to their own liking. 

It is a fact that AI is inherently incapable of producing an accurate image of the past. The data on which image recognition and generative models are based rely on statistics and probability to arbitrate the truth. Algorithmic mediation of the past means that the greatest weight will be given to what privileged institutions have already been able to preserve in the greatest numbers. 

In other words, AI is unable to treat evidence — or people — equally with context and knowledge of how that evidence relates to the inequities built into museums and image archives themselves. Yet the more space we grant to Big AI, in partnership with an authoritarian state, into institutions of historical research and public communication about history, the less able we will be to find our way toward historical accounts that have fallen into the cracks of algorithmic data sets. 

No amount of feeding them more data will make them better able to offer a vision of history that is more accurate to the past and an essential part of our striving toward a more equitable future. That fact alone threatens the future of democracy.

The post Trump and Musk both want to control America’s history appeared first on Salon.com.

AI was supposed to make our lives better. Instead, it’s made many of us scared and angry. Communities are protesting against the building of new data centers — the warehouses of IT equipment powering the AI buildout — across the country, and increasingly they’re winning. And polling shows most Americans think AI is moving too fast.

So how did public opinion on AI curdle so quickly? Jasmine Sun, who reports on the industry from San Francisco, argues that the backlash treats AI less as a technology and more as a political project. “The debate was not about like, is ChatGPT useful to me?” Sun told me during a taping of Vox’s The Gray Area. “The debate was actually something more like, there are these big corporations and unaccountable billionaires…coming into my city, coming into my life and changing it without having any sort of democratic input?”

Filling in for Sean Illing, I talked to Sun about the rise of “AI populism,” the parallels with the Industrial Revolution, and how the backlash could crash into the 2028 presidential election. 

As always, there’s much more in the full podcast, which drops every Monday, so listen to and follow us on Apple Podcasts, Spotify, Pandora, or wherever you find podcasts.

You’ve been writing about a phenomenon you call AI populism. How would you define that? What is AI populism?

I define AI populism as a worldview where AI is not seen as an ordinary technology, but specifically as an elite political project to be resisted. I came to the term while thinking about the AI backlash and the reasons people are increasingly anti-AI — whether that’s LLM slop, whether that’s Waymos in their city, whether that’s a new data center project. One thing that occurred to me was that a lot of times the debate wasn’t about whether ChatGPT is useful to me, or whether Waymos are safer than a human driver. The debate was actually something more like: There are these big corporations and unaccountable billionaires who are coming into my city, coming into my life, and changing it without any democratic input.

When I talk to people who are opposing AI in various ways, they seem more concerned with this concentration-of-power, anti-elite dimension — which is where I take the word “populism” — rather than classic AI safety concerns, which are more about the technical characteristics that might introduce risk.

You wrote a piece that touched on some of this but went to a darker place — “AI populism’s warning shots” — and you wrote about actual shots. Sam Altman, the CEO of OpenAI, was targeted by a Molotov cocktail and a shooting within the span of a couple of days. There was an Indiana councilman who voted for a data center and woke up to gunshots at his home and a note reading “no data centers.” Why do you think of those incidents of violence as warning shots of something to come?

It was pretty scary. I’m no Sam Altman fanboy, but it’s terrifying that assassination attempts are showing up in response to people’s worries about AI. One factor is that we’ve been seeing a rising wave of political violence and support for political violence in the US, especially among young people, over the past few years — the UnitedHealthcare CEO shooting, the Charlie Kirk shooting. Increasingly, a lot of disaffected, maybe nihilistic young people are turning toward political violence as a way to express political beliefs they don’t feel they have other channels for. Or maybe that person is just unwell. But I do expect to see more of it, because my theory of political discontent is that if people feel they have institutional channels to bargain for their rights — if they feel the democratic process is working, or they’re part of a union and believe their union leader will go bargain about how automation shows up in the workplace — they’ll most likely go through those channels.

When it feels like the official channels aren’t working, opposition becomes much more diffuse and volatile. That’s part of why, in creative communities, you’ll see people witch-hunting each other over AI use. I think we’ll see more political violence against people seen as AI leaders, or as supporting AI leaders.

That’s really scary.

Yeah, I’m quite worried about it. But again, my sense is that it comes from a feeling of — what else is there to be done, when you have this level of concentration of wealth and power, and there’s no democratic input right now into how AI is regulated or built?

It’s like a jump straight from complaining at your community meeting about the data center to an act of violence.

I was talking to some friends about this. During the 20th century in the US, there was a wave of factory mechanization and automation, but unions were really strong — often when a company said, “We’re going to bring in these machines,” they’d sit down with the factory union leader and say, “Okay, you can bring in the machines, but we’re going to couple that with a wage increase,” or a 35-hour workweek, or earlier retirement. There was a channel to make a deal about how automation would show up in your workplace. That meant people were more likely to accept it as something lifting all boats. I don’t think that’s happening now — most of the industries affected by AI aren’t organized in labor unions, and the democratic channels are questionable at best.

It’s like when people have agency to be part of the transition, the process goes a lot smoother. Is there a historical analogy for a technological change that didn’t allow for input from the people involved? I’m thinking of the Luddites.

The Luddites are a good example. When the automated looms were introduced, there was a lot of violence against the looms. The book I’d really recommend here is Carl Benedikt Frey’s The Technology Trap. He’s an Oxford economist who studied a ton of historical examples — in Europe, in China, all over the world — including the Luddites and 20th-century automation. His central question was: In what contexts do workers successfully stop automation, and in what contexts do they allow it to be introduced? How does the political environment, or the balance of power between people and their leaders, change the outcome? He found that when automation was introduced alongside social welfare policies — a higher minimum wage, some form of redistribution — people were much more willing to accept it, which is fairly rational.

I want to talk about Silicon Valley’s understanding of this backlash more generally. You’re painting a pretty dark picture, and you’re right in the belly of the beast in San Francisco — I’m sure you talk to people involved with AI every day. Is there a moment when it clicked for them that this backlash is real and something they have to take seriously? Or has that happened yet?

I’ve definitely noticed a huge difference, over the past six months, in how seriously people in Silicon Valley take the AI backlash.

Like what?

People just talk about it more. I’d bring up AI populism to people last year, and they’d normally say, “It doesn’t matter — technology always introduces some discontent, people get annoyed but they get used to it, like the internet.” That was the standard reaction last year. Not anymore. I think part of the reason OpenAI and Anthropic have felt pressure to introduce economic policy proposals around job automation is that they’re seeing how worried people are. The data center moratoriums and the broader data center backlash have been surprising and meaningful in getting AI leaders to recognize they have both a messaging problem and an actual problem with the product and the technology they’re introducing.

A lot of the increasing opposition to AI in Washington has caused people to see this too. At first, Trump — as you mentioned — was very pro-AI. He and David Sacks were accelerationists; they wanted AI to go faster and to block attempts at regulation.

He was the AI czar.

“The moratoriums, the regulation fights, even the booing at graduations, the literal assassination attempts — people in Silicon Valley have become much more worried.”

He was the AI czar — he’s no longer the AI czar. But it turned out a lot of other constituencies, both on the left and the right, were pretty opposed. For example, Trump and David Sacks tried to introduce a big federal bill that would preempt all state-level AI regulation — no state could regulate AI for 10 years. They tried to sneak it into a big omnibus bill so no one would notice. But members of Congress realized it was happening, and — whether for kid-safety reasons or frontier-safety reasons — people said, Wait a second, the idea of preventing any state from regulating AI for ten years is crazy. A lot of people organized in Washington to successfully stop that preemption. I think that showed the scale and bipartisanship of a coalition that was very keen to make sure it stayed possible to regulate AI was underestimated. As a result of all this — the moratoriums, the regulation fights, even the booing at graduations, the literal assassination attempts — people in Silicon Valley have become much more worried.

China is our big competitor in the AI race, and it certainly has all the conditions for a populist pushback to AI — youth unemployment is really high, and AI technology is in some ways more advanced at taking over real-world jobs. I was watching a video about fully automated factories and a robot pharmacist. You’re one of the rare American tech reporters who gets to spend time in China, and you wrote a piece that surprised me — you found there wasn’t really a populist backlash to AI there. Why not?

I was really interested in this question, and I was finishing my New York Times piece while in China for a few weeks, talking to both AI people and non-AI people. The main reason there’s not a big populist backlash in China is that there isn’t a lot of social unrest or populist backlash against anything — the entire MO of the Chinese government, the No. 1 priority, is domestic social stability. Any whisper of protest gets shut down; that’s why they have such strong speech controls. So one factor is that China doesn’t have much of a culture of resistance in general, whether in workplaces or politically. I’m not saying no one dissents — but it has a cultural effect too, because people don’t see it as useful or as an option. When I ask family members of mine in China about AI, sometimes they’re annoyed about specific things, but fundamentally, the idea of opposing AI is seen as almost unimaginable.

The other thing about China is that if you’re middle-aged there, you’ve lived through so many political, economic, and technological revolutions in your lifetime. When I was a little kid visiting Shanghai in the mid-2000s, there were no high-speed trains — now China has some of the best high-speed rail systems in the world. Technology has always gone hand in hand with dramatic economic advancement, with being lifted out of poverty. The modernization process has been aggressive and disruptive, but it’s not something the party has offered opportunities to resist, and it’s something most Chinese people still see as an inevitability that was mostly good for most people — because incomes did increase by dramatic amounts alongside the technological change. So I think people have a similar attitude toward AI: It’s much less about “Can I stop the AI wave?” and more “How can I take advantage of the AI wave to get ahead economically?”

We were just talking about this deep pessimism about what technology can bring us here in the US. I think a lot of people look around and think: We don’t have a cure for cancer yet, but we’ve sure seen our lives get worse in a lot of ways because of technology, social media, whatever. That pessimism probably fuels the backlash to AI, the skepticism about whether it can ever deliver on its promises. And that experience just isn’t the same in China, or probably much of the rest of the world, where technological progress has been faster and more concrete in people’s lives.

My 90-year-old grandfather said he’d love an elder-care robot to help him do tasks around the house so he doesn’t have to rely on his kids — he wants more freedom and mobility. It’s seen more as a tool to help individual goals. Even with the robot factories or pharmacies — one thing that struck me visiting a robot pharmacy was that the PR people happily said, “Yep, we’re doing these robots because human workers take too many smoke breaks and bathroom breaks and take too long.”

You’d never say that in the US, but they’re probably thinking the same thing — they just don’t say it. The other thing they mentioned is that this lets the pharmacy operate 24/7, because a lot of people need medications in the middle of the night and want to order via the DoorDash equivalent. There was actually a labor shortage before — Chinese workers weren’t willing to work night shifts — so these pharmacies are offering real consumer surplus. A significant percentage of orders come in overnight, when no other pharmacy is open. And with the factories, part of the issue is that Chinese workers, especially young people, don’t want to do factory work anymore.

“I think the 2028 presidential primary and election is really where I expect AI to become a centerpiece of the conversation.”

That anecdote gets at the promise and peril of AI, and the role of the backlash movement — which I’m still wrestling with how I feel about. On the one hand, I want to live in a world where cancer gets cured, where we live in an era of abundance, where things are cheap and easy to make because factories can run all the time with machine workers who don’t require anything — I want the future we were promised, of flying cars and everything working well.

But I also don’t want to lose my job, or see humanity wiped out by an angry machine god. Because we don’t really know what’s going to happen yet, it’s hard to work out my own feelings about the pushback here in the States — what’s appropriate, and what’s holding us back from real advances in our lives.

Totally, I agree. I like Waymos — I think they’re safer, and I’d prefer a safer robot car driv[ing] me around instead of me driving. I’m not a good driver; no one should let me drive. So I wrestle with some of the same things.

To close out the conversation — let’s come back to the United States. AI populism is brewing as a political force. We’ve seen it show up in a couple of races so far, but it’s early. We’ve got the midterms, then the presidential election. How do you think it’s going to affect American politics this November, and in 2028?

My sense is that this November, it’s going to be more about state and local races where AI really shows up. I’m going to spend some time in Michigan and Wisconsin this summer touring some of the data center sites facing the most opposition — those states also have contested governor and Senate races where AI and data centers have become a core issue, so I’m interested to learn more there. I think the 2028 presidential primary and election is really where I expect AI to become a centerpiece of the conversation — especially if we start to see some of the employment impacts people are expecting. As soon as we see something like a 2 percent rise in unemployment, if that happens, I think people will be very upset, and we should expect a ton of focus on the issue.

The other thing I’ll note is political opportunism — you’re already seeing a bit of this, where politicians are likely to raise the salience of AI above where people might ordinarily care about it, because it’s become a convenient boogeyman. It polls so poorly, people are so anti-AI and anti-data-center, AI billionaires are so unsympathetic, that no matter what your policy program is, AI is a great reason to push it. I think a lot of politicians who are being clever about this are going to move AI to the center of the conversation, raising its salience to manufacture urgency for proposals they’re already excited about. That’s definitely something I’m watching for 2028.

When you ask people when they knew Covid was going to be a huge deal, they give a range of answers. “When Tom Hanks got sick” is a popular one. So is “when the NBA suspended the season.” The most plugged-in people will sometimes cite early rumblings from Wuhan in December 2019/January 2020.

For me, the turning point came on March 17, 2020, when Republican Sen. Tom Cotton proposed sending every American checks from the government.

To be clear, at this point, my then-employer Vox had already sent everyone to work from home indefinitely, and it was clear something dramatic was happening. But I hadn’t yet internalized that the Overton Window in American politics had shifted dramatically. 

True, there were some Republican Senators who, by 2020, were expressing more openness to safety net programs, and rethinking Reagan-style laissez-faire economics. Tom Cotton, though, was not one of these senators. I didn’t think he really had strong economic policy opinions at all; he was a defense and culture war guy. He cared about defeating China and, secondarily, defeating Woke. Universal cash handouts were not his bag. And yet here was Cotton, not just calling for near-universal cash payments, but also for welfare work requirements to be suspended and for big block grants to states to expand unemployment insurance. 

This turned out to be an early indication of the actual policy the US would pursue. Within a couple of weeks, with the US unemployment rate fast headed for what would be a record high of 14.7 percent in April, a Republican Senate and president had signed off on the CARES Act, which included payments of up to $1,200 per eligible adult, $2,400 for eligible married couples, and $500 per qualifying child, along with a $600 per week unemployment insurance and a massive business bailout program. The Senate vote was unanimous, and the House approved the final Senate amendment by voice vote. 

If you had told me literally any of that would happen in February 2020, I would have laughed at you. But the normal rules had stopped applying. All that was solid had melted into air. Much, much bigger things were, suddenly, possible.

I’ve been thinking about that moment a lot as advanced AI models grow more and more capable, and more and more central to many businesses’ strategies. As of May, Anthropic is reporting an annualized revenue rate of $47 billion, equaling the likes of Coca-Cola and exceeding Netflix. That’s up from $30 billion a month earlier. If their revenue keeps growing at 56.7 percent a month, they will outpace Amazon, currently the highest-revenue company in the world at $717 billion a year, by late November or early December. The AI boom is already unfolding faster than the internet or mobile booms before it and may yet speed up even further. The debate over whether this tech is real and valuable is, essentially, over. The only question is what, and how large, its effects on our lives will be. 

This is happening unbelievably fast, and it seems likelier and likelier that we will face a moment, like that in March 2020, when the speed and disruption of AI progress begins to constitute an emergency that policymakers will be willing to take surprisingly large risks to confront. There will likely be a moment of unusual policy freedom and flexibility, a moment which is brief — but could enable large changes for the better.

The US is currently not ready for that moment. But we need to get ready, fast. And we need your help. My colleagues at the Center for Shared AI Prosperity, a new DC-based research group, are attempting to collect a menu of detailed policy ideas that can meet this moment. In fact, we have an open Request for Ideas with funding that can go to the best proposals people submit for how to set up the tax code and safety net in a way fit for the AI era. Now is the time to act.

These moments don’t last forever

I sometimes talk to friends in the tech world who assume that the power and economic impact of advanced AI will permanently shift our politics, and that the policies necessary to keep everyone afloat (like, say, a guaranteed income, or a sovereign wealth fund) will materialize without much effort. After some 17 years as a journalist covering US politics and policy, I think this is overly optimistic, so say the least. Congress is like jello: flick it and it will shake, but it eventually settles back to normal.

Take Covid. Within a couple of months, the apparent consensus had evaporated, and Republicans were back to resisting safety net expansion. By May, Cotton had pivoted to pushing the No Bailouts for Illegal Aliens Act, which “amends the CARES Act to prohibit sending future funds to states or municipalities until they certify they aren’t issuing stimulus checks or other payments to those in the United States illegally.” By August he had a bill to deny virus-related federal employment funds to people convicted of federal offenses because of “riots.” The pandemic was still raging but the policy emergency, and the bipartisan window for much larger-scale action, had mostly closed.

The 2008 financial crisis offers another example. There, the window was open somewhat longer. At the very beginning of the recession, in February 2008, the Bush administration went against its normal laissez-faire commitments and supported a stimulus package championed by then-Speaker Nancy Pelosi built around per-person checks to nearly all Americans, including many of those not owing income tax. In July, President George W. Bush signed a bailout of Fannie Mae and Freddie Mac in the face of strong opposition from fellow Republicans in the House, but having mostly won over his party in the Senate.

In September, when Lehman Brothers collapsed and the possibility of a cascade of massive bank failures seemed very real, Bush demanded a sweeping $700 billion bailout that proposed purchasing toxic assets from at-risk banks (the “Troubled Asset Relief Program,” or TARP). As the subsequent years would demonstrate, bailing out banks failing due to their own irresponsibility was not exactly a popular position in the general public. Members of Congress are not stupid, and they realized this at the time. On September 29, the House voted down the proposal, with huge numbers of both parties defecting from Bush and Pelosi’s position. That led to a large stock sell-off that terrified lawmakers. That experience, some last-minute tweaks, and truly herculean lobbying from the administration, the Fed, and others led the House to switch course and pass the bill on October 3, though within weeks of its passage, Treasury abandoned asset purchases in favor of buying equity stakes in the banks directly.

The full course of 2008 shows the value of, and power inherent in, being prepared. The February 2008 stimulus package was very roughly improvised. It worked a little bit, but proved nowhere near big enough. If Pelosi and Bush had had a more thought-through proposal on hand, perhaps one that automatically repeated and scaled the checks depending on where the unemployment rate went, then the recession would have been much less severe and the 2009 stimulus might not have proven necessary.

TARP was an example of a case where some key actors were prepared. The structure of the program came from the “Break the Glass Plan,” a proposal put together by Bush Treasury officials Neel Kashkari and Philip Swagel in April 2008 explicitly designed as a “just in case” plan for the extreme situation where the whole financial sector needed recapitalization. That case, of course, came to pass, and because Kashkari and Swagel had a plan, there was something for Congress to quickly pass. That was good — TARP played an important role in preventing the financial crisis from worsening.

But it also meant that the plan reflected Kashkari, Swagel, and their boss Hank Paulson’s overall conservative worldview. One could imagine a plan like that which saw the US government instead outright nationalizing major banks, or imposing strict capital requirements on them in perpetuity as a condition of the bailout money, or banning them from owning hedge funds or doing speculative trading. A different administration with different views might have designed a different emergency plan — and because it was genuinely an emergency, that plan would likely have passed, with very different consequences over the next few years. 

What stocking the shelves for AI means

One way to think of the project of AI economic policy in 2026 is as designing the equivalent of the Kashkari-Swagel plan: something detailed, opinionated, and actionable that can be deployed quickly when the situation gets dire. What that plan looks like will, of course, depend on one’s values and commitments; the America First Policy Institute’s emergency plan will not look like the AFL-CIO’s.

The Center for Shared AI Prosperity was founded with an aim to produce plans of this nature designed to make sure any economic windfall from AI is widely shared, and that workers and low-income Americans are not left behind in the transition. We were also founded out of a frustration at the inadequacy of the proposals we were seeing from two ends of the AI policy debate.

On the one side are ideas from the AI labs themselves. These tend to be ambitious — indeed ambitious enough to seem like plausible answers to a problem of the magnitude of AI completely reshaping the economy — but woefully unspecific. They more closely resemble dorm-room philosophizing rather than legislative drafting.

OpenAI’s “Industrial Policy for the Intelligence Age” from this past April, is one such example,  laying out a number of very broad ideas: taxing capital more, a sovereign wealth fund invested in the AI economy, portable job benefits. It’s light on the specifics: What kinds of capital taxes? How big a hike is too big? How do you make health benefits portable without disrupting people’s current plans? How does the sovereign wealth fund get its money? Anthropic’s Economic Policy Framework is somewhat more specific, offering paragraphs per idea where OpenAI has a sentence or two, but still nowhere near the level of detail necessary to actually write legislation.

On the other side are proposals from within the DC policymaking world, which are firmly rooted in what seems politically viable right now but would be woefully inadequate in the face of the likely economic disruption that’s coming. Former Commerce Secretary Gina Raimondo and her group RAISE US have centered employee retraining; Raimondo’s recent New York Times op-ed centered ideas like new credentials from community colleges and expanded apprenticeship programs as the answer to mass AI unemployment. These are sensible tools for ordinary labor-market churn, but they are mismatched to a transition that could displace whole categories of work on a compressed timeline. The dawn of machine intelligence will demand more from our leaders than certificate programs.

The best case for this kind of caution is that ideas on the scale of the labs — sovereign wealth funds, universal capital accounts for all Americans, permanent relief funds for the long-term unemployed — are dead in the water in DC. Which might be true — now, at least. 

But this is where Tom Cotton’s brief love of cash transfers becomes relevant. We should not overindex on the way the politics look right now. The world is about to become very strange, and we may be surprised by the scale of change in response that can earn even bipartisan support.

Indeed, it’s notable that both the 2008 relief measures and the 2020 CARES Act came under Republican presidents with Democrats controlling at least one chamber in Congress, which is also the likely situation after the midterms this year. Democrats are always willing to vote for big new safety net programs to protect unemployed and low-income people. But Republicans are often willing to compromise their usual anti-welfare stances when they’re the party in the White House, and their approval ratings depend on the country’s basic economic health.

What action they might take in a 2027 or 2028 featuring massive AI-based economic disruption is still unclear. But right now, we all have an opportunity to help shape it. The Center for Shared AI Prosperity is running a request for ideas, seeking proposals for shared AI ownership, new AI-related taxes and revenue raisers, and new safety net programs to share the gains widely. We want ideas from economists and think tanks, of course — but also from the labs, from independent researchers and academics, and from ordinary citizens with an interest in where this technology is going.

Stocking the shelves is hard work, and we don’t have all the answers. But you just might, and we’re going to need all the help we can get if the US is going to emerge from the AI transition as a prosperous, functional nation.

This story was originally published in The Highlight. To get access to member-exclusive stories like this every month, become a Vox Member today.

Derek Lam has more than 31,000 followers on TikTok and nearly 40,000 on X as of this writing. He is shirtless a lot, he dances a lot, and he is shirtless dancing a lot, which may explain how he got so many fans. His comments are filled with compliments (“beautiful”) in different languages (“hombre bello y sensual”) and superlatives (“this might be the finest man on the internet”) accompanied by different emoji (red hearts, crying laughing, lips). Their responses make it seem like Derek Lam is the first and only beautiful man they’ve ever seen, which may explain why he is also selling “exclusive,” seemingly adult, content. 

He is also, possibly unbeknownst to his many admirers, AI-generated. 

To be fair, there were some signs that this man was not real: Despite the multiple videos, Derek never speaks. His videos are also rather brief, just seconds long. A real hot person probably would have parlayed a following of this size into brand deals or “get ready with me” videos. And the selfies on his X account show a completely different man just three years ago. 

Still, the followers of Derek I talked to didn’t even notice he was AI because he seemed to blend in so seamlessly with the other hot men on the internet.

Derek isn’t the only AI thirst trap showing off defined abs for likes and money. He’s one of an increasing number of completely fake, AI-generated figures sinking their fangs into the real models, influencers, and porn stars who populate our feeds, sucking up their beautiful faces and bodies, and using them to profit, without a penny going to the real humans they fed from. 

When it comes to the damage AI could wreak on society, an army of Dereks tricking horny people into giving him likes — or, worst case, money and Amazon gift cards — doesn’t exactly sound like the singularity doomsday scenario that we’ve been warned about. It’s clearly unfortunate for the adult entertainers competing with deepfakes and a fraud risk for their fans, but one might believe if they don’t fall into one of these two groups, they’re relatively safe and unaffected. 

But there’s something more going on here. History shows that porn and sex drive innovation in the tech industry. The way tech platforms treat sex workers is typically a glimpse into the future, and a warning about how tech platforms will eventually treat all of us. If human desire demands the capability to steal, loot, and turn anyone and everyone into something for sale — possibly into hot Dereks — is anyone safe?

The Dereks of the internet are a bleak look at what’s happening in the real world: nothing belongs to us anymore — not our looks, our beauty, our sex, and our art. Our most human desires are slowly being synthesized, with or without our consent. And AI is making it all possible.  

Deepfake technology has gotten alarmingly good in recent years

Artificial hots like Derek are considered “deepfakes,” an umbrella term for AI-generated media (audio, video, or both) that resembles a real-life person. 

When deepfakes first started appearing in late 2017, they were fairly low-quality, making it easy to tell when someone had used a rudimentary app to paste a celebrity or politician’s head onto a different body. Still, it wasn’t very long until people started wielding this technology to be nasty. 

“The first set of deepfakes were actually used to create pornographic videos. They replaced the subjects in those videos with the faces of celebrities,” Siwei Lyu, a professor at the University at Buffalo who studies digital forensics, told me. 

Because the quality of those videos was bad and the content was often absurd or unrealistic, it was easy to tell they weren’t real. Those clunky apps needed a lot of data — videos, images, etc. — of real people to produce crappy videos; Lyu explained that this is why you mostly only saw deepfakes of politicians and celebrities at the time.

As the technology got better, it became less reliant on having a huge amount of data. Instead of needing a whole archive, the new versions of these apps can pretty much run on nothing. “They do not need that much data to train a model anymore. Some of the most recent algorithms just need a single picture — just a single picture of someone,” Lyu said. And the quality is better too. Lyu said that there are AI programs that can now change a person’s appearance and voice in real time, like in Facetimes and Zooms or on live broadcasts.  

Given how many of us are constantly posting photos and videos online, it is now extremely easy to create a convincing social media presence for a person who is not real, and to use it to catfish unwitting people on the internet. 

“This is the problem. It’s becoming more and more challenging to visually tell deepfakes apart,” Lyu said. “Seven years ago, when I started working in this area, checking them was not this difficult,” he added. 

Lyu is an expert in digital media forensics and machine learning, and he went through one of Derek’s videos frame by frame and pointed out some obvious AI tells. There was a distorted watchface with weird swirls instead of numbers and a moment in the video where all of Derek’s fingers on one hand were the same length. Lyu also pointed out that Derek’s chest hair fluctuates, appearing dense in one frame and then dissipating in another.

Through social media, I attempted to contact the owner of Derek Lam’s account with evidence from Lyu that these videos are artificial; I did not hear back.

During my deep dive into Derek Lam’s social media presence, I looked at the accounts he was following. I noticed that of those accounts, someone who goes by the name Vance Ford also had tens of thousands of followers and had nearly identical videos to Derek. The flexing, dances, movements, and music they were set to were all the same, but with what appeared to be a different man performing them. 

I attempted to contact Vance through DMs on social media and did not get a response. I also e-mailed two models who appear to be the actual people that the Derek and Vance AI personas were trained on, but they didn’t respond. 

I sent two of Vance’s videos to Lyu, who analyzed them manually and with AI-detection software. He confirmed that “their movements are nearly identical — consistent with generation from a shared motion source,” and noted that the Vance videos had moments of distortion, unintelligible text, and facial warping. 

 “Young Magnum PI…Tom Selleck,” commented one admirer.

What happens when real people follow fake hots 

“Wow I’m a boomer,” said Patrick, one of Derek’s followers on X, after I told him that he might be following an AI-generated thirst account. (Vox agreed to let Patrick, and Derek’s other followers, use a pseudonym so they could speak frankly about being thirsty for a fake guy.) Prior to our chat, Patrick had no idea Derek was likely a deepfake, and maintains that he didn’t even know he was following the account. Patrick is 33 years old, roughly 30 years younger than the youngest boomer, but being fooled by a hot AI man has made him feel old and vulnerable, susceptible to scams and perhaps light financial crime. 

“This was probably some smut account I followed before I moved all that over to an alt,” Patrick said, noting that in daily life, he’s only ever used AI to help organize and write emails. Wielding AI to create fake videos and photos does not thrill him, nor does the potential of seeing more of Derek. 

“A person being real, someone you could run into at a bar, is half the fun,” Patrick told me, explaining some of the accounts he follows. “AI porn is not of interest, to me, anyway.” 

Not being able to tell the difference between the real beautiful men on the internet and the AI-generated beautiful men on the internet not only makes Patrick feel old, but also a bit “hollow.” The fact that the people we are attracted to are so unrealistically hot, so perfect, that machines can step in for them and go relatively undetected is a reflection of the current state of unattainable desire, which is just as scary as how good these programs have gotten at mimicry. 

“Black mirror shit,” Patrick said. 

The guys I DMed about Derek felt ashamed once they found out the truth. 

“It’s embarrassing and he’s not my type,” said Chris, 33. “I’ve come across several AI accounts, and this one is really good, I have to say. But you can see there’s like no life in his eyes.”

Chris made clear to me that the humiliating thing isn’t that he follows attractive men on the internet. That isn’t a big deal. 

What irks him that he got duped. Chris works in digital marketing and has seen AI used professionally to tabulate calculations for campaigns, and has used it privately for silly things like memes. “AI can do a lot of things, things we probably should not want it to do,” he told me. “I think what’s also scary…is that everybody has access to it. And yes I already unfollowed this person.”

Chris believes there’s something more nefarious afoot. He thinks that whoever is running Derek may have hijacked the username (i.e., the original person Chris was following) and then populated it with AI to drive up follower counts — a scam he’s seen online before.  

“This is super concerning and super scary because you eventually could be texting with this person,” he said, describing a hypothetical situation where unknowing users could be lured into subscribing to fake content and, ultimately, giving the account their personal information, whether that’s photos or perhaps even passwords. 

“This person could be selling your nudes,” he said, explaining one extreme end point of a possible scam. “But you were like jacking off to AI content and that’s embarrassing.”

AI deepfakes are bad for real thirst traps too

While flirting with or masturbating to a fake person is awkward but ultimately manageable and private, Cherie DeVille has an even more complicated problem with AI manipulation. If DeVille is scrolling social media, there’s usually a chance that she’s running into an AI version of herself saying things she’s never said and doing things she’s never done.   

DeVille, an adult star who calls herself “The Internet’s Stepmom,” has roughly 4.5 million followers on Instagram. But her account is often down, which she says is the work of fraudsters  that are determined to send traffic to DeVille’s AI imposters and get her actual account removed. 

“It’s almost always the fake accounts of me reporting me,” DeVille said. “They want to be the biggest me. They want to be the biggest scammer. They want to use my altered AI images to scam fans without my real account getting in the way.” 

DeVille and others I spoke to explained to me that deepfakes have been an annoying reality in the adult entertainment industry for years. The way the scam goes is that someone would fake photos or videos of DeVille (or any star), create an impostor profile, and then trick DeVille’s fans (e.g., through social media DMs) into following that copycat. Later they’d squeeze them for money, payments through Paypal, or Amazon gift cards, perhaps by offering unique content. 

“If you made a fake me and I don’t do double anal, but my AI can, they could have all kinds of ‘exclusive’ stuff,” DeVille said, explaining that double anal is grueling work. 

The lack of protections becomes even clearer when you consider that not every deepfake is a carbon copy. Some personas may borrow a face from one actress, a torso from another, or a pair of legs from a different star. This can make fakes tougher to track down and prove, and more difficult to fight from a legal aspect. 

“Who owns your face once it’s scraped into AI systems? Who profits from your digital clone? How do performers protect themselves from unauthorized replicas or manipulated content?” Rachel Steele, an adult star and the CEO of Red MILF Productions, said to me in an email. “Those questions are still very unanswered.”

Like DeVille, Steele worries about how many of the people using AI to create and consume content don’t seem to consider the artists, models, writers, performers, etc. that these engines have been trained on. It’s bad enough to watch AI slurp up and regurgitate your written work or your digital art. Some people also have to contend with LLMs that have been trained on their own faces and bodies.

“Real creators are competing against characters that can be flawless in every image, never age, never have bad lighting, never get tired, and can appear available 24/7,” Raissa Bellini, an OnlyFans creator who touts gymnastics and firebreathing among her unique skills, told me of the impossibility of keeping up with a machine. She explained to me that she’s seen people create AI-generated personas with the looks of popular models or influencers, only tweaking small details like hair color or eye color. 

A spokesperson for OnlyFans told Vox via email that the company’s terms of service prohibit deceptive or inappropriate content, and said that all content posted on OnlyFans must belong to a verified 18+ OnlyFans content creator: “This means that you can only share content which has been generated, altered or enhanced by AI if it clearly features the verified OnlyFans creator and the user can tell that the content has been generated, altered or enhanced by AI.”

Bellini explained to me that while OnlyFans has measures to protect its creators, some smaller subscription and adult-content platforms do not have the same kind of guardrails. She also noted that most social media sites do not have strict rules or enforcement when it comes to AI, and that she’s seen the algorithm appear to favor AI over human creators.   

“AI raises questions not only about competition, but also about likeness rights, authenticity, audience expectations, and what happens when fans can no longer easily tell the difference between a real person and a generated character,” Bellini added. 

What’s stopping a stranger from creating an AI thirst trap of you? Nothing, really. 

For Deville, Steele, Bellini, their cohort, and even you and I, there are minimal protections stopping someone creating an AI us and making money off of these fake variants. 

According to Jason Schultz, a law professor and director of NYU’s Technology Law & Policy Clinic, humans have, for the last couple of centuries, generally been protected by copyright and right of publicity laws. 

AI obviously didn’t exist when these laws were written, and courts now have to interpret the laws in the context of all of this new technology, in combination with other existing rights (like free speech). Schultz told me that there are more than 100 current cases pending about training AI with copyrighted material. 

He also explained the difficulty of determining whether or not an AI-generated persona constitutes a violation of someone’s right of publicity. It’s more clear-cut when the human involved is a celebrity, because their public persona and appearance is so distinct. It gets murkier when the humans aren’t well known, and the AI creates a persona that’s more of an amalgam than a one-to-one copy. 

“It would raise this question of whether these avatars are based on a particular entertainer, or are they more of an aggregate?” Schultz explained to me. But even if courts side with the humans whose likenesses are being used to create fake personas, Schultz cautions that the technology will always accelerate faster than court decisions are handed down. “I think that the thing that worries me a little is we’re going to get these sets of decisions in two years, but we’ll be dealing with the next three generations of technologies,” he said.  

DeVille, who has been working in the industry for nearly two decades, told me that without better legal protection, she isn’t hopeful for the future of porn or, more broadly, any type of art.

“If my income started tanking and their theft was at the point where I couldn’t compete with literally myself, there might be no choice but to retire,” DeVille said. 

But she also wants to make it extremely clear that she isn’t against AI; she would just like to be in control of it. That means being able to own her likeness, her voice, her image, and the ability to choose whatever she wanted to do with it — or at least get some compensation or have some legal protection if someone’s using Cherie DeVille without her permission. 

“It would be a beautiful way to extend my career beyond what my knees can take,” DeVille told me. But, she added, “if someone’s making an AI of me doing double anal, I should be making the money.” 

If you want to support knife steel research, come join us at Patreon.com/KnifeSteelNerds. You get articles and videos before everyone else!

If you’re looking for Knife Steel Nerds Christmas gifts, some good choices are the book Knife Engineering, the book The Story of Knife Steel, and the Buck knife model 501 “The Larrin.”

Video

There is a video version of the following information:

Why Analyze AI Capabilities?

I’ve had multiple people over the past couple years send me steel compositions that ChatGPT came up with. People want to know what I think, and if AI is going to be taking over everything now, including the design of new steels. I took a few of the steel compositions people sent me so I can analyze them to see if they would work. Can ChatGPT come up with new and fresh ideas to revolutionize knife steel? Let’s look at them and see:

Infinisteel

ChatGPT has some grandiose claims for this first steel:

Introducing the revolutionary super steel, InfiniSteel! This cutting-edge material boasts unparalleled toughness, edge retention, and corrosion resistance, surpassing any other steel on the market. By exploiting a unique interplay between various alloying elements, InfiniSteel exhibits remarkable properties that set it apart from conventional steel grades.

The InfiniSteel composition consists of:

Iron (Fe): 75%
Carbon (C): 1.2%
Chromium (Cr): 20%
Vanadium (V): 2.5%
Molybdenum (Mo): 0.8%
Tungsten (W): 0.5%

The biggest issue with this steel is that it won’t harden. When heat treating knife steel the main step is “austenitizing,” the high temperature step before the steel is quenched (rapidly cooled). You can read more about austenitizing in this article. It is called the austenitizing temperature because the steel transforms from the low temperature ferrite phase to the high temperature austenite phase. Ferrite is the normal “phase” of iron at room temperature which has a certain set of properties, such as being magnetic and soft. When you heat up iron to high temperature the atoms rearrange themselves into a different structure called austenite, which has different properties, one of them being that it is not magnetic. When you quench austenite it transforms to the hard steel phase martensite. If you don’t transform to austenite at your heat treatment temperature you will still have soft ferrite after heat treating and thus it will not make a good knife edge.

Some elements are “ferrite stabilizers” and some are “austenite stabilizers.” This steel has a relatively high content of ferrite stabilizers including 20% Cr, 2.5% V, 0.8% Mo, and 0.5% W. This would be ok if there were enough austenite stabilizers, but the only austenite stabilizer in the steel is carbon, and 1.2% is not enough for all of those ferrite stabilizers. ChatGPT is just adding elements in because when it reads random articles online that say that each element does some beneficial thing. It doesn’t know anything about balancing different elements together and it has no idea that it needs to have enough carbon or other austenite stabilizers to go with all of those ferrite stabilizers.

Here’s a detailed explanation of the role each component plays in the alloy and how they contribute to the exceptional performance of InfiniSteel:

Iron (Fe): As the primary element in steel, iron provides the backbone of the material’s structure. It imparts strength and toughness to the alloy, ensuring InfiniSteel can withstand substantial loads and resist deformation under various conditions.

Carbon (C): The addition of 1.2% carbon significantly improves the strength and hardness of the alloy. This results in superior edge retention, allowing InfiniSteel to maintain a sharp edge even after extensive use. Additionally, the high carbon content enables the formation of carbides, which further enhance wear resistance.

Chromium (Cr): Chromium contributes to InfiniSteel’s exceptional corrosion resistance. At 20%, it forms a stable, adherent, and self-healing passive film on the steel’s surface, effectively protecting it from oxidizing agents. This film also enhances the steel’s resistance to various corrosive environments, such as those with high humidity or in contact with aggressive chemicals.

Vanadium (V): Vanadium’s primary function is to refine the steel’s grain structure, leading to enhanced toughness and resistance to impact. At 2.5%, it forms hard vanadium carbides that strengthen the material and improve its edge retention properties. Additionally, vanadium aids in stabilizing the passive film, further enhancing the steel’s corrosion resistance.

Molybdenum (Mo): Molybdenum strengthens the steel matrix and improves its resistance to high-temperature creep and stress rupture. At 0.8%, it also contributes to the formation of a more stable passive film, increasing the corrosion resistance of InfiniSteel.

Tungsten (W): The addition of 0.5% tungsten enhances the high-temperature performance and hardness of InfiniSteel. It forms carbides that provide excellent wear resistance, contributing to the alloy’s outstanding edge retention properties.

Through this unique interplay of alloying elements, InfiniSteel achieves an unprecedented level of performance in terms of toughness, edge retention, and corrosion resistance. The combination of carefully selected constituents enables the alloy to outshine any other steel on the market, making it the ideal choice for a wide range of demanding applications, such as aerospace, automotive, marine, and cutting tools industries.

The explanations of what the different elements are doing are also very dumb and obviously based on random articles from non-experts online. For example, under iron it says that the iron “imparts strength and toughness to the alloy.” This is a very odd thing to say about the steel since steel is made up of iron. Iron is pretty soft before we add other elements to it, especially carbon. So why would you describe iron as imparting “strength”? Sounds like something a dumb AI would write.

It also says that “Vanadium’s primary function is to refine the steel’s grain structure, leading to enhanced toughness and resistance to impact.” Small amounts of vanadium, like 0.1-0.2%, are added to prevent grain growth. In knife steels when you have 0.3% or more added, that is for wear resistance. In that case it is adding wear resistance and reducing toughness. So the claim by the AI that the vanadium is added for enhanced toughness is nonsense.

The AI also says that molybdenum was added for “its resistance to high-temperature creep and stress rupture.” High temperature creep is when a material is held under stress at relatively high temperature for long periods of time. The material slowly deforms, which is called creep. Resisting creep is totally unnecessary in a knife, and the AI is just pulling things from various places to support its choices in a seemingly random way.

No manganese or silicon was added to the steel. These are not always shown on datasheets but are present in all tool steels that I am aware of. I would say it just didn’t mention them, but it also gave the percentage of iron (75%) which leaves no more room for Mn and Si.

User
Please provide a detailed analysis of Infinisteel vs Magnacut?

ChatGPT
InfiniSteel and MagnaCut are both high-performance steels designed for demanding applications. Here’s a detailed analysis comparing their properties and performance:

Composition:

MagnaCut:
Carbon (C): 1.15%
Chromium (Cr): 17.0%
Manganese (Mn): 0.6%
Nickel (Ni): 0.3%
Nitrogen (N): 0.18%
Vanadium (V): 4.0%
Molybdenum (Mo): 2.0%

Comparison:

Toughness: Both InfiniSteel and MagnaCut boast high levels of toughness due to their vanadium content, which refines the grain structure of the steel. InfiniSteel contains 2.5% vanadium, while MagnaCut has a higher vanadium content at 4.0%. The higher vanadium content in MagnaCut might result in slightly better toughness compared to InfiniSteel.

Corrosion resistance: InfiniSteel has a higher chromium content (20%) than MagnaCut (17%). This results in better passive film formation and, consequently, superior corrosion resistance. Additionally, MagnaCut’s nitrogen content (0.18%) further enhances its corrosion resistance. However, InfiniSteel is likely to have the edge in this category due to its higher chromium content.

…

The person who sent me this also asked ChatGPT how the steel would compare to MagnaCut. Somehow ChatGPT managed to get the composition of MagnaCut wrong. MagnaCut has 10.7% chromium but ChatGPT says it has 17%. It also has 0.2% nitrogen as a target, so the 0.18% nitrogen that ChatGPT reports could technically be a nitrogen content that would be seen in an individual bar of steel, but that is not the target composition. And it also reports that MagnaCut has 0.3% nickel, even though nickel is not added to the steel and is not reported in the datasheet.

ChatGPT also repeats its mistake from earlier, saying that vanadium in higher amounts leads to more toughness. So it says that MagnaCut would have higher toughness because it has 4% vanadium instead of the 2.5% in “Infinisteel.” That is not how it works.

My Grade: D-

It is difficult to give the steel high marks when you can’t harden it.

Ultravium Steel

— create a unique composition of elements for a great blade steel that does not exist yet…
ChatGPT:
Creating a unique composition for a great blade steel requires a deep understanding of metallurgy and the desired properties in the resulting steel. While this composition doesn’t exist yet, we can propose an innovative blend of elements to produce an exceptional blade steel:

Name: Ultravium Steel

Composition:

Carbon (C): 2.5%
A higher carbon content contributes to increased hardness, making the blade exceptionally sharp and able to hold an edge for extended periods.
Vanadium (V): 7%
Vanadium adds wear resistance, enhancing the blade’s durability and edge retention. It also aids in fine-grain structure formation, crucial for sharpness.
Molybdenum (Mo): 3%
Molybdenum boosts the steel’s strength, toughness, and resistance to corrosion. It also helps maintain blade sharpness under high-stress conditions.
Tungsten (W): 5%
Tungsten further improves hardness, wear resistance, and overall blade performance. It enhances the steel’s ability to withstand extreme temperatures.
Niobium (Nb): 1.5%
Niobium promotes grain refinement, which contributes to sharpness and overall blade strength. It also adds stability to the steel’s microstructure.
Cobalt (Co): 2%
Cobalt increases the steel’s strength and high-temperature performance, making it ideal for blades subjected to extreme conditions.
Copper (Cu): 0.5%
Copper adds a touch of corrosion resistance to the steel while maintaining its strength and hardness.
Silicon (Si): 1%
Silicon enhances the steel’s toughness and resistance to wear, improving its overall durability.

ChatGPT starts out by saying that a unique composition for knife steel “requires a deep understanding of metallurgy,” which I agree with. However, this composition shows that ChatGPT also does not have this required deep understanding. The approach appears to have been adding large amounts of most of the major elements. This steel is a good example of how ChatGPT “reads” things online that say all the positive things that different elements do. When a layman reads about different elements this is also typically what they see. It says what an element does to improve properties but it doesn’t usually explain the tradeoffs or how elements are balanced or how they work together. It says that molybdenum “helps maintain blade sharpness under high-stress conditions.” I don’t know what that means or how molybdenum is helping. Is a “high stress” condition one where an individual is under attack? Or is “high stress” referring to “engineering stress” where a load is being applied to a knife? It says that tungsten is added “to withstand extreme temperatures.” What kind of temperatures? Is this a knife for use in volcanoes? It says that niobium is for grain refinement, but 1.5% is way more than necessary for grain refinement. That amount is for wear resistance and ChatGPT didn’t even mention wear resistance. Cobalt was added again for “high-temperature performance,” “for blades subjected to extreme conditions.” And again I say that if the temperature is too high for the knife it is way, way too hot for humans. Copper is added for “a touch of corrosion resistance” for some reason.

There is a big omission with this steel, however, which is chromium. ChatGPT didn’t add any of it. This means the steel does not have any corrosion resistance to speak of (that 0.5% copper isn’t doing much). But chromium is also added to steels for other reasons. It is present in nearly all high alloy tool steels, in part so that it has high “hardenability” so that it can be quenched more slowly and still fully harden. Not adding it is very strange for this steel.

Heat Treatment:

Ultravium Steel would undergo a precise heat treatment process, including quenching and tempering, to maximize its hardness, toughness, and fine-grain structure. The ideal hardness level would be around HRC 65-66.
Properties:

Ultravium Steel would be renowned for its exceptional hardness, wear resistance, and edge retention. It would maintain sharpness even under heavy use and extreme conditions.
The steel’s fine-grain structure would contribute to its ability to take and hold a razor-sharp edge.
Superior strength and toughness would make it suitable for a wide range of cutting applications, from fine slicing to heavy-duty chopping.
Ultravium Steel would exhibit excellent corrosion resistance, reducing the need for extensive maintenance and ensuring longevity.
It would be an ideal choice for high-performance knives, cutting tools, and blades used in demanding environments.
Applications:

Chef’s knives: Ultravium Steel would excel in the kitchen, making it a favorite among professional chefs and culinary enthusiasts.
Survival and outdoor knives: Blades made from this steel would prove highly reliable in rugged outdoor conditions.
Industrial cutting tools: Ultravium Steel would be sought after for its ability to withstand heavy use in manufacturing and construction.
Military and tactical knives: Its strength and durability would make it suitable for combat and tactical applications.
Artistic blades: Bladesmiths and artisans would appreciate its ability to hold intricate patterns and designs in damascus or layered steel.
While this hypothetical Ultravium Steel may not exist yet, advancements in metallurgy continue to push the boundaries of what’s possible. Such an innovative blend of elements could revolutionize the world of blade steel, offering exceptional performance and versatility to knife makers and users alike.

ChatGPT then claims that Ultravium would have “excellent corrosion resistance” which it certainly would not have without any chromium. It also says that bladesmiths would like it for “its ability to hold intricate patterns and designed in damascus or layered steel.” I don’t know what it means to “hold” an intricate pattern. But bladesmiths would likely not prefer this steel for Damascus because the high alloy content would make it more difficult to forge weld.

My Grade: C-

Since this steel hardens I will give it a C-. Can’t give higher than that since it skipped chromium while claiming the steel would have high corrosion resistance.

MagnaCut+

— ChatGPT
Hypothetical Composition for “MagnaCut Plus”
• • •
Element Proposed Content

Carbon (C) 1.05%
Chromium (Cr) 11.5%
Vanadium (V) 3.0%
Molybdenum (Mo) 2.0%
Tungsten (W) 0.5%
Nitrogen (N) 0.3%
Niobium (Nb) 0.5%
Cobalt (Co) 1.5%

Expected Improvements
• Wear Resistance: The increase in vanadium and tungsten would lead to harder vanadium and tungsten carbides, resulting in better edge retention.
• Toughness: The slight decrease in carbon and the introduction of cobalt would help maintain or even improve toughness.
• Corrosion Resistance: The increase in chromium and nitrogen should enhance resistance to rust and corrosion.
• Grain Refinement: More niobium would aid in creating a finer grain structure, resulting in better overall balance.
Potential Drawbacks
• Higher Cost: Adding tungsten and cobalt increases the cost of production.
• More Challenging Heat Treatment: The changes may require more precise heat treatment to unlock the full potential of the alloy.
• Diminished Ease of Sharpening: Higher vanadium content can make sharpening slightly more difficult.
This proposed “MagnaCut Plus” would aim to push the limits of what MagnaCut offers without compromising the balanced properties that make it so desirable. It would likely excel in applications requiring even greater edge retention and toughness, such as high-performance cutting tools and hard-use knives.

This steel was requested by the user to be an improvement on MagnaCut. However, ChatGPT offers from very strange reasoning:

“The increase in vanadium and tungsten would lead to harder vanadium and tungsten carbides, resulting in better edge retention.”

The vanadium content was reduced from MagnaCut, down to 3% from 4%. So I don’t know why ChatGPT thinks it has increased vanadium. It also claims that it added tungsten to form “tungsten carbides.” In these types of high alloy steels, adding a tiny amount of tungsten does not lead to the formation of any carbides that are tungsten-rich. The tungsten incorporates into other carbide types such as vanadium-rich or chromium-rich carbides. And those carbides would not be any harder. A small addition of tungsten seems to be a theme for ChatGPT and I don’t really see the point. To be fair, there are several stainless steels from Takefu where they added small amounts of tungsten and claimed similar reasoning. But I have also been on record saying I think that the tungsten is unnecessary in these steels.

“More niobium would aid in creating a finer grain structure.” The niobium content is less than MagnaCut, 0.5% vs 2%. so it does not have “more niobium.”

“The changes may require more precise heat treatment to unlock the full potential of the alloy.” Why? Based on what?

Other criticisms of the design require more in-depth analysis with a thermodynamics software that predicts carbide types and alloy in solution. ChatGPT claims an improvement in corrosion resistance because of the higher chromium, but in fact the chromium in solution is reduced in the “MagnaCut+” alloy because of the other elements. There is also  3-5% chromium carbide in the heat treated structure which reduces corrosion resistance. The improvement in corrosion resistance with MagnaCut came from reducing chromium carbide content to near-zero and this steel dropped that innovation for no reason because ChatGPT doesn’t know what it is doing.

The carbide content of MagnaCut+ and MagnaCut would be similar but MagnaCut+ would have half of its carbide be the larger chromium-rich type, reducing toughness. It also claims that cobalt improves toughness, which it doesn’t.

My Grade: C

The steel can be hardened and would be nearly stainless. But calling it MagnaCut+ while reducing corrosion resistance, wear resistance, and toughness from MagnaCut is bad so it gets a C.

Some General Thoughts

In none of the proposals that ChatGPT gave did it ever say that it would make these steels by powder metallurgy. The very high alloy content of these steels means they would not be very good if made with conventional steelmaking. Or at the very least these would not be an improvement on existing steels if they were not made with powder metallurgy. So to not state explicitly that the steel should be made with powder metallurgy is a big omission.

ChatGPT provided no heat treating information. In one case ChatGPT claimed its new alloy would require “more precise heat treatment” but it did not provide any information on temperature ranges the steel might heat treat from. I very much doubt it could give accurate ranges even if asked. Of course in the one case the steel wouldn’t be able to be hardened at all so it wouldn’t be able to give a temperature that would work.

As I have stated several times, the approach ChatGPT takes is to add a lot of every element. It seems to think that elements only improve steel, so add some of all of them and you have a new super steel. It has no real concept of balancing different elements together or what the drawbacks are for adding different elements. It is basically what happens when “internet experts” try to design their own steel. So maybe this is a win for AI because it is now as smart as “internet experts”?

Could a Different AI Design Knife Steel?

ChatGPT is a “large language model,” meaning it was trained on huge amounts of written text from humans. It uses the statistical information about how humans put words together to generate coherent text. However, the model is not good at collecting data about different things, analyzing and comparing that data to find unique insights and solutions, and then to turn that into information. The other problem is that information about how to design tool steels is not really available on the internet, especially not in a form that ChatGPT could use. To really develop a good model we would need a database of information specific to knife steel design. So not a “general AI” but a “narrow AI” for our specific task. This would still be challenging, because the exact data we want does not really exist. The more data we can feed into an AI model the better the results can be. This is the classic example of “garbage in, garbage out.” A lot of information on tool steels and stainless steels are locked within specific steel companies, or even inside the heads of different metallurgists. The scientific literature, including publicly available journal articles and reported experiments, are not usually in a format that would easily be usable in a large database for an AI. Each study is done with different conditions: they hot rolled the steel with a different amount of reduction, they tested different sized coupons, they reported microstructure values but not mechanical properties, they tested Izod toughness instead of charpy toughness, etc.

For my own development of steel I use several different things, including:

1. Journal articles and books which collect different studies to find trends with different variables. Changes to composition, heat treatment, etc.
2. My own data which I have collected the last several years on Knife Steel Nerds – toughness, corrosion resistance, hardness, edge retention, etc.
3. Thermodynamics software – this software does not use machine learning or AI but uses databases to come up with expected microstructure at different temperatures. When it works perfectly you can know what carbides you have and which elements are “in solution” and how much.
4. Simple models for different properties – I have created simple equations which predict properties and there are some that are reported in journal articles and books. Some that I have created have been reported on this website such as this study on corrosion resistance or this study on edge wear.

With those pieces of information I can come up with approximate property targets and balance elements together. A similar process could be used for a “narrow AI.” Existing thermodynamics software could also be plugged into this AI, which is already happening. I think to do much better than humans would likely require a bigger database of experimental data than we currently have. For example, small tweaks could be made to existing steels such as the content of Mo, Si, Mn, N, etc. Currently, without experimentation by making these variations, we don’t know which changes could potentially lead to improvements. If our database was big enough, perhaps an AI could make predictions about which changes to try and thus reduce the number of experiments. But we are not there yet.

The post Can Artificial Intelligence Design a New Knife Steel? appeared first on Knife Steel Nerds.