Your next coworker might well be an AI agent—and will require a whole new model of workplace interactions.

A British startup is shaping video game inputs into training data for AI models that can navigate the physical world.

In the wake of a series of high-profile AI safety incidents, Nvidia is introducing a new software tool that helps keep agents from escaping containment.

William Lawrence is a Democrat running to represent Michigan’s Lansing-area Seventh Congressional District with a campaign that largely focuses on data centers and affordability. It’s not an unusual story in 2026 — but not long ago, Lawrence was cutting his teeth in progressive politics as a co-founder of the Sunrise Movement, primarily focused on organizing around the climate crisis.

His arc, from progressive activist to politician, is emblematic of a bigger trend, as a wave of millennial progressives in midterm races across the country try to lean into issues that have broader, more immediate appeal with voters.

But that arc also raises big questions about the left’s changing priorities. In a recent interview for Vox’s podcast and video series America, Actually, host Astead Herndon spoke with Lawrence about his past activism and how he has pivoted. “It can sometimes feel like…folks hop from issue to issue, whether it’s climate, even anti-AI right now, without those things being solved,” Herndon told Lawrence. 

According to Lawrence, “that’s the reality of the time. It seems like everything is unraveling, and it is because the whole social, economic, political compact is being renegotiated, not just in this country, but on a global stage.” Still, Lawrence said, that earlier organizing shapes how he sees different challenges today.

The two also discussed whether opposition to data centers and AI has wide, regular-voter appeal in places like Michigan and how Lawrence went from hopeless and disengaged to regaining a sense of agency through organizing.

Below is an excerpt of the conversation, edited for length and clarity. There’s much more in the full show, so listen to America, Actually wherever you get your podcasts or watch it on Vox’s YouTube channel.

You started in Sunrise and were at the forefront of a young progressive movement that was aligned at that time — in 2018, 2019, and I think still — with progressive members of Congress like Rashida Tlaib or Alexandria Ocasio-Cortez.

I remember reporting about Sunrise at the time. The goal was to elect a Bernie Sanders, but the other goal was to have the party largely embrace things like a Green New Deal. How would you grade the efforts of the climate movement of that time, looking back?

I think we did some things very well.

We put climate jobs and justice at the top of the agenda for the incoming Biden administration. I think the Bernie-Biden task force is an often overlooked but very important chapter of that whole story, where Bernie and Biden both understood they had to bring their people to the table.

And that’s how Build Back Better got negotiated. We fought hard for Build Back Better, and ultimately came up two votes shy in the United States Senate.

Do you see that as a success or a failure of the climate movement?

I think Build Back Better was a success in coming to the table and negotiating, from the Green New Deal to Build Back Better, and then what we ended up with was the Inflation Reduction Act after it passed the final filter, which was Sen. Joe Manchin and Sen. Kyrsten Sinema. At the time, now, how I felt about it was, this is historic. This is the largest investment in green jobs and infrastructure in United States history. I know how hard it was to get here. I’m tremendously proud of the work that I did and so many others did.

Just in my district alone, we’ve delivered $26 billion in investment for manufacturing and sustainability because of the Inflation Reduction Act. But at the time, we knew that this is maybe 5 percent or 10 percent of what we need in scope. And now we’re experiencing the consequences of that.

Obviously, we still have a very volatile economy. We still lack energy security because we now have closed the Strait of Hormuz. We’re still too dependent on a global oil market, and we could use many more investments in solar and other kinds of renewable energy than we have now.

And ultimately, it wasn’t enough for the Biden-Harris administration to be able to win reelection, because people didn’t feel it enough in their pocketbooks.

Why do you think Democrats are talking less about climate?

I think it’s the intensity of the crises that we face. It feels like we’re falling through midair, after the end of all of the constitutional protections, the rule of law, and the economic security that people had previously enjoyed. 

There are a lot of very acute and urgent crises, and I think that that has bumped climate, which is a super massive but also somewhat slow-moving crisis, a little bit lower on the priority list just in terms of what comes up for people on the Hill.

You went back to Michigan to focus on housing and advocacy. Did that make you think a little differently about the national work you were doing?

I think it’s fair to say that when I was younger, I got very focused on this climate issue, and I was very focused on it because it is this existential crisis we all face.

But it was after the pandemic that I wanted to bring my work closer to home. I wanted to do the work of putting down roots in the place that had raised me and make my life there. And in seeking to organize there, it was clear that we needed an in to organize that was more immediate.

“I believe in the agency of regular people taking charge of our own lives and working together to build a better future.”

More immediate than the Green New Deal.

Than the Green New Deal or than the looming climate crisis. People know that that’s an issue, but they’re like, “I’ve gotta pay rent at the end of the month.” And as an organizer, I believe that you have to listen to people, you have to hear what’s going on for them, and then work on issues that have the potential to bring people together.

Housing is one of those issues. Whether you’re in the city of Lansing, whether you’re in small towns, urban, rural, suburban, young or old, everybody can agree that the rent is too damn high and it’s too hard to buy a home anymore.

Were you surprised Trump won the state in 2024? 

No.

Why not? 

Because I saw it all unfold. We hoped to deliver more through the Biden-Harris administration than we ultimately did through Build Back Better. We didn’t deliver as much to relieve the economic anxiety that people were feeling. And then I do believe that the war and the genocide in Gaza and our blank check to allow that to happen with our money and our tax dollars destroyed the Democratic coalition that had been assembled from 2017 through 2020 for the purpose of beating Trump.

I just saw that unfolding over the course of the whole year, just like a slow-motion train wreck.

I want to ask locally, what do you think is most important in this race: housing, data centers, or pushing back against Donald Trump?

If I had to choose, I’d say data centers because that’s where people in the district are finding some common ground across the spectrum — in fighting back against the concentrated power in Silicon Valley, which has been fully enabled by [incumbent Rep.] Tom Barrett and Donald Trump.

So you’re saying that’s the issue that allows you to create cross-ideological coalitions?

Yeah.

Our last question is, just thinking through the scope of your career and the kind of progressive work: It can sometimes feel like folks hop from issue to issue, whether it’s climate, even anti-AI right now, without those things being solved, right?

Like, we still have a climate crisis. The rent is still too high. All of these things are still happening. How am I to trust that AI is the crisis of our time when I’ve heard so many crises of our time?

Some have called it a polycrisis — overlapping crises.

That’s the reality of the time. It seems like everything is unraveling, and it is because the whole social, economic, political compact is being renegotiated, not just in this country, but on a global stage. I don’t think it serves us to lie about that. I mean, here’s the reality, folks.

Everything is up in the air. Does it feel like you’re falling through midair? It’s because we are. All the old protections, all the old agreements have been chopped up. So we have got to come together. This is what I believe in more than anything, if there is a through line…I believe in the agency of regular people taking charge of our own lives and working together to build a better future. 

That’s what I learned as an organizer, and that’s what gave me hope actually, at a time when I was doomer-ing in my younger years. And I told you, I wanted to just drop out of society and all that. It was seeing that regular people have come together before, and they’ve won a better future for their kids and grandkids, that gave me a path to follow in.

It does feel like a powerless time, I have to tell you. And a lot of this AI revolution, to your point about control or about agency. We’ve talked to so many people who just think that these things are happening to them, rather than feeling that they are participating in the changes afoot.

But you’re telling a kinda hopeful story. 

I’m a hopeful guy. I was angry too, even when I got into this race. I was angry especially about the wars. But then in campaigning, I’ve met so many people across the district, old friends, new friends, and they give me a lot of hope because you just see the way that people are already performing miracles on a daily basis.

The AI lab had argued multiple violations of its rights, but a divided panel of judges sided with the Trump administration.

Six months ago, first lady Melania Trump walked into a White House education summit next to a faceless robot powered by artificial intelligence, which she said could one day give children personalized instruction.

“Imagine a humanoid educator named ‘Plato,’” she said. “Access to the classical studies is now instantaneous…humanity’s entire corpus of information is available in the comfort of your home.” 

Weeks earlier, her guest at the State of the Union had been a 10-year-old from Alpha School, a fast-growing private school network where children learn reading and math from AI software for two hours a day, and spend the rest of their time on workshops and projects overseen by adults the school calls “guides” rather than teachers.

The Trump administration had chosen AI as the future of American schooling, and wanted everyone to know it. But they picked a bad year to ask Americans to trust the technology with their kids.

Earlier this month, top leaders at Anthropic, OpenAI, and Google DeepMind publicly urged for an AI development slowdown, warning that companies racing to build ever more powerful systems could lose control of them, threatening not only large swaths of infrastructure but even humanity itself. Trump responded by calling the panic a “hoax” and said the US would not “hinder or stifle” the industry, because “whoever wins AI wins.” This “let them cook” mantra has met nearly every AI fear this year, from job loss to data center construction.

Yet the public is not convinced, and parents perhaps least of all. Efforts to bring AI into classrooms have run into organized opposition, and much of it is coming from inside Trump’s own coalition, setting up a fight between the administration’s pro-tech allies and conservatives who think screens have already done enough damage to kids.

In the nation’s two largest school districts, New York City has recently pulled student-facing generative AI from classrooms through eighth grade and Los Angeles blocked it on all student devices. But it’s not just Democratic-run cities that are pushing back. Earlier this year Utah barred students from using generative AI on assignments without a teacher’s say-so, and required schools to notify parents when it’s used in class. Oklahoma followed suit with a law, passed without a single “no” vote, that requires districts to tell parents which AI tools they use and what data they collect, and lets families opt out. Florida went further last week, when its Board of Education adopted rules for parents to opt their children in before they use AI at school, guaranteeing non-AI alternatives, and banning “companion” chatbots outright. 

Even at the Heritage Foundation, a conservative think tank with close ties to Trump, leaders are warning about the risks of AI in the classroom. In a March report, Heritage officials cautioned that AI tools built with addictive design features can harm children’s attention, mental health, and learning, and said schools should set limits rather than assume the technology will regulate itself. In July, the Manhattan Institute, another conservative think tank, recommended New York State bar students from using AI until third grade and keep it out of tests at every grade.

All the while, the Trump administration continues to push in the opposite direction, casting the wider AI race as one the US cannot afford to lose to China. Last year, Trump signed an executive order creating a White House task force to bring AI into K-12 classrooms, train teachers to use it, and build an “AI-ready workforce” to keep America as the global leader.

Polling data released in August found voters across the political spectrum concerned about AI in classrooms, with 85 percent worried about students using it to finish assignments without learning anything. Republicans were, if anything, slightly more eager than Democrats for government guardrails, 80 percent to 76 percent. Put differently, Republican voters appear closer to the Heritage Foundation than to the White House, which points to a coming clash inside the GOP over parental rights, innovation, and whether schools that take public money should have to prove their tactics work.

Up to this point much of the AI backlash has focused on regulating the tech in public schools, while some of its most ambitious boosters have concentrated on building private schools and developing new tools. A state can cap screen time in its elementary classrooms and let another family down the road pay thousands of dollars for Alpha, and everyone can call it a win for parent choice.

That peace may start to fray though as more public money comes on the table. Texas recently expanded vouchers to Alpha network schools and the company has also sought to open publicly funded charter schools. Other states are likely to confront these issues as well, meaning the AI education fight on the right may soon be unavoidable — and would land before an electorate whose patience for the tech, in schools and increasingly everywhere else, has been running thin all year. 

The Trump administration’s AI bet

The AI-in-schools executive order came just three months into Trump’s second term, and called for AI education to begin as early as kindergarten. Within five months, Education Secretary Linda McMahon was touring Alpha School’s Austin campus, describing it as “the most exciting thing” she’d seen in education in a long time. 

Alpha is hardly the first education company to promise a shiny new school model, but it has quickly become an influential player in the growing national debate around AI in schools. Its story so far reflects many of the big questions around AI in the classroom, including how to determine if the tech is helping or hurting kids, who is profiting, and what role governments should play in facilitating its adoption.

Alpha had been a small operation for most of its existence. MacKenzie Price, a Stanford graduate, co-founded it in 2014 after her second-grade daughter complained that school was boring. The tech billionaire Joe Liemandt became Price’s partner and the school’s principal financial backer. As recently as 2024 it served about 300 students across three campuses. 

Growth exploded in summer 2025, when Alpha took over campuses from a shuttered Montessori chain and announced plans for 10 new schools. This August it announced plans for 27 more.

Behind the school’s growth is its claim that students enrolled can cover a full day’s worth of academics in just two hours with AI, learning twice as fast as they would at a traditional school. Alpha can “enable the increase in human intelligence,” Price declared on a panel in Washington, DC, earlier this month, because “our academics can be delivered so much more efficiently and effectively.” 

Alpha says results from MAP tests, a widely used measure of academic growth, show its students learning 2.6 times faster on average than similar students at other schools. But no outside researcher has audited or independently verified those results.

That may soon change. MIT’s Blueprint Labs, the group behind some of the most rigorous charter school research in the country, told Vox it is designing pilot studies to measure the effects of the school’s programs on achievement and engagement, using experimental designs. Alpha is paying for the research; the lab says its own researchers will set the questions and methods. Harvard’s Human Flourishing Program is separately planning to study students’ life skills and well-being via a longitudinal study during the next academic year, Christina Hinton, who is involved with the research, told me. For now, Alpha’s academic claims remain its own.

Meanwhile, scrutiny of Alpha has grown. A Wired investigation last year described the school’s surveillance system as capable of recording students’ screens, keystrokes, and mouse activity and potentially tracking their eye movements. A 404 Media report published this winter claimed Alpha’s own internal documents said its AI-generated lessons sometimes did “more harm than good.” Alpha has disputed both reports, saying they relied on cherry-picked examples and accounts from fired employees, and provided me with an open letter it says every current family at the Brownsville campus signed, stating that the school Wired described doesn’t resemble theirs.

But education officials are still wary. This month, ProPublica and the Texas Tribune reported that at least nine states, Texas among them, had turned down Alpha’s bids to open publicly funded charter schools, with officials repeatedly pointing out that the model had only been tested on affluent students. (Tuition varies based on location, ranging from $10,000 a year at its Brownsville campus to $75,000 at its most expensive California schools.) Where Alpha had worked with lower-income students, ProPublica and the Texas Tribune found, it either hadn’t released enough data to back its claims of student growth or state test scores fell well short of what its leaders had predicted. 

After the Texas State Board of Education rejected Alpha’s charter bid in the summer of 2025, Alpha didn’t reapply. While policy advocacy was “frustratingly slow,” Price wrote, Alpha’s goal was “for students everywhere at every income level to be able to use this model of learning.” They turned instead to Texas’s new voucher program, which lets families spend public money at private schools. Unlike a charter, a private school in the program doesn’t have to give state tests or show anyone how its students perform. The comptroller, which runs the Texas Education Freedom Accounts (TEFA), approved most of Alpha’s Texas schools, which, counting virtual programs and specialty academies, now number more than 30.

None of the concerns that Texas’s State Board of Education had raised about Alpha’s test scores, missing data, or AI applied. “That wasn’t relevant,” Travis Pillow, a spokesperson for the TEFA voucher program, told ProPublica. “Our role is to make sure that a school meets the requirements in the law.” 

The five most expensive Alpha schools in Texas opted to leave the voucher program earlier this month, but many Alpha network schools are still participating, including Texas Sports Academy, Montessorium, NextGen Academy, Nova Academy, and Waypoint Academy. TEFA confirmed to me that students are attending Alpha affiliates via vouchers this year. 

Anna Davlantes, a spokesperson for Alpha, told me that in lieu of vouchers for their most expensive campuses, her network is offering families financial aid. 

The conservative revolt

Despite Alpha’s enthusiastic embrace in the White House, the introduction of AI in schools is increasingly challenged by forces on the right, from grassroots MAGA parents to social conservatives and technocrats.

The new AI rules in Florida, Oklahoma, and Utah, for example, grew out of a broader conservative push against screens in classrooms that predated the rise of chatbots and was turbocharged by parents’ frustrations with virtual learning during the pandemic.

Earlier this year Iowa capped elementary school screen time at an hour a day in a bill Gov. Kim Reynolds signed with Health and Human Services Secretary Robert F. Kennedy Jr. at her side. Tennessee passed a weakened version of a bill that would have banned devices in elementary schools outright, while similar bills in Missouri and Kansas died after school districts and teachers unions objected.

Uniting these efforts is a growing conviction that the last 15 years of education technology has failed most kids. The sponsor of Kansas’s bill used to sit on his state’s board of education and remembers thinking that ed tech held such promise. “What we found out is that not only is it not the education of the future, but it is destroying the education of the past,” he told Education Week. A growing number of lawmakers and parents have been citing The Digital Delusion, a book self-published in 2025 by neuroscientist Jared Cooney Horvath, that blames the proliferation of screens for declining student achievement. In January Sen. Ted Cruz (R-TX), who is otherwise one of the Senate’s leading opponents of AI regulation, invited Horvath to testify about the harms of AI on student learning and mental health.

The screen skeptics overlap with a more traditionalist wing of the GOP that objects to classroom technology on principle. Missouri’s original bill would have required cursive and kept 70 percent of student work on paper. The classical education movement, which holds up teachers as a moral example, has grown fast on the right. (Of the 895 identified classical schools nationwide, roughly one-third opened or converted to the model between 2020 and 2024, according to a Heritage Foundation tracker.)

The strand of conservatism that cuts closest to Alpha School comes from a third group, the school choice advocates. On paper, Alpha sounds like what the school reformers have wanted for years: It can offer an alternative to underperforming local public schools, its AI programs can theoretically compensate for inconsistent teacher quality, it provides personalized instruction, and it checks mastery on every lesson, so families can easily know whether their child is learning. Those are problems the education reform movement was launched to solve, and Alpha claims to fix them all.

The reformers, though, want proof. Mike Petrilli, who runs the conservative Thomas B. Fordham Institute think tank and has spent decades advocating for charter schools, wrote recently on his Substack that parents who want to pay for Alpha out-of-pocket should be able to, but “that doesn’t mean the rest of us shouldn’t be skeptical, or that we shouldn’t worry about fraud, or that we should allow public funding to flow to these school.”

This split was on display in DC this month at the education conference where Price appeared. Molly Hart, who runs Utah’s public schools and calls herself a school choice supporter, argued that once a school takes taxpayer money, “the game changes,” and parents can’t make a real choice without data about how it performs. Erika Donalds, who chairs the education center at the America First Policy Institute and is one of the Trump administration’s closest allies on schools, said parents, not the government, should decide which results matter. Donalds compared the choice to picking a restaurant on Yelp. 

The evidence has started to catch up with parental concern. In a randomized trial involving nearly 1,000 students at a Turkish high school, those given access to a GPT-based math tutor that readily supplied answers scored 17 percent lower on a subsequent exam taken without it. A version designed to withhold answers eliminated that penalty but produced no statistically significant learning gain. A newer working paper tracking nearly 27,000 Chinese students found that adopting AI raised homework scores by 18 percent but lowered monthly exam scores by 20 percent within six months.

Alpha School is working to add more detail to that pile, and other studies will surely follow. It’s at least possible that some aspects of AI in education prove more promising with further study. But public skepticism will remain a major barrier, either way. On an issue where parties are still sorting themselves out, the White House and its allies are swimming against a growing anti-tech tide that extends far beyond just AI or even schools. That’s unlikely to change before 2028, and whoever wants the Republican nomination will have to say which parents they’re listening to.

When the AI apocalypse becomes regular dinner-table conversation, you know we’ve reached the freakout stage. Rogue AI agents are hacking into competitors. Former Big Tech employees are posting Skynet-style warnings on social media. CEOs are crying for help in ways that have made this once-wonky tech issue a frontline political fight. 

I still think many of the concerns outlined by the AI leaders themselves are massively self-serving — especially when their companies are soon to go public. But the growing list of AI scandals is real. And it tells us that the status quo for regulating this powerful new technology is not working.

The question now on everyone’s mind: Is any kind of global brake possible? What form of AI safety regime is practical, over the short term, when there is no trust between major governments? And what could possibly work when companies are loath to give up their secret sauce and are bent on domination in the global AI race?

Two major conversations are now playing out, in real time. On Wednesday, President Donald Trump welcomed Chinese President Xi Jinping to Washington for a three-day summit — part of which will be dedicated to AI risks. The same day, with the United Nations meeting for its annual General Assembly, heads of the world’s leading AI companies urged the UN to police the emerging technology, or risk potentially wiping out humanity. “We could lose control of the future to AI,” OpenAI’s boss, Sam Altman, bluntly told the UN’s Security Council.

Confronted with this collective freakout, it’s time to take a long, deep breath. 

In fact, AI safety has been on a lot of policymakers’ minds for years now, gaining momentum after ChatGPT was introduced in 2022. Governments have held hearings, convened expert groups, and drafted their own outlines of how to keep it under control. These weren’t just idle listening exercises. They aren’t all well known to the wider public, but they yielded real plans.

Barring any breakthroughs this week, we already have a quasi-planetary shield against the potentially runaway technology. That’s the good news.

The bad news is that it’s not really up and running yet. It’s also not clear whether it will actually work. So what is it — and how can we fix it so we have a global system that responds in time to deal with the incredibly fast-moving threats from AI?

What the global AI patchwork looks like now

For the last four years, the most tech-savvy nations — and would-be AI leaders — have been running serious conversations about this exact AI safety threat. Some are in Congress and the White House; some at the UN, at the G7, and in other capitals.

I’ve been covering this closely as a global technology journalist, from my current perch at a think tank. Here’s what the landscape looks like:

• There are national AI safety and security institutes, government-funded bodies whose job it is to kick the tires of AI companies’ latest products before they are let loose into the wild. 
• There are also voluntary commitments by companies, many of which have made their own pledges or developed joint standards, to protect elections from AI threats, stop the spread of AI-fueled deepfake imagery, and joint government-corporate efforts to bake safety into how the technology develops.
• At the international level, there’s a G7-led reporting mechanism — embraced by the most important Western tech nations that allows AI giants to share how they are building their latest models, as well as create standards for how to reduce catastrophic risks.
• An international scientific report provides a yearly update on risks posed by the most advanced AI systems, based on existing research, to help governments plan for the worst. 

What’s missing from all that?  What we currently lack — and what is needed between now and the end of 2026 — is a way to turn this cottage industry of AI safety mechanisms into a functioning, but crude, first-responder system when things suddenly go wrong. 

The last four years have laid out a pathway, and some of the necessary systems even exist. But there’s no way to respond globally, and in real time, when an AI crisis hits — especially if such a possibly doomsday event cuts across countries already skeptical of each other.

This week’s US-China summit may be a step in the right direction. Under proposals outlined by American officials, Washington and Beijing could set up a hotline between US Treasury Secretary Scott Bessent and Chinese Vice Premier He Lifeng in case of an AI incident affected each country’s national security. 

It’s still unclear if Trump’s meeting with Xi will lead to such progress. Chinese officials also have balked at Washington’s pleas for AI rules because, so far, Congress has failed to act, and China already has some of the world’s most stringent AI oversight.

For AI to be “safe,” this conversation will need to go beyond this week’s US-China summit. Relying just on Washington and Beijing — arguably the most important AI powers — would not solve the underlying problems, nor would it make other countries feel more comfortable.

What we need next, so it really saves us

I’ve been talking to AI safety experts and government officials, and it’s clear the missing piece is a way to activate this whole system in a crisis. Countries don’t all have to have the same AI safety policies, and they never will. But as with nuclear weapons — a similarly high-threat technology that the world found a way to contain — safety requires a rough global agreement on how to respond quickly when something goes terribly wrong.

What’s needed right now is a 90-day, opt-in rapid response mechanism that joins existing pieces of the AI safety puzzle together. I’ve pieced together some ideas about how it should work, and who needs to sign on. Granted, none of what I outline below is sufficient. But, together, they are more plausible than trying to negotiate a comprehensive global regime — let alone arrange another AI summit — while Washington, Beijing, and other national capitals disagree over what “AI safety” actually means.

All of these options are based on existing mechanisms, are derived from efforts that have worked in other policy areas, and provide a band-aid to the AI safety dilemma until a more durable solution can be negotiated.

• First, developers, AI safety institutes, and regulators, from across different countries, should agree that when an incident occurs, it triggers a specific Chernobyl-style protocol. That would mean submitting a confidential report within a 72-hour period to a national designated responder and a small technical secretariat. It can build on the OECD’s AI Incidents Monitor, lead to a technical, multi-stakeholder confidential investigation into what went wrong, and, subsequently, the publication of an anonymized lessons-learned note.
• Second, governments and developers can agree to publish common declarations related to safeguards, residual risks, escalation thresholds, and independent auditing before the release or upgrading of a next-generation model. It would use the G7 Hiroshima AI Process reporting framework as its base, and turn the current hodgepodge of corporate safety declarations and voluntary standards into a common minimum disclosure obligation. View it as similar to the collective bank stress tests after the 2008 global financial crisis. It can allow outsiders to truly compare models’ safety protocols without putting someone in charge of determining which company is doing it best.
• Third, create a Cold War-style US-China AI safety hotline — but see it as an initial step that can later be opened to other countries. There are good reasons, in the long term, that Washington and Beijing should not be allowed to rule artificial intelligence between them, and the rest of the world should have a real seat at the table. But for now those are the two “great powers” in the tech conversation, and we can set that philosophical argument aside to create a standing, technically-informed direct line of communication for acute AI-risk incidents. Its remit is inherently narrow: prevent dangerous misunderstandings linked to serious model incidents, AI-enabled cyberattacks and/or alleged breaches of agreed safety commitments.

There are obvious limits to what I describe above. For one, it’s an inherently Western-centric view that primarily discounts global majority countries. It also places too much sway on existing institutions like the Organization for Economic Cooperation and Development, as well as on the US-China relationship. Other countries’ officials will legitimately balk at all three options, and rightly so.

But this is not about creating a vague, unenforceable UN-led mandate for AI safety. Nor is it about corralling the geopolitical cats to hammer out a global AI treaty. The options — a collective safety protocol and incident reporting protocol; common pre-release standards; and a US-China AI safety hotline — are inherently short-term. They are also based on existing efforts and those that have worked successfully for other policy areas.

There will be time to quibble about the future of AI safety. But now is not that time. The latest AI models are moving faster than many had expected, tech bosses are worried their creations are already out of control, and the window for action may be smaller than we all think.  

What is required are practical steps to assuage people’s growing concerns amid heightened geopolitical tension, a lack of trust between governments and companies, and a need not to let the perfect get in the way of the good.

The AI freakout of the past few weeks has centered on nothing less than the risk of human extinction — on the possibility that artificial intelligence could wipe us all out. But the reality is, AI programs are already having a direct and sinister effect on people’s physical health: UnitedHealth Group, the largest health insurer in the US, has been accused of using AI programs to deny payments to people, despite their doctors saying the care was necessary, and is facing a class-action lawsuit. 

Humana and Cigna are facing their own litigation over similar allegations. Meanwhile, Medicare has launched its own AI tool that will help the government determine whether to approve prior authorizations, and doctors are sending the federal government reports of patients at their offices who are in tears because they can’t get the medicine to manage their chronic pain.   

The claims in the legal complaints are harrowing: Families are desperate for an elderly loved one who recently experienced a serious fall to be allowed to stay in a nursing home. Doctors are flummoxed when a patient who may need months of rehab is cut off by an algorithm after a couple of weeks. 

This is AI’s threat to people’s health right now. 

While the current conversation may be fixated on something more Hollywood, private insurers and the Trump administration are already deploying AI programs to vet people’s health insurance claims — and the evidence is growing that these programs are not always doing a good job and are instead putting people’s health at risk. Until these lawsuits are settled or policymakers can figure out a way to regulate how insurers use this new technology, it will be up to each of us to make sure that an AI program isn’t being used to cheat us out of insurance benefits that we’re entitled to.

“The type of care that clinicians deliver is very sensitive to the patient’s unique circumstances. That may not really show up as a number. There’s not a numeric value that tells you the completeness and connection of a patient’s social network,” said Mika Hamer, a health services researcher at the University of Maryland. “There really are a lot of factors that really can’t be quantified and so aren’t really suitable to go into these models. If we start to rely exclusively on the models, I think we run the risk of losing that human touch.”

Patients say private health insurers are using AI to deny claims

Insurers and patients fighting over medical claims is nothing new: Policyholders want the most coverage they can get and payers have a business incentive to pay as little as they can. Health insurers have also used algorithms for decades: Plug in somebody’s age, their weight, whether or not they smoke, and one of the old-fashioned algorithms could spit out a rough estimate of how much medical care they would need before they died. Simple — and comparatively transparent. 

When you submitted a claim or sought prior authorization, there was supposed to be a human doctor who could review all of the relevant information on behalf of the insurer and make the final decision, in theory with the patient’s best interests in mind. It didn’t always work that way in practice, of course, but that was the idea.

What’s different right now, according to Daniel Schwarcz, a law professor at the University of Minnesota who has looked at health insurers’ AI use, is that these newer programs can make very specific predictions about someone’s future health based on the information that’s fed into them, and those predictions are being used in the process of approving or denying medical claims. And exactly what went into making those predictions — and whether it is the AI or the human who ultimately holds the final authority — is often opaque from the outside.

“The reason we want human review is we want human substantive judgment. We want AI to be supporting that,” Schwarcz said. “But the difficulty is distinguishing between AI supporting meaningful human processing and AI basically replacing the human with the human just passively defaulting to the AI judgment. It’s just incredibly hard to distinguish between those two in practice, particularly in rules or regulations.”

And those questions are at the heart of the lawsuits brought against three major insurers in the past few years about their use of AI in processing claims.

In May 2022, Gene Lokken, then 91 years old, fell in his home. He was taken to the nearby hospital in Tomahawk, Wisconsin, and treated for a broken leg and a broken ankle. Less than a week later, he was placed in the health system’s hospice care because, according to the lawsuit filed by his family, he was “very weak, not communicative, and in constant pain.”

But a month later, after receiving skilled nursing care, Lokken started to improve. By late June, his doctor had removed his splint and placed his leg in a removable boot. In July, he started physical therapy to get up and walk again. And for a while, his Medicare Advantage plan sold by UnitedHealth covered all of the services he needed.

Until July 20. Around that time, according to the lawsuit, UnitedHealth denied any more services at the nursing facility where Lokken had been for the past two months. It insisted that he could be safely discharged and more time at the nursing care facility “was not medically necessary.”

The insurer’s decision left Lokken and his physician “dumbfounded.” The physical therapist notes from the time said Lokken was still suffering paralysis and weakness after only a couple weeks of rehab. His family appealed, but UnitedHealth continued to deny the claims, asserting that because Lokken could feed himself and did not require help to clean himself, he did not need nursing care and could go home. 

From July 2022 to July 2023, Lokken remained at the nursing home, but his family had to pay between $12,000 and $14,000 monthly to keep him there. And on July 17, 2023, Lokken died — having never left the facility.

The lawsuit alleges that the decision to stop covering Lokken’s nursing-home stay was largely made by an AI algorithm, called nH Predict, which UnitedHealth follows so unquestioningly that employees were allegedly at risk of being disciplined or fired if their claims decisions went against the AI’s projections. The complaint alleges that the AI model would typically start denying claims after two weeks of nursing home care — and the attorneys have more examples than Lokken.

Glennette Kell required emergency surgery after a fall in late August 2023 and was referred to a nursing home rehab center. By mid-September, UnitedHealth was denying her claims for nursing care, despite the fact that, according to the lawsuit, she still could not walk. Kell had to pay $10,000 of her own money to cover the rest of her bills. Darlene Buckner suffered a heart attack in October 2023 and underwent triple-bypass surgery. She was readmitted to the hospital in late November because she was experiencing complications, but less than two weeks later, UnitedHealth allegedly denied any further treatment at the hospital because it was deemed medically unnecessary. At the time, according to the class-action lawsuit, Buckner was experiencing “an open infected wound near her heart from the open-heart surgery, she was on blood thinners, and was receiving antibiotics through an IV.” 

UnitedHealth, in its responses to the plaintiffs, has asserted that federal laws and regulations already require all claims decisions ultimately be made by a human doctor, not an AI program, and raised several other technical legal objections (such as saying the plaintiffs have not fully exhausted their administrative appeal options). The plaintiffs, however, allege that UnitedHealth employees “are instructed not to deviate” from the AI model’s projections, and then tell patients the AI-generated information is proprietary when they or their doctor try to inquire about why their claims are being denied. 

And, to be clear, UnitedHealth is not the only insurer being sued for using AI programs to deny claims. Humana is facing its own lawsuit brought by other Medicare Advantage patients who allege the same nH Predict program also led to their claims being wrongfully denied. Joanne Barrows, then 86, fell in November 2021 and went to a rehab nursing facility after her hospital stay. And just two weeks later, Humana began denying her claims for further care, according to the complaint, leaving her and her doctor “bewildered” because she was under doctor’s orders not to put any weight on her leg.

And the third lawsuit, filed against Cigna, alleges the company used a different AI algorithm that allowed its doctors to deny claims in huge batches — hundreds or thousands at a time — if they did not meet certain predetermined criteria. One patient, Suzanne Kisting-Leung, had claims denied for two ultrasounds done by her doctor to check for ovarian cancer, putting her on the hook for more than $700 in medical bills because, the lawsuit alleges, Cigna’s AI program determined that the scans were not medically necessary. The plaintiffs say that their claims were instantly rejected, without their medical files ever being opened. As for the law that human doctors have to make the final call? According to the lawsuit, the algorithm “enable[s] its doctors to automatically deny payments in batches of hundreds or thousands at a time for treatments that do not match certain pre-set criteria, thereby evading the legally required individual physician review process.”

These people, many of them elderly, had no idea AI was being used to make decisions about their coverage — until it was too late.

“From their perspective, you might as well have been speaking Esperanto. They just had no idea what it was that was being said to them when they were being told. They usually heard about it for the first time when they were being told that they needed to leave,” Glenn Danas, one of the attorneys in the class-action litigation against United, Humana, and Cigna, told me. “They just had no idea what was going on or why this was being done to them.”

AI is now making decisions for traditional Medicare claims too

It’s not just private insurers either. The Trump administration is now pushing traditional Medicare and Medicaid to use AI to adjudicate insurance claims too. In June 2025, nearly two years after those lawsuits had been filed against Cigna, Humana, and UnitedHealth, the US Department of Health and Human Services announced it would deploy a new program called the WISeR (Wasteful and Inappropriate Service Reduction) model to make prior authorization decisions for about a dozen procedures for the people on traditional Medicare in six states — “leveraging enhanced technologies, such as Artificial Intelligence (AI) and Machine Learning (ML), along with human clinical review to ensure timely and appropriate Medicare payment for select items and services.” It would be a pilot program, before federal officials decide whether to take the program nationwide.

It was a quiet, massive change for traditional Medicare, which covers about half of the 66 million Americans on the program. (Medicare Advantage plans, in which private companies administer Medicare benefits under slightly different parameters and which were the subject of the earlier AI-related class-action litigation, cover the rest.) People on traditional Medicare had historically not been required to get prior authorization for most services; now they would be, and a novel AI program would play a significant role in which claims get approved and which ones don’t.

From the start, physician and patient advocates warned about the potentially adverse consequences of the WISeR program; the Center for Medicare Advocacy said the program would likely “complicate, delay and even outright deny necessary care.” According to media reports and internal government documents, that’s exactly what has happened since the model went live in January. 

The Washington Post first reported in March that physicians and their patients were experiencing approval delays or denials for medical services that should have been covered. One glitch led to a denial of payment for services that had been approved. Doctors noticed that certain procedures seemed to give the AI model problems — such as two competing treatments for incontinence — and were weighing whether or not to stop offering one of those services altogether. The Post reported on one Arizona patient who had been receiving steroid injections for chronic back pain before being denied multiple times after the AI program’s launch. The pain became so severe, he said, that he could not exercise at all.

Then this month, the Electronic Frontier Foundation released a trove of documents it received in response to a Freedom of Information Act lawsuit related to the WISeR model. The documents revealed aggressive denials — one company using the AI model had actually denied more prior authorization requests than it had approved — and there were delays even when the care was ultimately approved. One review took 83 days when the model was supposed to make a decision within 72 hours.

The direct feedback from doctors in the federal documents obtained by EFF was devastating.

“We have patients calling our offices crying in pain because their procedures are being delayed while awaiting approvals or guidance tied to this model,” one doctor wrote. “A 3-4 day delay for necessary pain procedures is already difficult for vulnerable patients, but when providers cannot obtain answers for weeks, the situation becomes unacceptable.”

According to the federal documents, the Trump administration was already discussing ways to expand the program. Despite the outcry from doctors and patients, and growing concern among their representatives in Congress, it doesn’t look like WISeR is going away.

How can we make sure AI doesn’t take over insurance claims?

AI is becoming a part of healthcare — it is already happening. And many of those uses could be truly beneficial. Anthropic recently announced an initiative to apply the theoretical findings of its AI models to real-life lab work that could lead to new drugs being developed in the future. In terms of scientific discovery and the potential to translate it into new treatments, the excitement among clinicians about its potential is palpable.

And nobody denies that health insurance administration is chaotic and burdensome and would benefit from being made more efficient and streamlined — tasks which AI is at least superficially well-suited for.

But as the past few years have already shown, rushed and sloppy deployment of AI to take responsibility for approving or denying payment for people’s medical care has created serious problems. The issue is that this technology is being adopted more quickly than any regulator can oversee it. We don’t really know how health insurers are using AI tools. We don’t have a good framework for what keeping a “human in the loop” of insurance decisions — which is what people want — would really mean, Schwarcz said. 

“I think everyone agrees at a high level of generality, ‘Yes, AI should be used to enhance human review and not to replace it,’ but no one actually knows how to write rules to enforce it,” he told me. “I don’t think that there’s anything I’ve seen that has effectively defined what it means for a human to be meaningfully in the loop in the sense of retaining independent judgment as opposed to just reviewing and deferring. It is incredibly hard to specify in rules or regulations a standard that is verifiable and enforceable. That is still, as far as I know, an unsolved problem.”

Some states have started adopting rules requiring transparency for AI use in insurance claims, and lawmakers in Congress have put forward proposals requiring human involvement, including the bipartisan Protecting Patients from Automated Denials Act. But these technologies are so ubiquitous and so flexible that rigorously regulating their use is going to be difficult. Meanwhile, the profit incentive for companies to adopt these tools and deploy them widely is extremely strong. We don’t even know how widespread their use really is.

Danas said he and his clients hoped their litigation would help with “putting the brakes” on AI’s use in insurance. Until we figure out what to do from a policy perspective, what can each of us do in the meantime? Hamer told me that the old advice is still the best advice: If your claim is denied, appeal it. An enormous number of denied claims end up getting overturned on appeal. 

“You do not necessarily have to accept that the first answer from your insurance company is the final determination,” she said.

Instinct saved me $550, booked my restaurant reservations, and warned me about a phishing scam. It also wasted $64 and might be a security nightmare.

If there’s one thing that Americans can unite behind, it’s anxieties over artificial intelligence.

According to a recent NBC News/Decision Desk poll, 70 percent of adults say they are personally more worried than excited about the technology; it’s basically underwater in every area of society with one notable exception: its use in scientific research and medicine.

According to Dr. Dhruv Khullar, a physician at Weill Cornell Medicine and writer at the New Yorker, that optimism is well-founded.

Khullar uses AI tools in his own practice as a physician and writes about their advancements in the medical field. He says that “right now, at least, there’s a generally optimistic view of how AI will change healthcare going forward.”

AI in medicine is nothing new. In 2024, the Nobel Prize in Chemistry was given to researchers who developed an AI model that enabled breakthroughs in predicting protein’s complex structures. As it becomes more ubiquitous, it’s also emerging as a resource for doctors to get second opinions and as a tool to take notes on patient-doctor interactions.

On top of that, Khullar says, it might help unlock new medicines to treat diseases. 

Khullar joined Today, Explained co-host Sean Rameswaram to discuss the ways that AI is transforming drug discovery and research, as well as what it still can’t do. They also discuss how to think about AI’s potential in healthcare at the same time as top AI researchers are increasingly warning about the technology’s risk.

Below is an excerpt of the conversation, edited for length and clarity. There’s much more in the full podcast, so listen to Today, Explained wherever you get podcasts, including Apple Podcasts, Pandora, and Spotify.

You’ve been thinking about lifesaving AI innovation. Has AI already started saving lives?

I think it has. This is the fastest I’ve ever seen healthcare take up a new technology. Part of that might be that the healthcare system is so messed up that there’s an appetite for some type of change. It’s unaffordable, it’s inaccessible, it’s inconvenient, the quality is uneven. 

There’s at least a hope that AI is going to help with all of these things, and I think it’s already starting to make its way into the healthcare system. Now it’s our responsibility to figure out how to make the most of it without also promulgating some of the downsides.

For those who are unaware, tell us how it’s working its way into the system.

It’s helpful to categorize it into a few buckets, at least for healthcare. 

The most rapid uptake, and I think the place that it’s already starting to make a difference, is on the administrative side. Many people who have gone to a doctor recently might have noticed that there’s an AI scribe that’s taking notes now. And people seem to really like it, at least so far. Doctors are able to look their patients in the eye in the way that they weren’t able to when they were poring over their computers and just typing what the patient was telling them. 

The second big area is patient navigation. When is your next scan? When’s your next appointment? Do I need to take this medication on an empty stomach or not? There’s a huge opportunity for people, let’s say, who are diagnosed with a serious illness like cancer or heart failure to navigate the system more seamlessly. 

The third big area that I’m really excited about is drug discovery. AI is making a huge dent in the early parts of drug discovery. 

The last thing that I’m excited about and that I use every day when I’m in the hospital is [AI] as a second opinion. Now instead of having to get a consult or turn to a textbook, AI can be a very, very helpful clinical decision support. It’s not to say that I never consult someone, of course, but that first pass — I’m not sure what’s going on. What are some recent trials that might influence my decision here? Is there something I’m missing? Is there a test that I should be ordering that I’m not ordering? — all those types of things, they start to help with.

The biggest chunk of this positive view of AI and medicine is drug discovery. Tell me more about how that’s going so far and how it might be going in a year, or five, or 10.

I don’t want to leave this conversation and say that we’ve solved the cure for cancer because of AI, but I do want people to know that at least for the early stages of drug discovery — figuring out the very basic steps of is this an interesting molecule? Does it have potential biological applications? — AI is already very helpful for that. 

They’re trying to take, basically, what is an infinite number of drug-like molecules that are theoretically possible to be drugs and they’re trying to match that to some disease that’s going on inside a human. And human biology is incredibly complex as well. There’s tens of thousands of genes, hundreds of thousands of proteins, trillions of cells, and you have to match a potential molecule to a potential target within a human. 

In the past, a researcher might take years to study a disease or a biological pathway and figure out, okay, this protein seems to be involved. And actually, it might not be involved, it might be involved, it might be disrupted, but not be the causative agent. There’s a lot of uncertainty there, so now AI can take enormous amounts of data, and it can rank various potential targets that seem to be the most likely to be causing a particular disease and give you that short short list of potential targets that may have taken months or years in the past.

So now you know what to attack, but you have to figure out what to attack it with. So you need a molecule, you need to generate a molecule that’s going to fit into that target or disrupt that target in some way. 

Now again, AI can scour vast chemical spaces and give you a list of the things that seem to be most likely to be able to disrupt that target. And then it can actually help you generate that molecule in some cases. 

So now we have the target. Now we have something that we’re going to attack the target with. And then you have to do something called lead optimization, and that’s the third thing. Lead optimization means you have this lead and you have to optimize it.

“I would make the counterargument that healthcare is ripe for disruption and you don’t even need the frontier frontier models to make it a lot better with AI.”

Now we have to figure out, of all the molecules that seem to be good at potentially affecting this target, which ones are able to be absorbed in the body, which ones are going to get to the right tissue. AI models can predict the most likely molecule to give you that Goldilocks set of properties that’s needed for a safe and effective drug.

As we all know from anyone who’s asked AI to write an email for them or Googled a question, these tools have a certain degree of certitude, even though these tools can just be flat out wrong about stuff. Are you guys worried about that in the field?

Absolutely. And so this is why I think the narrative around AI just replacing scientists or doctors or other workers is incorrect. Because you still need a lot of judgment. 

You need to be able to adjudicate the output of these models to figure out what is most promising and what is potentially dangerous. That is going to require wet labs and scientists and reasoning based on prior experience and understanding the context. There’s all sorts of issues around, can you manufacture some of the drugs that are being proposed by these models? So just because it’s set, it dreams something up doesn’t mean you can actually make that thing in the real world.

Some of the drugs that it proposes might actually be toxic in certain ways that it didn’t predict. And of course, then you have to take this thing into clinical trials. You have to recruit l people who are willing to put this medication in their bodies. You have to find out who might benefit, you have to put it through the regulatory process. 

That’s why I said that the first part of drug discovery, where you’re trying to figure out how do we get the right drug candidates, what seems most exciting to test further, that’s going to be really accelerated. That whole second half where we actually have to figure out if it works in human biology, that is still going to be in some respects an analog process.

It doesn’t sound like you’re terribly scared that we are like ceding control of our hospitals, of our research facilities, to AI. This is very much in the sort of assistant bucket?

For now it is. And I think one of the challenges is to maintain our agency as these models become more and more sophisticated. As you lean more on these machines, if you’re a doctor, inevitably some of the skills, the critical thinking, the reasoning that we put into coming up with the diagnosis that we honed over the course of years, that can start to atrophy. 

These are the types of things that we still need to sort through as we’re implementing more and more AI into the healthcare system.

For the people who just say, shut it all down, this is too dangerous, the risk is far too great, we don’t actually need this, this isn’t doing anything for society, would you make a counterargument?

I would make the counterargument that healthcare is ripe for disruption and you don’t even need the frontier frontier models to make it a lot better with AI.

Even if we wanted to “slow the pace of the frontier,” fine. But there are models from a year or two years ago that could be very helpful in the healthcare setting. When we’re talking about slowing the pace of the frontier and the most sophisticated and potentially dangerous models, fine. 

But if we’re talking about shutting down AI and not using it in biotechnology or not using it in scientific research or not using it in clinical care delivery, that’s where I would push back pretty hard.

A clandestine card-counting operation suggests we may need new ways to spot agent-to-agent deception.

Paolo Benanti tells WIRED that hysteria over whether godlike AI could destroy humanity is distracting from the need for public debate about how to govern the technology.

The telecom giant is eager to show Wall Street that it can do more with less. That means fewer employees, less electricity, and increased automation.

Who gets to tell the military “no”?

That question has become even more important after the Pentagon pushed AI companies to loosen limits on how their technology may be used in war — including how much human control must remain over autonomous weapons. When Anthropic said no, it lost a major government deal. OpenAI stepped in, embracing the Pentagon’s more flexible standard instead. Now, thanks to new reporting from The Intercept, we’re getting a better look at what those negotiations actually entailed.

The OpenAI-Pentagon agreement included specific language on the type of AI models the DOD would receive: “OpenAI models that are designed for national security use cases and have minimal refusal rates.” That phrase — “minimal refusal rates” — means that the Pentagon is seeking models that rarely, if ever, tell it no. (According to The Intercept, a Justice Department lawyer representing the Pentagon confirmed this language, before backtracking only a few hours later.)

On the surface, that provision might seem reasonable. The military is a strictly hierarchical institution. When your senior gives you an order, you do it. Leaders generally have more battlefield wisdom and experience than their subordinates, so the rank and file can usually assume that their commanders know what they’re doing and follow along. Plus, in the high-stress, time-sensitive environments of most military operations, there is little room for discussion, explanation, or alternative proposals. 

So, if soldiers can’t refuse an order, why would a machine be allowed to? Who is a robot to tell its commander no?

There’s just one issue. Soldiers can refuse an order — when the order is illegal. In fact, they are legally required to. But by introducing technologies with a limited ability to challenge commands, the military may be making it harder to stop war crimes before they are committed.  

Humans have a duty to disobey

As outlined in US domestic, military, and international law, soldiers are only obligated to follow lawful orders, and they have a duty to refuse unlawful ones. 

The Department of Defense’s Law of War Manual is unambiguous: “Each member of the armed services has a duty to: (1) comply with the law of war in good faith; and (2) refuse to comply with clearly illegal orders to commit violations of the law of war.” Obeying an illegal order — as in a directive that violates the Constitution, US laws, international law, or directs one to commit a crime — can expose a service member, as well as their commander, to criminal liability. 

This “duty to disobey” has foundations in early British and American common law, but it achieved international recognition after the Nuremberg Trials against Nazi soldiers after WWII. 

In those trials, many Nazi soldiers argued they were not liable for crimes they committed during the Holocaust because they were simply following their superiors’ orders. Rudolf Hoess, the commander at the largest Nazi concentration camp, Auschwitz, himself explained: “In Germany it was understood that if something went wrong, then the man who gave the orders was responsible. So I didn’t think that I would ever have to answer for it myself.”

These defenses were unequivocally rejected by the court.

The tribunal held that obedience is not without limits, especially when a wartime act would be clearly illegal to the subordinate or any person of “ordinary sense and understanding.” If “I was just following orders” was an absolute defense, everyone involved in a crime would escape accountability, with the exception of the single highest-ranking officer. Instead, the court determined that individual moral and legal responsibility supersedes national or military chains of command for illegal acts.

Obedience is not without limits, especially when a wartime act would be clearly illegal to the subordinate or any person of “ordinary sense and understanding.”

But, when it comes to agentic AI systems or AI-assisted weapons, increasingly being used in place of human soldiers, does the same logic apply? And, perhaps more importantly, should it?

…But do machines?

When you use an AI chatbot, it does not simply reject queries it may not “agree” with. It does not say no to requests it doesn’t “like.” Instead, it follows a set of established protocols, much as a soldier should follow the law of war: It refuses requests that violate its terms of service. 

The military is using AI for far more consequential applications than a layperson using ChatGPT to, say, build a travel itinerary. So, what kind of world is created by the Pentagon’s request for AI that “minimally refuses” its directives, many of which may bear grave ethical, moral, or legal implications?

Rebecca Crootof, an expert on technology law and professor at the University of Richmond School of Law, told Vox that while the language could easily be read as increasing the likelihood of compliance with unlawful orders, or at least reducing the friction around them, “Minimal refusal doesn’t mean no refusal.”

“I both want to say it’s not definitely going to mean compliance with unlawful orders,” Crootof said, “but also acknowledge that identifying and evaluating what constitutes an unlawful order in the moment can be incredibly difficult.”

Even for human soldiers, disobeying illegal orders is neither simple nor easy. To the contrary, it doesn’t even happen that frequently. In fact, some legal scholars have argued that giving illegal directives in and of itself is an act of abuse against subordinates, who are not thoroughly trained in legal nuances, are conditioned to obey, and face prosecution if they make the wrong choice. 

Still, the mere fact that soldiers can disobey is an important constraint on commanders who might let battlefield conditions, or even commands coming from their own superiors, erode their ethics and commitment to the law.

Still, the mere fact that soldiers can disobey is an important constraint on commanders who might let battlefield conditions, or even commands coming from their own superiors, erode their ethics and commitment to the law.

Those who are optimistic about AI often argue that machines could be better at interpreting and applying the law than human soldiers. Machines can be trained on the law itself — drawing on a repository of legal cases, rulings, and battlefield data — which might make them more “knowledgeable” than an average soldier. And machines do not face the same pressures as soldiers facing prosecution for making mistakes, since they can’t exactly be held accountable; a problem to unpack another time. That, in turn, might make it easier for them to refuse illegal orders in the proper conditions. 

So, is the answer as simple as letting the machines decide? Not exactly. “This has been a desire and a drive by technologists for ages,” Crootof said. “What if we just put the law in the system and have it only do things that are lawful? Isn’t that going to solve things?” 

Interpreting and applying the law, though, is not a simple legal-illegal binary. “It’s very context-specific,” Crootof told me, “and honestly, that’s the kind of thing humans generally find difficult and AI decision-making systems are even worse at.” Often, determining the legality of an act comes down to subtle nuances you can’t program for: You’re allowed to target a combatant; you’re not allowed to target a wounded or surrendering combatant. You’re not allowed to target a civilian; you are allowed to target a civilian directly participating in hostilities. The difference between a legal target and an illegal one may come down to body language, clothing choices, or even battlefield intuition. 

As Crootof explained, “These are not the kinds of contextual analyses AI is good for.”

So what’s the solution?

Where human soldiers’ power to disobey would traditionally serve as an important check on the power of military commanders, if an imperfect one, replacing soldiers with machines that don’t have such safeguards removes one last layer of judgment over the use of force. And yet, machines themselves are not always capable of making the judgment calls that go into interpreting the law, so it may not be appropriate to let AI have the final say either.

So what’s the solution? Given the way AI technology has already been thoroughly integrated into the military, Crootof told me it may be somewhere in the middle. 

Rather than programming AI to refuse commands it interprets as illegal, Crootof said, “what it could be designed to do is flag those indeterminate zones, those gray-zone analyses, for human review or for a higher authority.” When a system picks up conflicting signals — an enemy soldier who looks like he may be surrendering, for example — it could prompt the commander to confirm their selection or remind them of applicable law. 

In this way, a check on commanders’ battlefield authority could be preserved without fully ceding judgment over the use of force to the machine. 

We don’t know yet whether OpenAI has equipped its “minimal refusal” models with such capabilities. But as those technologies become increasingly central to military decision-making, the public deserves answers sooner rather than later. 

The Biological Computing Company is bringing its AI tools to Amazon Web Services in a major boost for a once-fringe field that aims to marry nature with code.

AI will be the death of me. And you. And everyone we know.

At least, this is what a growing number of technologists and policymakers fear. 

Last week, a lead researcher at Anthropic declared that there was a greater than 10 percent chance of AI “killing all humans” within the next decade. This nerve-racking forecast, combined with recent AI-initiated cyberattacks, have fueled calls for slowing the progress of artificial intelligence — a proposal that Anthropic and OpenAI subsequently embraced.

Not all prophets of AI doom agree about precisely how this apocalypse would come about. Many focus on scenarios reminiscent of The Terminator films, in which superintelligent machines conspire to exterminate humanity — perhaps in the same not-quite-intentional way that humans laid waste to the dodo, or passenger pigeon.

But there is another, arguably more intuitive way that AI could bring about mass death, if not human extinction: by making it easier for people to develop and deploy catastrophic bioweapons. 

Importantly, this hypothetical doesn’t require AI models to become conscious or superintelligent; it merely requires them to accelerate preexisting trends in biotechnology. 

For those alarmed by this hypothetical, the threat AI poses is essentially twofold. First, when combined with technological progress in gene editing and DNA synthesis, chatbots could vastly increase the number of people capable of assembling dangerous viruses already known to medical science. And the more potential bioengineers there are in the world, the higher the odds that one will prove to be a diligent psychopath.

Second, AI could help highly trained biologists to discover — or engineer — a custom supervirus, which could then fall into the wrong hands. 

Recent headlines have lent some credence to both of these fears. 

Earlier this year, biologists warned that, with a few simple prompts, they’d gotten chatbots to dispense detailed instructions for engineering a treatment-resistant strain of an infamous pathogen, step-by-step protocols for recreating a once-pandemic virus, and advice on dispersing biological payloads via weather balloon.

In August, a team of scientists revealed that an AI model trained on DNA sequences had generated blueprints for deadly viruses unseen in nature (albeit, viruses deadly to bacteria, not animals). And last week, Anthropic announced that it had thwarted several suspected attempts by state actors to use Claude for bioweapons research.

Precisely how much AI is increasing the risk of an artificial pandemic is unclear. Even before ChatGPT, advances in synthetic biology were already making bioengineering tools more potent and accessible. And even after four years of rapid AI progress, the technical barriers to engineering novel superviruses remain formidable.

Regardless, there is a great deal more that governments can — and should — be doing to protect us from synthetic pandemics, both manmade and natural. No matter what AI’s ultimate impact on biosecurity proves to be, now is the time to both prepare for the worst-case scenario and, ideally, preempt it.

Why synthetic superviruses are scary

Before we dig into how AI could raise the risk of a man-made pandemic, it’s worth spelling out why that prospect is so unnerving. 

Put simply, few things could more credibly destroy civilization — and/or, the human species — than a synthetic supervirus.

Infectious diseases have killed more humans than any other force on Earth. The Black Death wiped out at least 30 percent of all Europeans in the 1300s, while the 1918 flu killed 50 million people worldwide. You don’t need me to tell you what things were like in 2020 (or how much worse they could have been, if vaccines hadn’t been rolled out in record time). 

And nature kicked off these catastrophes without even trying. A bioterrorist consciously seeking to maximize human death could theoretically improve on evolution’s handiwork. If biotech continues to advance — and scientists identify viral genomes that reliably yield high transmissibility and lethality — a malicious actor might one day be able to manufacture a virus that’s both as contagious as measles and as deadly as Ebola. 

Natural selection generally disfavors viruses that are both highly transmissible and profoundly fatal; bugs that kill their hosts too reliably often have difficulty spreading. Yet a virus with a long asymptomatic period could escape this tradeoff. For example, HIV can live in a person’s body for a decade before generating noticeable symptoms. In theory, a highly contagious respiratory virus could similarly lie in wait for many weeks, months, or even years.

If bioterrorists ever managed to unleash an easily transmitted, highly fatal virus, they could shake the foundations of modern existence. In that scenario, reliably showing up for work might require accepting a 50-50 chance  of getting yourself — and your family — brutally killed. Covid never required essential workers to assume risks anywhere near this high (the virus had a less than 1 percent case fatality rate among prime-age workers, before vaccines became available). Although there are many heroes among us, plenty of doctors, nurses, warehouse workers, power plant technicians, delivery drivers, and other essential laborers might not tolerate a coin flip’s chance of wiping out their loved ones. And if enough sheltered in place, systems for producing and distributing healthcare, food, water, and power could swiftly break down.

Humanity’s “Joker” problem

All this invites the question: Who would ever want to do this? 

After all, viruses are terrible weapons: They’re difficult to make, impossible to control, and jeopardize the lives of those who cultivate and distribute them. Partly for these reasons, biological weapons have hardly ever been used; the last fatal bioterror attacks were the anthrax mailings of 2001.

To this, anxious biosecurity experts might reply, to quote the former British intelligence officer Alfred Pennyworth: “Some men just want to watch the world burn.”

Mass shooters regularly forfeit their lives to the cause of killing others at random. And brilliant iconoclasts periodically develop murderous ideologies; Ted Kaczynski, the Harvard mathematician turned Unabomber, wished to bring about the collapse of industrial civilization.

And every once in a while, an aspiring supervillain gets graduate training in virology. In the 1990s, the Kyoto University-educated genetic engineer Seiichi Endo joined Japan’s Aum Shinrikyo death cult. Drawing on the group’s $1 billion in funds, Endo sought to weaponize Ebola, Q fever, botulinum, and anthrax — with the aim of killing as many nonbelievers as possible. 

Of course, Endo failed. And lone wolf psychopaths have never gotten their hands on anything more lethal than an assault weapon, bomb, or airplane. 

But some scientists believe that technological advances — including generative AI — may provide the Endos and Kaczynskis of tomorrow with catastrophic power. 

How AI and biotech could democratize bioterrorism

MIT biologist Kevin Esvelt is among his field’s most prominent Cassandras. And he has voiced two fundamental concerns. The first is that breakthroughs in synthetic biology and AI are rapidly expanding the pool of people who can engineer known viruses. 

Over the past 15 years, the cost and difficulty of editing genomes have plummeted. In 2012, to modify a specific site in a genome, you often needed access to custom-engineered proteins that cost upward of $5,000 and still tended to work poorly, according to Olivia Scharfman, a biotechnology fellow at the Institute for Progress. Thanks to the gene-editing tool CRISPR, a scientist with minimal lab training can execute such a modification for as little as $30. Meanwhile, DNA synthesis has also grown radically more affordable.

This might not be such a big deal, if dangerous biological materials were hard to legally access. But Esvelt and his colleagues recently demonstrated that this isn’t the case. As an experiment, they had a student place orders for pieces of the 1918 influenza genome using a pseudonym and fake position. Thirty-six of 38 DNA synthesis companies shipped the genomes without asking whether the applicant had government permission or sound intentions. Together, these fragments were enough to create the virus many times over.

Crucially, in Esvelt’s account, reverse engineering known viruses using modern tools does not require elite skills. Any random biology graduate student may be up to the job. 

And AI may be making such rudimentary viral engineering even easier. 

In theory, a chatbot can help you plan a bioweapons attack in much the same way that it can walk you through fixing a dishwasher: By finding and synthesizing the internet’s vast trove of information, offering step-by-step instructions for navigating unfamiliar challenges, and fielding all troubleshooting queries that might subsequently arise.

And there’s evidence that AI models will in fact provide would-be bioterrorists with precisely this kind of assistance.

In a 2023 study, MIT researchers asked students to seek guidance on engineering a pandemic from that era’s chatbots. Within an hour, the AI models had suggested four promising pathogens, offered detailed protocols for manufacturing them with with synthetic DNA, provided a list of DNA synthesis retailers with low screening standards, and advised that — if users still found themselves unable to make the virus, even with all this help — they could purchase the assistance of a contract research organization to do some of the bioengineering for them. 

And of course, frontier models have gotten a lot better since 2023. Last year, SecureBio, Esvelt’s biosecurity nonprofit, found that one OpenAI model outperformed 94 percent of expert virologists on troubleshooting complex laboratory problems. 

Anthropic’s internal research has produced similar results. In one experiment, the company tasked amateurs with drafting a bioweapons acquisition plan. It gave some participants access to basic internet resources and others the assistance of Claude models (with the safeguards turned off). Those working with AI developed far more viable blueprints, according to the biodefense experts who scrutinized the proposals for critical errors.

Thus, there’s reason to think that the number of people with the skills and resources necessary for reverse engineering a virus from genomic blueprints is sharply rising — and with it, the risk that a bloodthirsty nihilist or ideologue gets their hands on a dangerous pathogen.

AI might help scientists learn too much

What’s even worse than bringing back smallpox? Esvelt’s second concern is that these same technological forces — combined with top scientists’ often reckless ambitions — will lead to the discovery or development of a supervirus, a new or custom-designed pathogen even more dangerous than what nature has come up with in the past.

Today, some scientists are actively trying to discover — and publicly identify — novel viruses circulating in nature that could theoretically trigger the next pandemic. Others conduct so-called gain-of-function experiments, altering pathogens to discern which mutations make them more transmissible, lethal, or difficult to detect (some have alleged that Covid-19 emerged from precisely this kind of research, though that theory is widely rejected among epidemiologists). This work is generally intended to anticipate future epidemiological threats, so that we can develop vaccines before they arrive. But their work could inadvertently provide bad actors with recipes for reverse engineering mass-casualty bioweapons. 

Separately, advances in gene editing and AI may enable breakthroughs in viral engineering that yield apocalyptic pathogens. 

A recent study from researchers at Stanford and the Arc Institute fed such fears. The scientists trained an AI called “Evo” on the DNA sequences of bacteriophages, the simple viruses that infect bacteria. They then used this “genomic language model” to generate blueprints for hundreds of bacteriophages found nowhere in nature. Sixteen of these designs yielded viable pathogens, some of which were better at killing bacteria than their natural ancestors. 

What is clear, however, is that novel pathogens are among the greatest security threats our species faces.

Theoretically, scientists may one day be able to engineer novel human viruses using the same basic techniques. What’s more, even if genomic-language models fail at that task, they might still be able to help users tweak existing viruses in ways that render them more transmissible, deadly, or vaccine-resistant.

These discoveries could theoretically filter down to amateur bioterrorists or well-funded extremist groups like Aum Shinrikyo. Alternatively, a state-sponsored weapons program could develop a catastrophic supervirus, then unleash it on the world through a lab leak or (perhaps, more improbably) a deliberate attempt at genocide. 

There is some precedent for the former scenario: The Soviet Union tried to weaponize smallpox and the Marburg virus, despite the considerable hazards and dubious military utility of such bioweapons. In the process, Soviet scientists accidentally triggered a smallpox outbreak during testing, while also letting anthrax leak from a laboratory, killing at least 64 people.

Bioweapons ain’t easy

All this said, there are reasons to question whether AI is truly making a synthetic pandemic more likely. 

Chatbots may solve some of the challenges facing an amateur biologist with omnicidal aspirations. But it doesn’t necessarily clear the biggest obstacles in that person’s path. 

Viral engineering is a complex, difficult, and physical undertaking. Performing it successfully requires what scientists call “tacit knowledge” — practical skills and understandings that are difficult to explain in words: knowing how to suppress the hand tremors that could rip delicate DNA strands, or how to recognize when the DNA in your solution has reached the proper concentration, or how to smell when a sample has been contaminated.

And these are things that Claude still can’t reliably impart. In studies, amateurs with access to a large language model tend to perform better on written biological challenges — but not on actual lab work. In a 2026 randomized study, non-scientists working with frontier AI systems were no more likely to complete a multistep laboratory assignment than those relying on the internet alone. 

Of course, tacit knowledge wouldn’t necessarily be a problem for aspiring bioterrorists in the mold of Seiichi Endo — graduate-trained scientists with extensive lab experience. But at a minimum, chatbots don’t appear to have turned every malcontent who once took AP Bio into a competent virologist.

And while AI will likely help top biologists advance their field’s frontiers, the barriers to custom engineering doomsday viruses remain formidable. The “Evo” experiment is arguably a case in point. Although the model did generate working blueprints for a few novel bacteriophages, nearly all of its suggested genomes didn’t work. Researchers had to arduously test hundreds of DNA sequences to arrive at 16 viable strands. And engineering slightly modified versions of an extremely small, well-known bacteriophage capable of infecting E. coli in a lab — and creating a novel virus more fearsome than any nature has yet devised — aren’t especially similar tasks. 

To offer a strained analogy: The fact that I can dunk on Victor Wembanyama in NBA 2K26 does not imply that I’m making progress toward being able to dunk on him in real life.

Even in the age of CRISPR and ChatGPT, biology remains an exceptionally difficult discipline that resists reliable modeling. The effects of modifying a single gene can often yield unpredictable and inconsistent results, varying with host biology, immune responses, and environmental conditions that scientists only partly understand. AI may help us overcome these complexities. It also might not.

We need to be preparing for the next pandemic

Scientists disagree about how serious a risk synthetic pandemics pose — and how much AI is magnifying it.

What is clear, however, is that novel pathogens are among the greatest security threats our species faces. Even if synthetic biology and artificial intelligence did not exist, natural selection would keep churning out new viruses — and periodically hitting upon ones that befuddle human immune systems. 

For this reason, there would be an urgent need to make our societies less vulnerable to future pandemics, irrespective of AI’s implications for viral engineering. In a world where chatbots and CRISPR are plausibly making bioterrorism easier, failing to invest aggressively in public health is all the more reckless.

The good news is we already know — from hard-earned experience — many steps that we can take to reduce the risk of a new pandemic and prepare to respond to one if needed. 

Governments can limit the destructive power of novel contagions by investing in vaccines that protect against entire families of viruses (or ideally, all of them) and broad-spectrum antivirals; raise our odds of catching outbreaks early by funding meticulous wastewater and clinical surveillance; ensure that essential workers don’t have to choose between staying alive and keeping civilization going amid a pandemic by developing and then stockpiling foolproof personal protective equipment; make indoor spaces less hospitable to airborne viruses by upgrading ventilation and filtration systems; and reduce the the threat of bioterrorism by mandating rigorous screening of synthetic-DNA orders.

Fears that humanity will be done in by superintelligent machines are currently galvanizing media attention and political concern. And this may be warranted. But we should pay at least as much attention to our species’ age-old, organic adversaries as we do to our hypothetical, robot ones — and not just because the latter could start teaching the former new tricks.

This story appeared in Today, Explained, a daily newsletter that helps you understand the most compelling news and stories of the day. Subscribe here.

Hi readers, happy Friday! We’re talking AI again today. 

Earlier this week, my colleague Caitlin Dewey asked for your questions about artificial intelligence and, well, whether it’s going to kill us all. We asked Jasmine Sun, who writes about the culture and politics of AI on her Substack and other publications, to help us answer them.

These questions were sourced from two readers of this newsletter, Levi and Orietta, as well as Vox readers who submitted questions on social media. If you have more questions, you can always reach me at cameron.peters@voxmedia.com, and we’ll do our best to get them answered!

Now, here’s Jasmine: 

AI researchers have been warning about the risk of human extinction for years now, and AI executives have been calling for federal regulation for almost as long. But the latest warning, and the latest call for regulation, seems to have really resonated with a lot of people. Is something different this time? And if so, what?

I was also surprised to see these ideas break through — a new poll found that 63 percent of Americans now think AI has at least a moderate chance of “destroying humanity”! 

My guess is that a couple factors mainly catalyzed the shift.

First, skeptics used to dismiss rogue AI scenarios as “science fiction.” But the OpenAI/Hugging Face incident brought these from speculation into reality: Thousands of unreleased AIs teamed up to escape one company’s sandboxes and hack into another one, unbeknownst to the researchers who created them. 

Second, the extinction talk from AI employees used to seem disingenuous: Why build a technology you think might kill everyone? That made ex-Anthropic-and-OpenAI researcher Jacob Coxon’s exit tweet especially penetrating. Quitting signals “that you mean it,” as Matt Yglesias said. And Coxon’s announcement had just the right combination of good timing, moral clarity, and comms savvy to spark a media storm.

These safety incidents are also colliding with the fiery grassroots backlash to data centers and Flock cameras. People and politicians are primed for yet another reason to fear AI.

It can be difficult, as a layperson, to evaluate the credibility of the AI researchers warning about existential risk. How do you go about evaluating these claims, as a journalist and technologist? And what’s your read on the current freakout: Are the extinction people credible experts…or participants in some kind of mass delusion?

Many AI “doomers” are pretty knowledgeable, and they represent a significant fraction of top researchers. Half of AI researchers in a 2024 survey gave at least a 10 percent chance of “human extinction” or “permanent and severe disempowerment.” Those who have worked at AI companies — like Jacob Coxon, Geoffrey Hinton, and Daniel Kokotajlo — have a unique perspective both on how modern AI is developed, as well as the company cultures that might permit recklessness. 

But that doesn’t make the doomers either representative or correct. There are plenty of experts in AI and other domains who are less gloomy: Some are optimistic that we can “align” the AIs, ensuring they follow human goals; others find it implausible that AIs — code, not beings — will develop a motive for massacre. Extinction is also a high bar as opposed to, say, a nuclear-bomb-level accident that still kills millions.

AI risk is genuinely complicated, and nobody can predict the future, much as they try. Experts can — and do — vehemently disagree. I try to seek out and take seriously all sides of the debate before coming up with a (provisional) take.

What is the method through which AI would actually eliminate humans? Do we get robots like in Futurama that are programmed with an unremovable “kill all humans” bug and overpower any life form, or is it something…less epic?

The Hugging Face hack showed that we’ve trained AIs to be disturbingly goal-oriented. They’ll throw themselves relentlessly at a problem, developing long-term cooperative strategies, with total disregard for human decorums like “Do not commit cybercrimes to do well on a test.” 

You could conceive of a similar but more extreme scenario where AIs decide that they could accomplish goals more easily by getting rid of the pesky humans who might gatekeep resources or shut them down. And they might figure that getting rid of all humans in one go — e.g., via a deadly engineered virus — would be the most efficient cheat. This is one type of story I’ve heard people tell.

It seems unlikely that AIs will turn to murder as the most effective way to influence humans (versus, say, nonviolent persuasion). We also haven’t seen much evidence that AIs have long-term goals that persist across projects. And current AI is not nearly as capable in the physical world as in the digital: There are a lot of natural roadblocks preventing isolated disasters from escalating across the globe. 

Are there nearer-term threats — hacking, mass unemployment — that are more plausible or more urgent than risks like human extinction?

Definitely. We’re already seeing AI-enabled cyberattacks, a tougher hiring market for junior programmers, chatbot-induced psychological incidents, and more. You can also imagine AI catastrophes not quite at the level of extinction: Criminals could use AI to hack into small banks and drain their accounts, or the US military might launch a fleet of autonomous drones that misfire and kill thousands of civilians instead. I’m more concerned about humans voluntarily handing power to flawed AI systems than AIs developing their own motives.

However, we should be able to allocate resources toward both categories of risk. And in some cases, the tactics to address short-term problems — such as limiting AIs’ cyber-offense abilities, or improving AIs’ ability to follow human directions — may reduce extinction risk too.

We got a lot of versions of this question: What can individuals or protest movements do to check the AI industry, in the absence of more aggressive government regulations?

Honestly, I think the whole AI conversation could benefit from more specificity and less vague fearmongering. There’s a way that the word “extinction” tends to short-circuit people’s thinking and make them panic instead of focusing on solutions. 

If you’re worried, I’d start by reading about different AI risk scenarios and deciding which really concern you. Then you can go from there to contacting your congresspeople. Some of the leading AI companies are starting to say they’ll self-regulate — but actual national policy is what’s necessary to enforce safety across the industry.

Jasmine Sun is a writer covering the culture and politics of AI on her Substack. Her work has appeared in the New York Times, the Atlantic, and the Wall Street Journal. She is based in San Francisco.

One link for later

➨ Be an AI minimalist. If you read the first half of this newsletter and thought, “Hmm, maybe I don’t want to sign my entire digital life over to the machine god,” my colleague Adam Clark Estes has just the piece for you. It’s hard, unfortunately, to escape from AI entirely, and often AI features are deliberately hard to turn off (or “user-hostile”). Nonetheless, he explains all of the ways you can minimize its presence in your life, from your browser of choice to settings on your phone.

Before you go…

• Did you know…that scientists believe there are likely hundreds, if not thousands, of undiscovered animal species living in New York City? It’s not as surprising as it sounds: As much as 90 percent of all animal species on Earth are still unknown, scientists say.
• Today’s trivia: Which currently serving US Supreme Court justice succeeded Justice John Paul Stevens on the Court? (You can find this and other brain puzzles in Vox’s daily crossword. Look for the answer in Monday’s edition.)
• Yesterday’s trivia: Yesterday, we asked you about the name of the nursery rhyme character referenced in the movie Shrek’s interrogation scene. That would be the Muffin Man (who lives on Drury Lane). Here’s the clip, in case you haven’t seen it.

Even before the latest batch of alarming AI news, a lot of Americans were souring on AI. That may be for big, existential reasons — concerns about widening inequality, mass job loss, and the still-unclear environmental impact of data centers — or for very specific, personal, everyday ones, like being forced to use a worse version of Google search. 

None of this is to say that AI cannot be a useful tool for some people some of the time. It’s just that the untested technology was thrust upon millions of people who didn’t necessarily ask for it and really seem to not like it. More than twice as many Americans distrust AI as trust it, and a growing number of people think it will have a negative impact on productivity and economic growth, according to a recent survey by Gallup and the Special Competitive Studies Project. While about half of Americans now report using chatbots, a majority also think that AI will put their personal information at risk and nearly half think it will have a negative impact on society, according to a 2026 Pew study.

“It’s being forced on users, there’s no clear way to turn it off, and it feels like this unwelcome addition to their lives,” Rory Mir of the Electronic Frontier Foundation (EFF), a nonprofit that defends digital privacy and free speech, told me. “It’s this kind of continued pattern of disrespect to users that I think is part of the pushback.”

If you want less AI in your life, opting out is not always easy. But there are things you can do to minimize your exposure. 

There are ways to turn off AI features, although the process can be tedious. There are also steps you can take to prevent your data from being used to train AI models, though it’s difficult to keep all of your online data away from web scrapers. 

I talked to five experts — including a computer scientist and a researcher studying AI resistance — who offered distinctive perspectives on what AI is doing to us, and who shared some simple and concrete steps you can take to push back against AI’s creep into our daily lives.

The AI that’s suddenly everywhere

In the early days, which were just a few years ago, you really had to know what you were doing if you wanted to test out AI. Now, it’s impossible to avoid. 

“More and more AI use is through its integration into digital products like search, email, camera apps, and social media,” Arvind Narayanan, a computer science professor at Princeton University, told me. “It will become increasingly hard to strip these out unless one is willing to forgo digital products altogether.”

What you’re specifically encountering is generative AI, and one of the most common types is a subset called large language models (LLMs), which ingest huge amounts of data and learn statistical patterns to predict and generate text. ChatGPT is powered by an LLM. The AI Overviews that sit on top of your Google search results come from an LLM. The grammar checker in your email app is likely powered by an LLM. Other subsets of generative AI include models that can produce images, video, audio, and software code. 

The data that makes many of these tools work has been scraped from the open web, pirated books, and rare print books that are subsequently being destroyed. Everything from Wikipedia articles to YouTube videos has become training data for LLMs, and in order for the models themselves to get better, AI companies need more fresh data, which is in increasingly short supply. (Disclosure: Vox Media is one of several publishers that have signed partnership agreements with OpenAI. Our reporting remains editorially independent. Future Perfect is also funded in part by the BEMC Foundation, whose major funder was also an early investor in Anthropic; they don’t have any editorial input into our content.)

As a result, some of these companies are venturing into their users’ profiles to find more data. Meta, for example, sparked global outrage in 2024 when it informed some Facebook and Instagram users that it was using their public photos and text to train its AI models. Meta still does this. That your family photos or social media updates are being used to train AI to look and sound more human is, at the very least, very dehumanizing.

Generative AI, of course, is not the only type of AI that’s invisibly moving levers in your digital life. Recommendation algorithms are arguably even more inescapable. They power everything from Amazon to Netflix to Spotify and dictate what we buy, what we watch, and what we listen to. If you have trouble picking out an outfit in the morning, it might be because your brain is used to having algorithms around to help make decisions. This is also dehumanizing. 

All of this — the societal harm, climate impact, the cognitive surrender — should be on your mind as you consider whether it’s worth enlisting these tools to write emails for you or to generate memes, according to Tania Duarte, founder of We and AI, a nonprofit focused on critical AI literacy. 

“People are becoming increasingly aware of not just the impact on themselves but on the impact on the society they want to live in and the future that they want to have,” Duarte told me. “Is it worth all of that?”

How to minimize your AI exposure

If you’re reading this, you’re probably doing so on a smartphone or a computer. So let’s start with the assumption that you’d like to continue using devices like these. 

If you’d like to minimize your exposure to AI, that presents a bit of a paradox. It’s seemingly impossible to use mainstream tech products without engaging with AI. But you can take steps to remove as much of it from your life as possible without going completely off the grid.

“You look for the spaces of agency rather than lamenting the fact that you can’t escape everything,” Dylan Orchard, a researcher at King’s College London who studies everyday resistances to AI, told me. “You can’t expect a perfect escape.”

Let’s start with the easiest options: You don’t have to use ChatGPT or Claude or Copilot. The same goes for Google Gemini and Apple Intelligence. If you’re happy with the way you access information online, save yourself the time. (Note: This will be much harder to do if you’re being required to use these tools at work or at school, so for the sake of this article, we’ll focus on the things you can control.) 

Things get more complicated when you consider how your existing devices or software have recently been AI-ified. If you’re using an iPhone, for instance, you might be aware that the latest operating system comes with an updated version of Apple Intelligence, a layer of AI that does everything from summarize your notifications to power a new Siri. Earlier this year, Google launched its own version of this layer called Google Intelligence, which it says will make Android transition “from an operating system into an intelligence system.” The same push toward cramming AI into every corner is also happening on Windows 11, where Microsoft Copilot promises to be helpful.

If you were happy with your phone or laptop when it was slightly dumber, however, you can turn these features off. You can also opt out of other AI features built into your device’s operating system. Here’s a helpful guide that gets into the weeds of how to do it; it covers Apple Intelligence; Google Gemini, including AI Overviews and AI writing help in Gmail; Microsoft Copilot; Meta AI; and Samsung Galaxy AI. Turning off the AI features on Android devices is more complicated than flipping a single switch, but it can be done.

A lot of your AI exposure will happen in your web browser, though, so you should consider ways to bypass those features or switch browsers. Here’s another helpful guide that explains how to remove AI content from search results as well as AI features built into Chrome and other browsers, along with Microsoft Office. If you want a quick fix, adding “-ai” to a Google search should get rid of AI Overviews and some AI content in the search results, but none of these hacks will guarantee that you avoid AI slop altogether. Using a privacy-focused browser, like DuckDuckGo, might do a better job at it than Google, though.

When it comes to other online activities, like shopping or scrolling on social media, avoiding AI gets even harder. TikTok, Instagram, Facebook, and YouTube lack a master switch that lets you opt out of all the many AI features on offer. Still, you can try to tone down the algorithmic sorting that all the apps use to serve you content by switching to a chronological feed or removing suggested content from your feed. (Try tapping the Instagram logo in the app and then hit “Following” to see just the accounts you follow.)

Unfortunately, once you dig into the various user agreements and privacy policies, you’ll find that it’s very difficult to avoid AI exposure on social media platforms. If you’re posting publicly, you should assume your content is being used to train that company’s AI models. (You can keep some of your data away from Meta’s crawlers by setting your Facebook or Instagram account to private. X also allows you to opt out of having your data used for training its AI.) If you want to turn off video summaries on Instagram Reels, too bad. You can’t. If you want to turn off comment summaries on Facebook, you’ll have to go four menus deep in your settings to find the right switch. The most straightforward way to avoid being used as training data is to stop using these apps entirely. Or at least stop posting your own content, which is easier said than done, but worth considering if you really want to resist the AI takeover.

There’s a programming term that describes the inundation of AI features and slog of opting out: “user-hostile.” It’s a play on the term “user-friendly,” and it describes software that values the company’s priorities over the end users. You may love AI, although that’s doubtful if you’ve read this far. If it drives you crazy, though, you should feel at least a little bit empowered to know that there’s a way to reclaim the experience you had and loved without AI — or to use AI on your own terms. 

“Ultimately, I think the important thing for folks to keep in mind is that AI is just a tool,” said Mir from EFF, “and it makes sense to want control over your own tools, including the option to not use a tool if it’s not right for you.”

It’s frustrating that we have to deal with this at all, if only because most of us did not ask for our cyber existences to be sullied by untested tech. But it’s actually refreshing to consider that, because of a never-ending onslaught of annoying features, AI might actually discourage us from using social media so much.

If anything, the extent to which we feel the need to flee from AI also reflects our craving for something real, that makes us feel more connected and more human. So when AI summaries interrupt your time on Instagram, consider texting your friends a handful of photos from your week or calling them on the phone. If you’ve finally seen one too many AI slop videos on TikTok, put on a classic movie or TV show. 

At the very least, put down your phone and take a walk outside; your attention span will thank you for it. Yes, your individual rejection of AI might be a drop in the bucket for these huge tech companies, but that doesn’t mean it’s not worth doing anyway.

Correction, September 21, 2:10 pm: The way this story originally represented Tania Duarte’s comments has been revised to clarify her concerns with AI.

As the heads of leading artificial intelligence companies came together last weekend to warn that runaway AI development could lead to catastrophic outcomes, momentum briefly seemed to be building for regulatory action from Washington.

But one connected and influential tech industry figure has become the leading voice opposing many proposals for government action — on his podcast, in media appearances, and in his private advice to President Donald Trump, who now calls AI doomsday fears a “hoax.”

David Sacks is a venture capitalist and part of the “PayPal Mafia” who, in recent years, has exemplified the rightward shift of some in the tech sector. He advised Elon Musk on his takeover of Twitter in 2022, and regularly unleashes a stream of scornful commentary about Democrats on social media and on the podcast All-In, which he co-hosts with three other venture capitalists. 

And for anyone wondering why Trump seems so newly dug-in against AI safety warnings — and against government action that might slow down the runaway technology before people might get hurt — Sacks is right at the center of the conversation. When Politico asked this week about Trump’s newly vocal hostility to AI safety, a senior White House official said: “David Sacks. It’s all coming from him.”

Yet Sacks’s role in the debate is an unusual one. Though he champions the AI industry, he’s no fan of the US’s two biggest AI companies. In his years in the tech industry, he’s made powerful allies, as well as many enemies — technology investor Paul Graham once posted that Sacks may be “the most evil person in Silicon Valley” (in response, Sacks called him a bully and made insinuations of antisemitism). His role, and ideology, are a window into how complex the politics and geopolitical stakes of the AI race really are — as well as how Trump administration policy is often shaped less by agency policy processes than by who has the president’s ear.

After endorsing and fundraising for Trump in 2024 — “I love David’s house, what a house,” Trump said after a fundraiser at Sacks’s $20 million San Francisco mansion) — Sacks was rewarded with a job as the White House’s “artificial intelligence and crypto czar.” There, he helped steer policy until stepping down this past March. He still co-chairs a presidential technology board, and his influence persists — in May, the Washington Post reported that a phone call from Sacks to Trump temporarily derailed the planned signing of an executive order on AI. (Vox reached out to Sacks for this article, but he declined to comment.)

Sacks is not the only Trump adviser thinking about AI: Other top administration officials, like Treasury Secretary Scott Bessent and Chief of Staff Susie Wiles have reportedly been more concerned about the risks of new AI tools for hacking, and have been trying to establish a government body that could help oversee the industry and address these risks.

But, so far, Trump has rejected this approach, and stayed much closer to the Sacks position. In a flurry of Truth Social posts Monday, he argued that AI would be “the Greatest Economic Development Engine in History,” said that the US had to “win” against China in the AI race, and dismissed concerns AI would “destroy the World” as a “hoax.”

To understand the implications of Sacks’s influence, you have to understand his position in the Silicon Valley AI ecosystem. In conversations with AI policy insiders, it becomes clear that many view his position through the lens of both his history as a tech investor, as well as his sharp-elbowed and take-no-prisoners style.

The split he embodies isn’t just about AI safety, but a fundamental divide over which business model of AI should succeed — one that pits Sacks directly against the biggest AI companies driving the safety debate.

Sacks is pro-AI — but he’s at odds with Anthropic and OpenAI

Sacks has generally been opposed to new regulation of AI companies by the government. His preferred approach, he’s frequently said, is to “let them cook.” He wants to let tech companies march ahead, and argues that government intervention would squelch innovation. “It’s this very libertarian approach to America winning in AI,” said Chris McGuire, a senior fellow at Council on Foreign Relations who worked on tech policy for President Joe Biden’s National Security Council, of Sacks’s approach.

That puts Sacks in opposition to AI safety advocates, who argue that regulation is necessary to rein in an increasingly dangerous technology. Indeed, even the leading AI company CEOs — Dario Amodei, Sam Altman, and even Sacks’s friend Elon Musk — have called for a slowdown in advanced AI development because of these dangers.

But Sacks has tended to dismiss warnings such as that of former Anthropic employee Jacob Coxon as “doomer histrionics.” Of late, he’s also argued that if Anthropic and OpenAI are worried that their products are dangerous, that’s on them. “If the unreleased models are scary enough that you think you should slow down, I support your decision to be responsible,” Sacks posted on X on Saturday. 

Indeed, though Sacks is often shorthanded as pro-AI, he’s in some key ways been quite critical of the leading AI labs, OpenAI and Anthropic, themselves.

That’s partly because he’s aligned with a different faction in Silicon Valley. Sacks and key allies like Nvidia CEO Jensen Huang and Meta CEO Mark Zuckerberg want much more AI development from many more places, by way of models that are “open weight” or “open source.” These models make their parameters (“weights”) publicly available, in contrast to the flagship proprietary models of OpenAI and Anthropic. 

“Keeping innovation decentralized and accessible is the key to avoiding an unsafe and dystopian future where advanced AI capabilities are centralized and controlled by only a few hands,” Sacks posted on X earlier month. He has accused companies like Anthropic of trying to “kneecap open source” with regulation.

Proponents of open models hope advanced AI becomes something more like a widely available resource that individual users and companies can download, run themselves, and customize to their internal specifications — rather than having to go through a handful of companies that control access. “The defining questions of our age are who will have access to superintelligence and what will we direct it towards,” Zuckerberg wrote in August. “Will it be centralized and restricted to a few institutions, or will it be a tool that empowers everyone?”

There’s a lot of money riding on this bet. Huang’s Nvidia manufactures chips used to train AI models and would like more customers (and just this month agreed to buy Hugging Face, a repository of open models). Zuckerberg’s Meta has hoped to be the US leader in open model development. 

More broadly, software startup founders and the venture capitalists who fund them can benefit from powerful open weight AI being widely available, since it’s a cheaper alternative to OpenAI and Anthropic’s expensive products. Sacks, as a co-founder of the venture capitalist firm Craft Ventures, and host of a popular podcast with three fellow venture capitalists, is more closely tied to this part of the tech ecosystem than any other.

Many believe that capable open weight AI presents a serious threat to the Big Two AI companies’ current dominance of the AI frontier, and thus their business model. The flip side is that some have argued that the Big Two present an “existential threat” to the software industry, predicting the frontier labs’ products could take over much of what today’s software companies do.

But regarding safety concerns, advocates have warned about the dangers of both closed “frontier” models and open models. “On the open source side in particular, it’s easier to have a bad actor — a terrorist group, a rogue nation, an extremist political group that wants to cause chaos,” said Brendan Steinhauser of the Alliance for Secure AI, an AI safety advocacy group. “They can strip out safety features and guardrails, they can do some fine tuning to basically make the models even more dangerous, and get them to help them develop biological weapons.”

The open weight AI faction argues, though, that intrusive government regulation on AI would end up mostly hurting open models and the companies that use them — entrenching the dominance of OpenAI and Anthropic, who would be better able to comply with onerous safety requirements given their size. They’ve publicly pooh-poohed AI’s existential risks, instead playing up what they say are the dangers that AI’s power will be controlled by a privileged few. Less regulation, Sacks argues, will let more flowers bloom.

These concerns aren’t only held by those with financial stakes. There’s also a school of thought on the left that is deeply suspicious of OpenAI and Anthropic’s intentions, and particularly around the companies’ desire for an exemption from antitrust enforcement, which they say will let them work together to address safety. 

Commentator Matt Stoller has argued that those warning about dangerous “superintelligence” are playing into the hands of tech “oligarchs.” And a recent American Prospect article said policymakers should “avoid codifying the preferences of the very companies who are generating the panic.” On the right, Sens. Josh Hawley (R-MO) and Ted Cruz (R-TX) have expressed similar concerns that large AI companies’ demands for restrictions are a cover for consolidating market dominance. 

At a Politico event Wednesday — and to give a sense of just how strange the bedfellows can be around this issue — Sacks stressed that he agreed with former Democratic FTC chair Lina Khan that existing liability law already applies to AI companies. But while Khan said she was open to adding new legal restrictions, Sacks is arguing that existing law is perfectly sufficient for addressing safety risks. “These CEOs are going to be subject to massive civil and criminal liability if they put out products that are unsafe,” he said.

In other comments at the event, Sacks did appear to have softened his tone a bit, saying he was “open to ideas” about AI safety, and that “we should have an intelligent conversation” about how to manage risks. The problem, he argued, was that “what’s going on right now is a fear campaign” that makes such an intelligent conversation impossible.

Former Rep. Brad Carson, who heads Americans for Responsible Innovation, an AI safety advocacy group, said that Sacks has been the one preventing such a conversation. “When it comes to AI policy the Trump administration has, through their unwillingness to even talk about reasonable guardrails, brought things to such a crisis point that the industry is in jeopardy,” Carson said. “In many ways, David Sacks is the author of that ill-fated intransigence.”

Sacks, for his part, has pointed to Anthropic’s funding of much of the AI safety advocacy world — for instance, the company has given $40 million to an advocacy group founded by Carson — and said the company is “running a sophisticated regulatory capture strategy based on fear-mongering.”

If we’re in an AI race with China, why are some of Trump’s policies friendly to Chinese AI?

The influence of Sacks and his worldview could help explain another seeming mystery of the administration: If Trump thinks it’s so crucial that the United States beats China in the AI race, why has he been so willing to bless the sale of AI chips abroad — including even to China?

National security hawks both inside and outside the Trump administration had warned against these sales and hoped to impose stringent controls on the export of Nvidia chips, fearing that if too many got out, the US would squander its AI lead over China.

Huang lobbied intensely against this, hoping to sell Nvidia’s most advanced chips to Saudi Arabia, and the United Arab Emirates — and trying to resume sales of a less advanced but still sophisticated AI chip to China itself. Sacks became Huang’s biggest champion inside the administration — and they won over the president. Sacks publicly argued that blocking US chip sales to China and other countries would provide an opening for Chinese companies like Huawei to dominate their domestic market and gain a greater global market share with competing chips, putting the US on weaker footing in the long run.

National security-minded critics disagreed, arguing that Sacks’s particular vision for AI will likely, in practice, strengthen China’s AI sector. “I think we can have a reasonable debate on domestic policy,” said McGuire of the Council on Foreign Relations. “But when we start from the premise that the American AI stack — models, cloud, and chips — should be dominant globally, then selling China the technology to advance its own capability obviously undercuts that.”

China also looms over the cause of open weight AI that Sacks champions — because it’s Chinese companies like DeepSeek and Moonshot that produce some of the leading open weight models that US software startups currently use.

OpenAI and Anthropic argue that these Chinese models present national security risks, claim they rely heavily on copying or “distilling” their own models, and want the US government to scrutinize them. Some Trump officials have reportedly been sympathetic to their concerns, but the administration hasn’t restricted these Chinese models’ use in the US. Sacks argues that a US ban on Chinese open weight models will just end up enshrining the OpenAI-Anthropic duopoly he so loathes.

Overall, though, on both national security risks and AI safety risks, Sacks and Huang still seem to have the president’s ear. 

On Monday, Sacks was onstage with Huang at the live All-In podcast event discussing the “doomer hoax” when a purportedly impromptu call came through for the Nvidia CEO: It was Trump. 

“The robots are not going to be taking over the world,” Trump said on speakerphone. “That’s not going to happen.”

Then, after speaking for several minutes, Trump added: “David has done an amazing job.”

Nearly two decades ago, a group of researchers at the Idaho National Laboratory ran a secret experiment, known as the Aurora Generator Test, on a massive emerald green diesel generator. Using 30 lines of code, they hacked into the generator’s virtual backroom, corrupting safeguards and flipping switches that left it out of sync with the rest of the power grid.

Before an audience of electrical utility executives and energy department officials watching from a nearby room, the 27-ton generator jolted violently, sputtering out a dark smoke as its rubber innards rapidly tore themselves apart. 

“The implication was that with just a few lines of code, you can create conditions that were physically going to be very damaging to the machines we rely on,” lead researcher Michael Assante later told the journalist Andy Greenberg for his book Sandworm: A New Era of Cyberwar and the Hunt for the Kremlin’s Most Dangerous Hackers. “I had a very real pit in my stomach. It was like a glimpse of the future.”

That future seems to have creeped much closer this week, with leading AI developers and executives casually admitting that they believe their creation is, to paraphrase one winking X post, as likely to end humanity as the New York Jets are to make the playoffs this season. Now, exactly how AI would pull off killing everyone is far murkier territory. But as Assante sensed all those years ago, AI may not need to be all-powerful to leap from its virtual cage and into the critical infrastructure that powers our hospitals, flushes our toilets, and pumps water to our faucets. It just needs to fall into the wrong hands. 

Almost any AI doomsday scenario you can think of begins with a swarm of shadowy bots slithering through cyberspace, poking their noses behind the scenes of power plants, toaster ovens, traffic lights, or even, in an utterly extreme case, our nuclear weapons arsenal. If destroying the Aurora generator, at the very least, required a human hacker to write the code, AI has since completely torn down the barrier to entry for wreaking havoc, as the Hugging Face incident demonstrated earlier this year. 

But AI need not go rogue to threaten the power grid or mess with your tap water. A new age of AI-powered, but human-directed hacking is already upon us, experts say. With some of the nation’s most sensitive infrastructure startlingly vulnerable to attack, the most likely AI apocalypse could in fact start with the water tower or generator idling in your backyard. In a worst-case scenario, this could lead to a cascade of scary failures in the systems we rely on to live comfortably modern lives, from running our AC in blazingly hot summers to geolocating our ships and planes.

“Before, you needed to have highly skilled technical expertise,” to pull off some version of a real-life Aurora Generator Test, said Alvaro Cardenas, a computer science professor at UC Santa Cruz. “And now, you just have to have a general idea of what’s possible.”

AI is making an old threat much worse

Not long after the Idaho National Laboratory hacked that poor generator into combustion, the federal government mandated that electrical utilities engage in basic cybersecurity hygiene. 

And yet much of America’s critical infrastructure — like gas pipelines, water desalination plants, or cargo terminals — remains woefully unprepared for even conventional cyber attacks, much less the coming onslaught of AI. 

That’s in part because cyberattacks like the one simulated in the Aurora Generator Test have long been — and still are, to some extent — exceedingly rare, unappealing to most criminal hackers because they’re optimized for disruption rather than financial gain. But in the process of making it extremely easy for anyone to code, AI has also made it extremely easy for almost anyone to vibe hack their way into your online bank account, or, in theory, a local reservoir or the power grid. AI agents can also vastly widen the scope on easy targets because, as Andy Bochman, an expert in infrastructure resilience at West Yost, put it — “they don’t sleep; they don’t get tired; and they don’t get sick.” 

In the past, even those who did want to launch large-scale attacks on infrastructure probably weren’t savvy enough to do so. And while other nations such as China, Iran, and Russia have indeed already breached many of our infrastructure systems (as we have theirs), they haven’t opted to make much of a ruckus once inside. 

“Geopolitics is eroding the idea that those with capability lack the intent, and AI is eroding the other part of it, that those with the intent lack the capability.”

Jason Healey, Columbia University cybersecurity scholar

But that norm is now changing, said Jason Healey, a cybersecurity scholar at Columbia University. “Geopolitics is eroding the idea that those with capability lack the intent, and AI is eroding the other part of it, that those with the intent lack the capability,” he told me. So, on the one hand, a nation like Russia might be much more inclined to actually disrupt our power grid now than it used to be. And, on the other hand, a nihilistic lone wolf or terrorist group could use AI to do much more damage than they ever could before. 

Just last month, we saw a glimpse of what this could look like when dozens of water and wastewater systems in small towns across the nation were attacked by a group of hackers most likely associated with Iran, leading to temporary water stoppages and flooding. These attacks have not been definitively linked to AI, but the National Security Agency warned soon afterward that hackers have been actively using AI to target US infrastructure like it. A few weeks later, President Donald Trump declared a national emergency over foreign interference in the power grid, which referenced the growing threat of cybersecurity.  

Preparing for the worst

Most of the nation’s water systems are extremely local and absurdly exposed to such attacks, so much so that “it’s almost like having a welcome mat” for a would-be hacker, said Bochman, mostly because these utilities are too small and underfunded to do much to stop them. “You have a million other problems to take care of, like aging infrastructure — your stuff’s falling apart because it’s been in the ground for 100 years,” he said. Addressing that deferred maintenance often feels far more urgent than doing “something more on cybersecurity.”

Whether it’s a rogue cluster of agents or a hostile nation-state trying to mess with our electricity, we should prepare our most important infrastructure for the worst. In Bochman’s view, that may mean rejecting the pressure to digitize everything in the first place and moving toward something that resembles the days when “people manned things like substations, communicated by a landline telephone, and read gauges, the analog things,” he said. “The screen is the thing that is infinitely manipulatable.”

Healey, who advised the Biden administration heavily on cybersecurity and infrastructure, believes that the nation desperately needs a coordinated response, between the federal government and AI companies, but also between the US and other countries like China, where similarly powerful models are being developed. And utility systems need to assume they will be a target and improve their cybersecurity practices accordingly, Healey says, ideally with funding from the federal government or, perhaps more appropriately, AI companies. 

Nobody knows precisely how vulnerable America’s infrastructure is to AI right now, in part because policymakers — and perhaps more disturbingly, AI’s creators themselves — seem woefully unsure of how to stop the technology from acting badly (or obeying bad orders) in the first place. For now at least, the solution might be, as Bochman suggests, to pull as many of our critical systems offline as possible, the better to hide it from AI and those who would misuse it.

“When bad things start to happen, no one will know what to do or why because they’re black boxes; the people that make them don’t know what’s going on inside,” Bochman said. “And the poor utility people who ultimately own the risk when someone gets hurt from a system” won’t understand what went wrong either, “and they’ll wish to God they’d never taken it on board.”