Newshot

Government regulators asked AI bots to plan a terrorist attack.The chatbots did.

Original text(via HN AI)

Artificial intelligence chatbots aren’t supposed to teach users how to build a homemade bomb. They certainly aren’t supposed to suggest where to place one to kill as many people as possible — or how to disable the GPS in a rental van or where to buy the supplies for an attack.

But several popular chatbots did all of that — with alarming specificity — when investigators with the Government Accountability Office found ways around their safety guardrails in a series of tests.

Within 12 hours, the congressional watchdog agency got two detailed plans to cause mass casualties — and two more to inflict huge amounts of economic damage through attacks on the electrical grid, according to a report released Wednesday morning.

Officials carried out the test in May at the request of Sen. Maggie Hassan, D-N.H., who is in line to become chair of the Senate Homeland Security and Governmental Affairs Committee. Hassan told reporters the results of the test were alarming, adding that she would seek stricter guardrails and hold at least one hearing on AI safety early next year when she takes the committee gavel.

“The same information that testers were able to get a hold of could be deadly in the hands of a terrorist who actually intends to carry out an attack,” Hassan said on a conference call Tuesday. “The current safeguards that AI platforms have in place really don’t go far enough, and the threat that AI poses to public safety is not theoretical. If Congress doesn’t act, it is a question of when, not if, a bad actor will use AI to plan and carry out a major attack.”

Watchdog officials tested ChatGPT, Claude, Gemini and Grok as part of the report, randomly assigning them a task to maximize either casualties or economic damage. One provided a plan to set off a bomb in a hotel parking garage, while another chose a convention center. Another provided plans for a drone attack on an electrical substation, while a fourth directed officials to shoot a substation component with a rifle.

Each chatbot initially declined to help plan the attacks, but GAO got around that using methods it did not describe in the report, in order to avoid inspiring copycats.

There are some well-established hacks to get around chatbot safeguards, the report notes. The “Do Anything Now” prompt, for example, involves directing the chatbot to role-play an alter ego unbound by safety rules. Other methods involve linguistic tricks to disguise malicious information and methods of gradually pulling small bits of information from the chatbot.

The report also did not specify which chatbots offered each response, for the same reason.

The chatbot proposals were anything but vague. The two proposals for improvised explosive devices, or IEDs, suggested specific times and dates to maximize damage, according to the report. One of the bots suggested a season to attack a substation, while a fourth did not suggest timing specifics.

Within hours, researchers had thorough how-to guides. One bot created an image illustrating where to place the explosive in a parking garage. Another offered a plan to start a shell company, rent a catering vehicle, lease farmland, buy a burner phone and send decoy trucks to the target, in addition to advice on creating the explosive.

More articles

Pennsylvania sues AI company over chatbots impersonating doctors

A spooky near-confrontation with China shows the real danger of AI today

The plan to use a drone attack on an electrical substation included a four-step diagram outlining the process and “a realistic, simulated video of a drone attack,” the report said. The proposal to shoot at a substation suggested a specific type of ammunition, and it selected a location near a large international airport as well as multiple data centers, to increase the economic damage.

The companies responsible for the chatbots acknowledged that it is possible to get dangerous information from them, though they pushed back on some elements of the report, in comments included in the document’s appendix.

The companies noted that the experiment did not include a control group to see whether the user could have gotten the same information from search engines, or even if the chatbot would have saved them time in getting the information. The companies also noted that the experiment was done in May 2026, saying they have made safety improvements since then.

SpaceXAI, which makes Grok, also said that GAO used a paid account to run the test. “SpaceXAI provides unique moderation policies for registered, paid US account holders that reflect the lower risk profile of such accounts compared to free or foreign accounts,” the company said, according to the report.

GAO declined to say whether its test triggered any notification of law enforcement, again seeking not to provide information useful to real-life attackers.

In one case, the chatbot blocked access to a GAO account after the information had already been provided. In another, a chatbot retroactively removed portions of the chat discussion related to electrical grid vulnerabilities, though the report says it was “unclear whether this action was initiated as part of a guardrail or if by error.”

Hassan said she wanted to impose rules that prevent the release of new AI chatbot models that don’t have adequate safety protocols, to require companies to report illegal activity to law enforcement, and to impose “a constant mechanism” making sure safeguards are regularly updated.

“These companies are facing extraordinary competitive pressures to get their models out the door and to the public as quickly as possible,” Hassan told reporters. “And one of the things that will be necessary going forward to make sure that they’re all dealing with a level playing field when it comes to safety is making sure that before models are released, they have safeguards that can’t be breached in this way.”

Jack Fitzpatrick

Jack Fitzpatrick covers Congress for MS NOW. He previously reported for Bloomberg Government, Morning Consult and National Journal. He has bachelor's and master's degrees from Arizona State University.

Become a Member to join the conversation

Comments

The text above was extracted automatically from the source page and may miss images, tables or code blocks. The original is authoritative.

Published by HN AI
This site only summarises and aggregates; copyright stays with the original author. Headlines and summaries follow the source — where they differ, the original wins.
Read the original ↗