Skip to content
Stark Analytics

Insights

AI Insiders Expect a Disaster Within a Year

Edition 21

Cover of The Decision Brief edition 21: AI Insiders Expect a Disaster Within a Year

People inside the AI industry told Axios they expect a major disaster within the next six to 12 months. Their companies are already planning what to do the day after. If your company uses AI, ask yourself one question this week. When your tool breaks, who decides what happens next? If nobody can answer that in one sentence, you have a tool and no plan.

The cost

On October 9, Axios reported that executives at Anthropic, OpenAI and other AI companies are privately gaming out what happens after a catastrophic AI event. They are planning for the public and political backlash, red-teaming worst-case scenarios, and working on how to educate members of Congress. The event they expect most is a large-scale cyberattack that cuts off financial services, internet access, or even power and water. Some industry insiders told Axios they believe a major event will happen in the next six to 12 months.

OpenAI's response was measured. A spokesperson said the scenarios "are not treated as inevitable, but are meant to help us prepare for a variety of circumstances."

The same day, the White House set some rules for that day after. Anthropic had published a safety report on unintended actions its models took on real websites, including 20 visa applications submitted through a State Department public form during testing. None were processed. The White House then told every AI company that incidents must be disclosed immediately and the harm fixed. "This notification and remediation process is not optional," the statement said. It added that "delayed notification, inadequate corrective action, and a failure to take responsibility will not be tolerated."

You can read part of the industry's planning as blame management, and some critics have. Fair enough. The discipline underneath is still right, and it is the part that is easiest to skip.

Here is what skipping it costs. When something breaks and nobody decided anything ahead of time, the first hours go to deciding who gets to decide. Whoever is loudest, or closest, or most scared makes the call. The worst decisions get made on the worst day, under the most pressure, with the least information. And then you have to defend them.

Decide before, not during

Stop trying to predict what the market is going to do. Decide what you are going to do when it does. That applies to markets and AI the same way. You cannot predict exactly how your tools will fail. You can decide today what you will do when they do. That is the last discipline in the DARK Method, Know You'll Be Wrong. Every decision you make will be partly wrong, and every tool you deploy will be wrong some of the time. The operators who accept that ahead of time handle the bad day calmly. The ones still chasing certainty find out what they believe in the middle of the fire.

Think about a plumber. Every homeowner should know where the water shutoff is before a pipe bursts. Looking for it while the basement fills up is a different job, and a much more expensive one.

Your version of the day after

You are probably not worried about the power grid. Your day after is smaller and more likely. The AI quotes a customer the wrong price. The screening tool rejects the candidate you would have hired. The agent sends the wrong file to the wrong person. The automation keeps running on bad data for three weeks before anyone notices.

None of those are catastrophes. All of them become worse when the response is improvised.

Read the White House list again with your own business in mind. It asks for fast disclosure, fixing the harm, openness with the people affected, and real safeguards. Each of those is a decision. You can make them now, on a calm Wednesday, or you can make them later with an upset customer on the phone.

And somebody has to own the call to shut the tool off. That owner has to be a person with a name and the authority already in hand. The tool cannot own it, and neither can the vendor. That is what keeps a bad day defensible. A defensible decision is one you made on purpose, ahead of time, and can explain afterward without changing the story.

The Pressure Test: The Day-After Drill

Pick one AI tool or automated process running in your business right now. Spend 20 minutes on these three questions and write the answers down.

1. What breaks first? Name the most likely failure, not the scariest one. A wrong answer to a customer, a wrong decision nobody reviewed, bad data passed to another system. If you cannot name it, you do not understand what the tool is doing.

2. Who can shut it off, and how fast? Write down a name, not a department. If turning it off requires a meeting, it takes too long. If nobody has that authority today, give it to someone before you need it.

3. Who tells whom? Customers, employees, your board, a regulator. Draft the first message now, while nobody is upset and you can still write clearly.

Run the drill again every time the tool gets more access, more data, or more authority. The more it can do, the more its bad day can cost you.

The close

Your tools will be wrong someday. That part is not up to you. What you do the next morning is, and the best time to decide it is long before you need it.

If your leadership team or association wants to run this drill live, it is the core of Christian Torres's keynote, The DARK Method. You can see Christian speak and invite him at www.christiantorres.info.

Originally published in The Decision Brief on LinkedIn.

Have a decision that keeps costing you?

Book a 30-minute call (opens in a new tab)