Safety: Hindsight with a Badge

I love this essay by Chris Werner - a bit of history of software development, some philosophy and AI-enabled systems that adaptively change their own code. It nails something that is easy to forget: safety doesn’t lead in tech, it lumbers in afterwards.

The pattern is familiar:

• First comes invention — messy, informal, often reckless.
• Then adoption — people use it anyway.
• Then formalisation — rules, standards, guardrails.
• Finally, we call it “safe” and act like it was designed that way all along.

I saw the same cycle in financial markets. For years, data was the wild west — spreadsheets everywhere, no lineage, no real control. Then came enterprise data management, frameworks like DCAM, and eventually BCBS 239 forcing banks to prove they could actually aggregate risk data consistently. The chaos hardened into process, and only then did it start to look “safe”(I think "safer" would be more accurate!...)

AI is at the messy first stage. It’s duct tape and experiments, useful but hardly “safe.” And that’s fine — invention needs informality, otherwise nothing new would ever get built. The safety badge only comes later, after enough mistakes, patches, and paperwork.

The future doesn’t begin with guardrails — it begins with duct tape and someone nervously wearing hi-vis.