The enterprise AI discourse has a peculiar blind spot: it treats AI agent failures as unprecedented. They are not. There is a 75-year-old discipline — Human Factors Engineering — that has studied exactly what goes wrong when you automate human work. The foundational problems were identified decades ago. What AI agents have done is not create new problems. They have taken old, well-documented problems and stripped away every mechanism we had for detecting them.
Three Things the Field Already Knows
1. The Fitts List: What Should Be Automated?
In 1951, Paul Fitts published what became known as the MABA-MABA list — “Men Are Better At, Machines Are Better At” — a framework for deciding which functions to assign to humans and which to machines. In 1978, Sheridan and Verplank expanded this into a ten-level automation scale, replacing the binary human-or-machine split with a spectrum from fully manual to fully autonomous.
The Fitts framework has been critiqued and refined for decades, but its core question remains the right one: before you automate, ask what belongs to the machine and what belongs to the human. The enterprise AI agent conversation has almost entirely skipped this question. The discourse is about how to deploy agents, how to govern them, how to scale them. Almost no one is asking whether a given task should be an agent at all.
2. The Substitution Myth: You Can’t Just Swap a Human for a Machine
In 2002, Dekker and Woods identified what they called the “substitution myth” — the assumption that you can replace a human function with a machine function without changing how the rest of the system works. This assumption is always wrong. Inserting automation creates new tasks for the human: monitoring the automation, interpreting its outputs, deciding when to intervene, managing handoffs. These new tasks are often harder than the original work, and — ironically — they tend to require exactly the kind of sustained vigilance that Fitts’ original list identified as something humans are bad at.
3. The Ironies of Automation: The Harder It Works, the Worse It Fails
In 1983, Lisanne Bainbridge published a five-page paper titled “Ironies of Automation” that became one of the most cited papers in the history of human factors research. Her argument: automating most of a system’s operations does not eliminate problems with the human operator — it expands them.
Two ironies sit at the core. First: the designer automates everything except the tasks the designer cannot figure out how to automate. The human operator is left with, by definition, the hardest parts. Second: because daily work is now automated, the operator’s manual skills atrophy from disuse. But when the automation fails and the human must take over, the situation is almost certainly abnormal — meaning the operator needs more skill than usual, not less, at precisely the moment their skills are at their weakest.
Bainbridge’s paper has survived for over forty years because it describes a structural relationship between humans and automated systems, not a fact about 1983 technology. Skill atrophy, vigilance limits, the gap between monitoring and doing — these are properties of the human side of the equation. They do not change because the automation runs on neural networks instead of relay circuits.
What AI Agents Actually Changed
These three frameworks were built for a world where automation was deterministic, expensive, and engineered by specialists. AI agents have broken all three assumptions. Each break does not introduce a new category of problem. It disables a specific safeguard that previously kept the old problems manageable.
Automation is now non-deterministic. Traditional automation produces the same output for the same input, every time. When it fails, you can reproduce the failure, trace the cause, and fix it. AI agents are stochastic. The same input may produce different outputs on different runs. An agent’s failure mode on Tuesday may be unreproducible on Wednesday. This does not create a new class of problem — it disables debugging, which was the primary mechanism for catching Bainbridge’s ironies before they caused harm. The ironies still apply. You just can’t see them happening.
Automation creation has been democratized. Building a factory automation line required engineers, capital expenditure, and months of planning. This cost structure was itself a safeguard: it forced you to understand the process before you automated it, because getting it wrong was expensive. AI agents can be built by anyone in minutes at near-zero cost. The economic forcing function that made you do Fitts’ analysis — should this be automated? — is gone. Not because the question stopped being important, but because the cost of skipping it dropped to zero. People are not skipping the question out of negligence. They are skipping it because the financial guardrail that used to make them ask it no longer exists.
Outputs are in natural language. Traditional automation produces outputs that are auditable by nature: numbers, positions, states, measurements. These can be compared against specifications. AI agent outputs are natural language — fluent, confident, well-formatted text. Natural language inherently resists auditing because there is no specification to compare it against. A misrouted invoice triggers an error. A subtly wrong summary looks exactly like a correct one. This does not create a new problem. It disables detection, which was the mechanism that told the human operator “this is where you take over.” Bainbridge’s skill-atrophied operator at least got an alarm. The AI-era operator does not even get that.
What This Means
The standard framing — “AI brings new risks that require new governance” — is approximately backwards. The more precise framing is that AI agents have reactivated failure modes the automation field identified decades ago, while simultaneously disabling the safeguards that previously kept those failure modes from becoming catastrophic.
The problems are old. The detection mechanisms are gone. That is the actual risk profile.
For organizations deploying AI agents: the relevant literature is not in the AI safety papers published last year. It is in Bainbridge (1983), Fitts (1951), Sheridan and Verplank (1978), Dekker and Woods (2002), and Parasuraman, Sheridan, and Wickens (2000). The failure modes you are about to encounter were described, in some cases, before the internet existed. The question is whether you will read them before or after the post-mortem.
The author is an independent AI security and evaluation specialist.
References
Bainbridge, L. (1983). Ironies of Automation. Automatica, 19(6), 775–779.
Dekker, S.W.A. & Woods, D.D. (2002). MABA-MABA or Abracadabra? Progress on Human-Automation Co-ordination. Cognition, Technology & Work, 4, 240–244.
Fitts, P.M. (1951). Human Engineering for an Effective Air Navigation and Traffic Control System. National Research Council, Washington, DC.
Parasuraman, R., Sheridan, T.B., & Wickens, C.D. (2000). A Model for Types and Levels of Human Interaction with Automation. IEEE Transactions on Systems, Man, and Cybernetics, 30(3), 286–297.
Sheridan, T.B. & Verplank, W.L. (1978). Human and Computer Control of Undersea Teleoperators. MIT Man-Machine Systems Laboratory, Cambridge, MA.
Scopiology 