Risk Compensation
After adding a safeguard, watch what people now feel comfortable doing with it in place.
When people feel protected, they can start taking bigger chances and give back some of the safety the protection created.
Being able to undo a change made larger releases feel safe
An engineering team adds a button that can quickly restore the previous version of its software, checks that try a new release on a small group first, and a faster way to recover when a release goes wrong. Restoring the previous version is called a rollback.
Before that work, teams release small changes, test them carefully and avoid late deployments. The new protection works, and several faulty releases are detected quickly and rolled back without much customer impact.
Then release behaviour starts to change. Teams bundle more work because restoring the old version feels dependable. A change to stored customer data goes out with the new software, along with a job that sends messages to customers. Testing in smaller stages begins to feel less important, and larger changes are released later in the day.
One release fails. The team restores the old software, exactly as planned, but that can't undo data already rewritten or messages already sent. They still have to check and repair the changed records and deal with messages customers have already received.
Rollback did its job on the code. It also made bigger releases feel ordinary, even when those releases included data changes and customer messages it could not pull back.
That is Risk Compensation: once people feel safer, they may take bigger or more frequent risks and give back part of the safety the protection was meant to create.
Measure the protection and the behaviour change
It is easy to judge a safeguard as if the people around it will keep behaving exactly as they did before. If rollback cuts recovery from thirty minutes to three, the system looks safer on paper.
Once people know rollback exists, they may change how often they release, how late they release, how much they bundle together or how much they depend on one recovery path. A safer rollback tool can quietly lead to a riskier way of releasing.
The complete effect has two parts:
- Mechanical protection. How much harm does the safeguard prevent when behaviour stays the same?
- Behavioural response. How does perceived protection change what people attempt, how often they attempt it and how carefully they proceed?
Risk compensation doesn't require recklessness. A team may bundle two changes to save coordination time, deploy more frequently because recovery is faster or let a less experienced operator act without assistance, and each adjustment can look reasonable on its own while changing the exposure the original safety evidence described.
When failure feels cheaper, people can move faster, bundle more work or rely on the protection more heavily. A few successful recoveries make that confidence grow. Eventually the team can create a failure that sits outside the thing rollback was built to fix.
Treat irreversible data changes and external actions as separate release risks because application rollback can't cover them.
Where this happens
Economics and insurance. Insurance can produce moral hazard when protection from loss changes incentives, which is one contractual form of this broader behavioural response. Risk compensation can also happen without delegation when a person simply feels safer and increases exposure. Insurance makes useful activity possible because one loss no longer destroys the participant. A driver, lender or contractor may accept more exposure when someone else absorbs part of the downside. Deductibles, co-payments, exclusions and monitoring reconnect some decisions with their consequences. The design problem is to preserve the valuable protection without removing so much feedback that risk quietly grows.
Psychology and public safety. Sam Peltzman's 1975 paper on car-safety rules argued that drivers change how they drive when cars feel safer, reducing some of the expected benefit and moving some of the remaining harm towards pedestrians. The original paper made a strong claim that researchers still debate. The more useful lesson is smaller: when you add protection, measure what people do after it arrives instead of assuming their behaviour will stay the same.
Software, product and security. Rollbacks, backups, fraud guarantees and automated recovery all reduce harm, but they also change operating decisions. A reliable backup can encourage greater dependence on one data store. A generous fraud guarantee can reduce customer caution while transferring more detection work to the provider. A rollback system can support faster delivery while hiding the distinction between reversible code and irreversible data or external actions. The safeguard should therefore be tested against the behaviour it's likely to enable, not only the behaviour that existed before it arrived.
In each example, measure the direct protection and the behaviour change after people know it exists.
Set limits for the risks rollback can't cover
Risk Compensation is useful when adding a safety control, backup, fallback, guarantee, approval layer or automation to a system where people can adapt.
Before calling the intervention successful, ask:
- Which exact failures does the protection cover?
- Who knows it exists, and what downside do they believe has changed?
- Which behaviour could increase: size, speed, frequency, concentration or dependence?
- Which failure modes remain outside the protected boundary?
- What early signal would reveal that behaviour has changed?
- Which limit should move with the new protection?
- Does total harm still fall after the behavioural response is included?
Keep the ability to restore the old software, but review changes to stored data and actions such as sending messages separately. Limit how much changes at once, and allow time to watch a risky release before starting another. Check whether teams are releasing later, bundling more work, or needing more repairs now that they feel safer.
Where the model gets misused
Risk Compensation is often overstated into a claim that safety measures are useless because people always become more careless. The evidence doesn't support that general conclusion.
People may not notice the protection, may not change their behaviour, or may adjust too little to offset the mechanical benefit. Seat belts, backups and medical treatments can remain strongly beneficial even when some adaptation occurs. The effect must be measured in the specific setting.
The model can also become a way to blame users for poorly designed systems. If a product advertises complete protection while hiding exclusions, predictable reliance is partly a design consequence. Operators need clear boundaries, useful feedback and recovery paths, rather than a lecture about complacency after the uncovered failure occurs.
Removing rollback would make ordinary release failures worse. Keep it, then put separate limits around data changes and messages it cannot undo.
How this differs from moral hazard and iatrogenics
The Principal-Agent Problem includes cases where one party can take risks while another bears the result. Moral Hazard is the more specific insurance and contract pattern in which protection from downside changes incentives. Risk Compensation is broader: behaviour can change without deception, delegation or a conscious attempt to transfer harm. A person may simply feel safer and act accordingly.
Iatrogenics covers any harm created by an intervention intended to help. Risk Compensation names one particular path to that harm: the intervention changes risk-taking behaviour. A safeguard can also be iatrogenic through complexity, delays or direct side effects even when behaviour stays the same.