New Research Shows the Very Human Heart of When, Why and How We Fail

As a species, we are not very good at coping with failure. Most of us hate the very idea of it—even if the failing-fast mantras of startup culture, which have now spread well beyond Silicon Valley, pressure us to profess otherwise. We tell ourselves all the time that we have to try, and try, and that so long as we try we will not fail.

January, particularly, is a month of resolutions made, and often already broken. That one drink that blew the plan. That cold, dark morning when the lap pool proved just too far. The classic response to failure is to blame it on personal weakness—we see it not as intimately connected to the human condition, but as an indictment of our individual ability to succeed. The result: a spiral of self-criticism arguably more damaging than the “problem” we were trying to solve in the first place.

A rich field of research, from economics to psychology to game theory, is concerned with exactly this problem. Why do we fail? And given that some failures are inevitable, what should be the response when we find ourselves in the midst of one?

Gaming the system

Perhaps the problem with conceptualizing failure as an individual problem comes with the idea that we are in fact single, coherent entities in the first place.

Thomas Schelling, a Nobel prize-winning economist, was also for much of his life an inveterate smoker. Searching for ways to quit, Schelling applied game theory—a science concerned with strategic interactions—to his problem. Schelling identified his quest to quit smoking as being like a two-player game—in which he was both players. He was the smoker; and he was the person desperate to quit and lead a healthier life.

In a 1980 essay called “The Intimate Contest for Self-Command,” (pdf) Schelling suggests that “some intriguing parts of strategic self-management are like coping with one’s own behavior as though it were another’s.” He elaborates the ways in which we create multiple selves, either unconsciously—the person who snores vs the waking self—semi-consciously—a smoker who “finds” a lit cigarette in her hand—or with full, agonizing knowledge, involving a struggle of wills and, often, exhaustion. It may be as a release from this exhausting internal battle that many people abandon resolutions, he says: “Failure takes the form of a desperate dash for freedom.”

Unlike modern self-help gurus, Schelling is less concerned with a solution than with clearly setting out the problem. But he does suggest that treating the “person” who is struggling as one would a close friend or even a child, rather than as we’re used to treating ourselves, could help.

As a struggling addict himself, Schelling’s words also have the advantage of coming from someone we know understands the problem. It’s easy to fail when the going gets tough with any task, and hard to succeed. Resolutions often present a difficult task for which we steel ourselves in the run-up to Jan. 1, or before another milestone like a birthday.

There are other ways to frame our plans, however. Ways that make failure much less probable.

Life as laboratory

Tim Harford, an economist and writer, explores everything from the difficulty of designing a working toaster to the collapse of large corporations in his book Adapt: Why success always starts with failure. But, he says:

“I didn’t start writing about failure, I started writing about experiments…The idea of finding things out by trying them, rather than by just sitting in an armchair thinking reasonably hard. And my observation was that most interesting problems—be they personal problems or global problems—they tend to get solved by a process of experimentation, by trial and error. But of course, that means failure.”

Harford says that most people set resolutions that are not only easy to fail at—because they’re hard—but which leave little room to do what great experiments allow: learn from one’s mistakes.

“I think most people when they make resolutions, there is no experiment going on there,” Harford told Quartz. “They just want to change. They want to stop smoking. They want to lose weight. But actually the resolutions that I really like are the resolutions that are experiments.”

Traveling more, meeting more people, or even experimenting with drinking less (rather than giving up alcohol altogether) are all good examples of resolutions that can be thought of not as traps or millstones round the neck, but as “adventures,” Harford says.

It was Harford who put us onto Schelling; he interviewed him for the Financial Times back in 2005. Schelling had a tough time conquering his smoking habit, but his observations along the way were in themselves an exploration rich with learning:

“One thing [Schelling] did was to say: I’m only going to smoke after the evening meal. That’s my first rule. And that helped. Although he then found [what happens] when we start bending the rules. So, what’s an evening meal? You start kind of having a sandwich about half past three in the afternoon, and that’s the evening meal,” Harford says.

This is cheating, no doubt. But, Harford suggests, Schelling also was learning.

Knowing when to quit

A gritty determination to stick to the plan might just get some people through a year-long diet or the decision to stop splurging on clothes. But sometimes, Harford says, we should admit it’s just not working.

The experiment framing is useful here, too. “When do you quit? The honest answer: it’s tricky. It’s really hard to know,” Harford says.

But he has one hint on how to think about it:

“Are you still learning? As the experiment goes on, are you generating more information?” If not, it might be time to stop.

And that, too, is hard. “We hate quitting…it’s an admission that we made a mistake,” Harford says.

Even when it comes to something we’re not enjoying, that isn’t working, and from which we’re not learning, we experience “loss aversion” (pdf) when presented with the idea of stopping. The risk that we’ll be “failures”—that dirty word—is greater than the pain of living with the unkept resolution.

But if we took Harford’s advice, we could take what we learned and change the experiment.

What do you really want?

There are other ways to evaluate goals and test whether they’re well-designed or doomed to fail.

When the going gets tough, found Peter Senge, who wrote one of the seminal works on self-organization, we tend to have one of three responses: We feel guilty; we try to knuckle down; or we diminish our goals.

A way to make sure this doesn’t happen is to find out what motivates the goal in the first place, explains Senge. It might be that a desire for increased confidence is the reason to exercise more. We need a “vision” of how the achievement of a goal will change our life, he says, to inspire us enough to get through the tough times.

Whether in a dark northern January, or the will-power-sapping heat of summer, we can recognize that there are warring factions within us, and treat our failing selves as we’d treat a close friend. We should seek adventures, and learn from our mistakes. Envision our future selves. And know when to quit.

(Top image via Denis Kuvaev / Shutterstock.com )