There was an extremely interesting guest post by Eliezer Yudkowsky on I Will Teach You To Be Rich the other day. The upshot was that the more you “know” about the details of a project the less qualified you are to judge how long it will take.

While it sounds counter-intuitive, I’m absolutely 100% on board with this hypothesis. As a ADD-type person who has tried to compensate for her own lack of ability when it comes to estimating by engaging in ever more detailed and anal planning sessions—I have noticed a decidedly inverse relationship between the amount of detailed planning and the accuracy of my own estimates of how long a project will take.

I had attributed this “effect” to my own numerous shortcomings, but apparently it’s not just me.

From I Will Teach You To Be Rich:

This phenomenon is more generally known as the “planning fallacy”. Roughly, the planning fallacy is that people think they can plan.

A clue to the underlying problem was uncovered by researchers who found that asking subjects for their predictions based on realistic “best guess” scenarios, versus asking subjects for their hoped-for “best case” scenarios, produced indistinguishable results.

See, when you ask someone for a “realistic” scenario, they envision every event happening the way they think is normal - which usually means, just like I planned it. No unexpected delays, no unforeseen catastrophes - what people envision by default is the best-case scenario.

Reality, it turns out, usually delivers results somewhat worse than the “worst case”. When you ask people to envision everything that can possibly go wrong, their schedule gets a lot closer to reality - but still not close enough.

Unlike most cognitive biases, the planning fallacy has a simple remedy - though I’ll warn you, you’re not going to like it.

The same researchers asked another group of students to describe highly specific plans for their Christmas shopping - plans that described where, when, and how. Another group was simply asked when they expected to finish their Christmas shopping. The first group, with the detailed plans, expected to finish shopping more than a week before Christmas - the second group expected to finish an average of 4 days before Christmas - and in reality, both groups finished an average of 3 days before Christmas. That’s right: detailed planning made the students more optimistic.

Why? Another study, done in Japan, helps to illuminate the answer: A group of Japanese students expected to finish their essays, on average, 10 days before the deadline. They actually finished 1 day before deadline. Asked when they’d finished previous essays, they said: “One day before deadline.”

You see, you do have a reliable source of information about how well you’ll do. It’s how well you did last time. But the more details you visualize, the more chance you have to be optimistic - to visualize everything going exactly as planned - instead of remembering how long it took last time, when things didn’t go as planned.

I plan on immediately incorporating this information into my basic “monotasking” philosophy, because it’s quite obviously true, useful and awesome. For projects you have not undertaken before, you just find someone who doesn’t know the details but who has done similar projects.

…if you’ve got the strength to stomach it. Just ask how long it took you the last few times, without considering any of the special reasons this project will be different. Better yet, ask an experienced outsider how long broadly similar projects have taken (and be sure not to tell them the details).

You’ll get back an answer that sounds hideously long, and clearly reflects no understanding of all sorts of important particulars. This answer is true. Deal with it.

Amen.

The Planning Fallacy [I Will Teach You To Be Rich]