Another Stab at Work Estimation

The Story

A couple of days ago, I was at a client, helping with his company’s migration from one version control system to another. As part of the work, we had to get the latest version of the source (i.e. download them to a local folder on the PC), copy it to the target system’s workspace (i.e. perform a standard windows copy to another folder on the PC), and then check in the sources.

I want to discuss the copying process. Yes, the copy.

Not much to copying you say? Well, essentially, you’re right. You select the folder that you are going to copy from (I selected a folder that was just over 1GB in size, and just over 50,000 files altogether) . Then you copy it to the target folder. That’s it. Oh, and you wait.

That’s what I really want to discuss. The Wait.

This was my experience:

1. First I had to wait for a few seconds (I think it was about 10 seconds) while windows calculated
2. Next, it started copying, and said it would take roughly 3 minutes.
3. After a few more seconds, of copying it reevaluated the work at 4 minutes or so.
4. After that windows started to progressively update the estimate until it reached about 5:30 minutes.
5. At that point, the estimation seemed to be on track, as the remaining time steadily went down.
6. About 5 seconds or so from the end of the copy, it began taking longer for each second to complete (i.e. it took longer than 5 seconds to complete from that point, and it took longer than a second to tick off one second from the remaining time).
7. Finally, Windows reported that it was done. 0 remaining seconds. Means it’s done, doesn’t it? I thought so. Windows seems to have a different definition of done, apparently. It took 10 more seconds till it was really done.

Sounds familiar to anyone? It should. Anyone who ever used a computer (I’ll hazard a guess that other PCs and operating systems have similar experiences) probably doesn’t even notice it anymore, seeing as it is such a common occurrence.

That’s not what I’m talking about, though. I’m talking to all of you corporate developers, team leaders and managers.

The Moral

In my experience many (perhaps most?) teams go through a frighteningly similar cycle whenever they estimate the cost of development on a feature, or a project.

This is (all to often) my experience:

1. The team spends some effort “calculating” the effort required to develop each feature. Usually, it seems like the same 10 seconds are spent, the results are equally optimistic, and just as “accurate”.
2. The team makes their estimate (optimistic and inaccurate).
3. Next, they start working on the problem. Soon enough, the team encounters the first problem they didn’t think of. The estimate goes up.
4. This goes on several times, pushing the deadline into the slack assigned to development.
5. Eventually, the team reaches some rhythm, and it seems like the team is on track. At least they report that they are.
6. Towards the end, when the deadline is just around the corner. In what is sometimes called “Crunch-time”, or the “robustness” phase, the team keeps announcing “we’re almost done”, or “we’ll finish tomorrow”. This usually takes quite a bit more than a day, of course.
7. Then they say that they’re done. Only they’re not. They completed part of the work – coding. “Testing? What’s that – that’s what QA are for. We meant done coding, not ready to deploy. What’s the matter with you?!?” It will take some more time before it actually is fit to let out in public.

And everybody is surprised.

That is what I don’t get. A computer, performing a copy, one of the simplest and straightforward tasks, can’t get the estimates right. And I was copying locally! And that was to another folder on the same disk (even the same logical drive).

How do you expect a human being, performing an act of research & development, doing something nobody did before (or at the very least not done by this human being, in this environment), to out-perform a computer?

I know managers and customers depend on these numbers. I suggest a 12 step program to deal with that. There are alternatives.

Welcome to the world of Agile development.

See you in the next post..

Assaf.

P.S. The next folder I had to copy, I used FastCopy. Took me less than half the time. Didn’t waste any of it on estimations.