Project Estimation: A better way

In my many years of development both as a manager and consultant, the one piece of the agile process and especially scrum points is they never seem to give you an estimate that those controlling the budget or making financial choices can use. I am an avid agile proponent and an early adopter (signed the Agile manifesto around 2004). However, the point estimation process of many agile methods leaves the person with the budget sort of holding the bag. There is no easy way to equate "x" number of story points with a monetary cost estimate. In addition, resource assignment is also difficult. Now I know some of you are saying "Yeah but, if the team is established and stable a rough number of iterations can be given, and that’s your answer." but that rarely is the case in the consulting business. The reality is having a ready-made team with history rarely is the case for most consulting jobs and in most organizations.  In the consultants world, the more important information is how much and how long. Iterations, scums, agile etc. have very little meaning at the point we need to plan the project. Once the project begins, we run 2 week iterations with stories and tasks but the estimation process happens pre-engagement. The development process was somewhat irrelevant to the estimate work. Inside most companies, many "go, no go" choices on projects are also done outside the development process with estimates determined by other methods than the typical agile planning.

After four or five projects at SUM Global doing the "estimate to plan to story" conversion thing, it really started to bother me. There was so much time put into the estimate to get it right yet when the project started it was not really used. We typically went back to the requirements and wrote stories. Double the work, for no appreciable benefit. I also found that more often than not, the estimates were off from the agile release plan and nearly always longer than the estimate. I got tired of being behind the delivery "8 ball" all the time.

To go about creating a better, more effective estimation process pre-project I went back to my Agile roots and turned back to eXtreme Programming. Here at SUM Global we had mastered the art of Scrum project management, but had lost our software development methodology. Many companies we consult with (including our own) don’t understand the difference, and tend to have no methodology to go with the management…XP is a fully developed software development methodology (approaching its 23 year anniversary in 2019 I might add) so I went back to my roots. I digress (sorry about that). What I learned in the early 2000’s as I helped champion the move to XP at the company I worked for at the time, was that no matter how well you execute your chosen development methodology, the person managing the budget wants to know only one thing…. how much, and the person in charge of what gets built wants to know how long. They didn’t care about iterations or velocity or any of those things. Software development methods were changing, but the business expected the method to produce the numbers they needed. Its was my job then, and now to translate all this Agile jargon and data into values the folks making business decisions could/can use, and to be as accurate as possible to make those business decisions valid.

The SUM Global Way

I will readily admit that the concepts used are not unique to SUM Global. I read about the "ideal days" concepts and implemented them during my early management years. That being said, the application of the concepts in practice is unique to SUM Global. It has proven to be both practical and extremely valuable to our customers and our business. Here is what that early implementation has evolved into over the last decade.

Requirements Breakdown

The first step is to break the requirements down into Epics. This usually involves some architecture and design to get a feel for the technology stacks, long-term plans for the new project (is this phase 1, or the final evolution), maintainability (ability to find resources in the future to evolve the code) and deployment targets for the project (cloud, internal, OS etc.). Typically this is done with documents from the client and iterative communication with the stake holders. The Epics become the requirements interface between the business stake holders and the development/QA team. Once we all agree the Epics represent the requirements, estimation and planning can begin.

Epics to Stories

If you have stayed with this post so far, this is why you are reading. Step two involves breaking the Epics into Stories. Where we deviate from some Agile methods is the Stories are not always "User Stories" and don’t follow the "as a <insert role> I want to be able to ….". In practice, we have found this to be a very poor way to describe functionality, and it gets a bit wordy for the developer that needs to implement it. Rather than rely on the individual developers interpretation of Stories written in that format, we write them in developer terms. Usually in the format "Create <insert functionality> to <insert the purpose in <insert the layer>". As an example, the story might be "Create a set of RESTful CRUD services to manage addresses in the API layer." In the description we add details like  "The services should be secured and utilize the roles and permissions defined in the <security epic title>".  This does 2 things, it gives the details needed to plan resources and time and from a project stand point, we get to see the interdependence of the functionality each Epic and Story describe. We get to be much more accurate with the estimations this way.  We can assume certain functionality is complete when the story is started. The best part is the iteration/release plan starts to naturally take shape.

Stories to Points

The purest in me likes to say that Story Points have no equivalent to "time" just complexity. The realist in me knows that no decision maker will accept this as an estimate. As I stated before, the real challenge is not the estimation, its being able to keep the work product used to provide the estimate when its time to do the project work. To accomplish this, we put a "time" value on a Story Point. Each point represents an uninterrupted, perfect developer week (40 hours). We discuss the notion of "perfect" all the time, but the general idea is that this is the theoretical amount of work one developer could get done in one 40 hour week with no meetings, no context switching, no phone calls and no fatigue.  Of course this can never happen in real life, but it does eliminate the guess-work around the proposed working environment.The total point value also includes acceptance testing the story. Once we have this measurement of both complexity and some relationship to time we can move forward with the estimation.

Points to Estimate

So with every Epic broken down into Stories no bigger than 2 points we can finally estimate the project. The first step is to simply add the story points for the project. This gives the total ideal developer weeks in the project. So, if we have a project that has 4 Epics with 14 stories and a total story points of 21, we have the first part of the picture. Next, looking at the stories we have to decide if there are any dependencies, and which things can be done in parallel. This gives us a high level release plan that we can use for planning each iteration as well as some idea for the number of optimal resources. In our example case, above, lets say there is a web UI layer, a service layer and a backing database. Depending on the complexity of the model this project could use  2 or 3 development resources.  We now have all but one number we need to give an estimate on cost and delivery date. The only number that is more guess than experience is what we call the utilization factor which represents the actual percentage of and ideal week a developer is actually working on the project. Extremely well run organizations can be as high as 80%, micro-managed or meeting heavy organizations can be as low as 50%, but typically we see about 70% utilization.

So now we have a general idea of how many actual developer weeks the project is going to take. If we take our result and divide them by our projected number of resources (we’ll pick 3 for our example) we get 27.3/3 = 9.1 actual weeks. Now we can give an approximate date of 64 calendar days from the start of the project plus holidays/vacations. We can also provide a rough cost estimate with the 27.3 (which I would typically round-up to 28) giving us 28 weeks * 40 hours per week * the average resource rate (say $200 for our exercise) ≈ $224,000.

Summing it up

The key to getting value from your estimation process is to tie the estimation process to the development process so that the artifacts of your estimations can both feed your iteration planning and answer the questions required by the folks making the business choices. To do both some “compromise” has to be made and points have to have some rooting in time. Its a natural way to think about complexity of a development task, so its not too hard a stretch… “is this hard?” … and the response is “… yeah, looks like it will take me a couple of weeks to figure it out and then a couple more to code it”. We have this conversation all the time outside iteration meetings, and often in them, then we toss that information out and call the point 10. To handle both cases, moving points to a more natural place gives us the proper grounding we need to estimate for the business, give time tables, adjust resources AND feed our iterations. Once we get a true velocity for the team, we adjust the utilization (yep, you guessed it, utilization is a function of iteration velocity). If actual utilization is not at or above the estimate, we have two options to fix it. We can change the day-to-day management of the team to increase velocity by removing distractions (have the team work remote 3 or 4 days a week), shorten or eliminate meetings and remove or streamline any processes or tool issues that are hindering productivity (code delivery is productivity). If this is not possible, then the delivery dates, and costs need to be adjusted to reflect the actual utilization.

Final notes on managing the process

Utilization/velocity is king when it comes to managing any iterative/agile project. Small changes can make big differences in cost and delivery dates. In the example above, just a 5% change in utilization can swing the delivery date +/- 1.5 developer weeks which is a cost swing of $12,000 and moves your delivery date +/- about 4 days (rounding a bit on that one). Obviously if there are more than 3 members on the team it’s even more significant to watch this number. When working with a client, I spend a lot of time talking about this number and things that affect it to manage costs and expectations.