Step Seven: Estimate Validation and Review
Step Seven: Estimate Validation and Review
At this point in the process, your estimate should already be reasonably good-but it may not be as good as you think. It is important to verify your methods and your results in a step called validation, which is simply a systematic confirmation of the integrity of an estimate. By validating the estimate, you can be more confident that your data is sound, your methods are effective, your results are accurate, and your focus is properly directed.
It may be tempting to skip this review due to a lack of time, personnel or budget, and there is the unappealing possibility that a close examination may reveal faults in the logic of your process. Nevertheless, the costs involved in performing a proper validation will be dramatically less than the cost overruns that are likely to develop during a poorly managed software project.
There are many ways to validate an estimate. Both the process used to build the estimate and the estimate itself must be evaluated. Ideally, the validation should be performed by someone who was not involved in generating the estimate itself, who can view it objectively. The analyst validating an estimate should employ different methods, tools and separately collected data than were used in the estimate under review.
Step Six: Quantify Risks and Risk Analysis
The best managers of software projects seem to have an uncanny ability to anticipate what can happen to their projects and devise just-in-time mitigation approaches to avoid the full impacts of the problems. In reality, this ability is simply the skillful application of well known risk management techniques to the well known problems of software management. Unfortunately, too many software managers are skilled in seeing potential risks and then ignoring them outright.
Before we explore the risk management process and how to apply it to the risks associated with sizing and estimation, it is important to understand what a risk is and that a risk, in itself, does not necessarily pose a threat to a software project if it is recognized and addressed before it becomes a problem.
Step Five: Prepare Baseline Estimate
Budget and schedule are derived from estimates, so if an estimate is not accurate, the resulting schedules and budgets are likely to be inaccurate also. Given the importance of the estimation task, developers who want to improve their software estimation skills should understand and embrace some basic practices. First, trained, experienced, and skilled people should be assigned to size the software and prepare the estimates. Second, it is critically important that they be given the proper technology and tools. And third, the project manager must define and implement a mature, documented, and repeatable estimation process.
To prepare the baseline estimate there are various approaches that can be used, including guessing (which is not recommended), using existing productivity data exclusively, the bottom-up approach, expert judgment, and cost models.
Software Productivity Laws
These laws of software productivity help explain the dynamics of an engineering development project, and they illustrate some of the reasons that just using productivity to estimate is inadequate.
Law 1 - Smaller teams are more efficient. The smaller the team, the higher the productivity of each individual person.
Law 2 - Some schedule compression can be bought. Adding people to a project, to a point, decreases the time and increases the cost as larger teams work together.
Law 3 - Every project has a minimum time. There is an incremental person who consumes more energy than he or she produces. Team size beyond this point decreases productivity and increases time. (Law 3 is also known as Brooks’ law.)
Ten-step Project Estimation Process
Software Estimation Concepts
Many project managers and project management offices have unrealistic expectations about estimates. The definition of the verb estimate is to produce a statement of the approximate value of some quantity. Estimates are based upon incomplete, imperfect knowledge and assumptions about the future. For these reasons, many estimates of software costs tend to be too low due to omissions of important product functions and project activities. Most importantly, however, all estimates have uncertainty. There is no such thing as a precise, single-value estimate. Managers should always ask how large the uncertainty of an estimate is! A manager can use the size of this uncertainty in conjunction with other factors such as perceived risks, funding constraints, and business objectives to make decisions about a project.
How can projects address the uncertainty of poor estimates? How can the risks associated with initial estimates be identified, managed, and controlled? The answer is straightforward: by defining, establishing, planning, and applying a consistent, repeatable, and effective estimation process.
A software estimation process that is integrated with the software development process can help projects establish realistic and credible plans to implement the project requirements and satisfy commitments. It also can support other management activities by providing accurate and timely planning information. Realistic plans will also describe how the resources that are required to undertake the initiative in accordance with the schedule will be secured. The planning process, as critical as it is, is difficult and takes time to perform correctly. Managers often truncate the planning process by using “easily available” information that is often inadequate; by employing whoever has the time, even if those individuals are not qualified to perform the estimate; and by using only one estimation method to save time.
Successful software engineering requires the application of engineering principles guided by informed management. The principles must them-selves be rooted in sound theory. While it is tempting to search for miracles and panaceas, it is unlikely that they will appear. The best course of action is to stick to age-old engineering principles. There simply are no silver bullets.
Cost estimates are projections of required effort, time, and staffing levels. Because all estimates, particularly those made at the beginning of a project, are based on assumptions, they should be considered probabilistic. Cost estimates in particular should provide a range with an indication of accuracy, i.e., least, probable, and most, with the least and most values representing the upper and lower bounds of the projected cost.
Project Estimation Process
Ideally an estimate should be produced using the ten-step process.
During October we will cover each step on this blog.
Figure 1 Ten-step project estimation process
- Establish estimate scope
- Establish technical baseline, ground rules & assumptions
- Collect data
- Size software
- Prepare baseline estimates
- Quantify risks & risk analysis
- Review, verify, validate estimate
- Generate a project plan
- Document estimate & lessons learned
- Track project throughout development
10 Step Estimation Process Sample Checklist
10 Step Estimating Process Checklist
This should be tuned to the individual companies needs
10 Step Summary
|
Estimation Step |
|
|
1. Define Scope & Purpose |
|
|
2. Identify Technical Baseline and GR&A |
|
|
3. Collect Data |
|
|
4. Software Sizing (new and reused) |
|
|
5. Prepare Baseline Estimate |
|
|
6. Identify Risk items |
|
|
7. Estimate Validation and Review |
|
|
8. Build a Project Plan |
|
|
9. Document Estimate |
|
|
10. Track Project |
|
Risk Analysis & Conference In the Netherlands
I spent this week with our international group… at two conferences (ISPA / SCEA and SSCAG) in the Netherlands.
There were several excellent papers given by Galorath people as well as many SEER users.
Risk analysis has always been one of my favorite features of SEER. And as the risk analyis community matured the risk gurus asked for the ability to fine tune the risk methodology. I was pleased to hear the results of a risk survey and to see how many people use SEER’s internal risk features. SEER includes internal risk analysis and, for those who want to fine tune risk even more, SEER has an interface to Oracle’s Crystal Ball. The Crystal Ball interface allows users to specify more specific information regarding correlations and other risk details. The Oracle people were also at this conference and were very pleased with the SEER use of Crystal Ball. How are you using SEER’s risk features? Do you use SEER’s Crystal Ball interface?
