Software Estimation Much More Complicated Than Some Perceive
How many times have we been asked to submit a 5 minute or off the cuff estimate to stakeholders… Then be held to that estimate even though we missed the complexity in our haste.. Of course if we didn’t prepare an estimate until everything was known the estimate would be of little value. That is why uncertainty must be part of the process and off the cuff, manual estimations should be avoided when possible.
I thought these diagrams showing the perceived simplicity of the software estimation process versus complexities in the software cost estimation process were very interesting.
Step Ten: Track Project throughout Development
Refining Estimates throughout Project
Estimating software size, cost, and schedule should be an ongoing process. Preliminary estimates may be required to bid a job or to initiate the development process, or you may need to conduct a cost-benefit or return-on-investment (ROI) analysis to evaluate a project’s feasibility.
Preliminary estimates are the hardest to develop and are the least accurate because of the incomplete nature of the information available and the other factors discussed.
You can improve the accuracy of a preliminary estimate by using the sizing methodology identified in Step 4 or by using two different estimation techniques and having your analysts normalize the differences. There will still be a significant risk in using the preliminary estimate to structure a project or to evaluate risk in the early stages of a project life cycle.
Once a project has started, you will need to complete more detailed estimates to accurately plan the project and throughout the conduct of the project you will need to monitor the actual effort and duration of tasks and/or phases against planned values to ensure you have the project under control.
Summary
Software cost estimation is a difficult process but a necessary part of a successful software development. You can help ensure useful results by adopting a process that is standardized and repeatable. Several of the steps we have discussed, particularly those that do not result directly in the production of the estimate (Steps 1, 6, and 7) are often deferred or, worse still, not performed at all, often for what appear to be good reasons such as a lack of adequate time or resources or a reluctance to face the need to devise a plan if a problem is detected. Sometimes you simply have more work than you can handle and such steps don’t seem absolutely necessary. Sometimes management is reluctant to take these steps, not because the resources are not available, but because managers do not want to really know what they may learn as a result of scoping their estimates, quantifying and analyzing risks, or validating their estimates. This can be a costly attitude, because in reality every shortcut results in dramatic increases in project risks.
Previous:
Step Nine: Document Estimate and Lessons Learned
Step Nine: Document Estimate and Lessons Learned
Each time you complete an estimate and again at the end of the software development, you should document the pertinent information that constitutes the estimate and record the lessons you learned. By doing so, you will have evidence that your process was valid and that you generated the estimate in good faith, and you will have actual results with which to calibrate your estimation models. Be sure to document any missing or incomplete information and the risks, issues, and problems that the process addressed and any complications that arose. Also document all the key decisions made during the conduct of the estimate and their results and the effects of the actions you took. Finally, describe and document the dynamics that occurred during the process, such as the interactions of your estimation team, the interfaces with your clients, and trade-offs you had to make to address issues identified during the process. Cost models, which are based on the actual costs of past projects, can be calibrated and their accuracy can be demonstrated by comparing the costs of your current estimates with both past project data and the actual costs of your completed project, thereby adjusting the model input parameters to improve future accuracy.
Step Eight: Generate A Project Plan
The process of generating a project plan includes taking the estimate and allocating the cost and schedule to a function and task-oriented work breakdown structure. Models such as SEER Client for Microsoft perform this function automatically. The eight major software development phases
are: (1) concept, (2) acquisition, (3) requirements, (4) design, (5) code and unit test, (6) integration, (7) acceptance, and (8) post deployment.
Determining Costs from Effort Estimates
At this point in the estimation process, you should have a reasonably accurate projection of your project’s size and required effort, that is, an estimate of the number of person-hours by component and a sum of these projections, and you can now begin to price the estimate. As software estimation models generally account only for costs related directly to development, you may need to translate the required effort to a cost and finalize the estimate by adding in essential nonlabor costs. You can do so by answering the following questions:
What types of individuals do I need and when do I need them?
Identify the specific personnel requirements by task area (direct software management, software systems engineering, design, programming, configuration management, quality assurance, etc.). Develop a strawman schedule from the work breakdown structure or a staffing plan such as the one produced by SEER-SEM.
How experienced do they have to be? Assign specific staff levels to the task requirements and identify the level of experience required to satisfy the task. Some of the automated cost models will identify the tasks and develop a task-based schedule, which will minimize but not eliminate all of the work required to produce the software development plan.
Step Seven: Estimate Validation and Review
Step Seven: Estimate Validation and Review
At this point in the process, your estimate should already be reasonably good-but it may not be as good as you think. It is important to verify your methods and your results in a step called validation, which is simply a systematic confirmation of the integrity of an estimate. By validating the estimate, you can be more confident that your data is sound, your methods are effective, your results are accurate, and your focus is properly directed.
It may be tempting to skip this review due to a lack of time, personnel or budget, and there is the unappealing possibility that a close examination may reveal faults in the logic of your process. Nevertheless, the costs involved in performing a proper validation will be dramatically less than the cost overruns that are likely to develop during a poorly managed software project.
There are many ways to validate an estimate. Both the process used to build the estimate and the estimate itself must be evaluated. Ideally, the validation should be performed by someone who was not involved in generating the estimate itself, who can view it objectively. The analyst validating an estimate should employ different methods, tools and separately collected data than were used in the estimate under review.
Step Six: Quantify Risks and Risk Analysis
The best managers of software projects seem to have an uncanny ability to anticipate what can happen to their projects and devise just-in-time mitigation approaches to avoid the full impacts of the problems. In reality, this ability is simply the skillful application of well known risk management techniques to the well known problems of software management. Unfortunately, too many software managers are skilled in seeing potential risks and then ignoring them outright.
Before we explore the risk management process and how to apply it to the risks associated with sizing and estimation, it is important to understand what a risk is and that a risk, in itself, does not necessarily pose a threat to a software project if it is recognized and addressed before it becomes a problem.
Step Five: Prepare Baseline Estimate
Budget and schedule are derived from estimates, so if an estimate is not accurate, the resulting schedules and budgets are likely to be inaccurate also. Given the importance of the estimation task, developers who want to improve their software estimation skills should understand and embrace some basic practices. First, trained, experienced, and skilled people should be assigned to size the software and prepare the estimates. Second, it is critically important that they be given the proper technology and tools. And third, the project manager must define and implement a mature, documented, and repeatable estimation process.
To prepare the baseline estimate there are various approaches that can be used, including guessing (which is not recommended), using existing productivity data exclusively, the bottom-up approach, expert judgment, and cost models.
Software Productivity Laws
These laws of software productivity help explain the dynamics of an engineering development project, and they illustrate some of the reasons that just using productivity to estimate is inadequate.
Law 1 - Smaller teams are more efficient. The smaller the team, the higher the productivity of each individual person.
Law 2 - Some schedule compression can be bought. Adding people to a project, to a point, decreases the time and increases the cost as larger teams work together.
Law 3 - Every project has a minimum time. There is an incremental person who consumes more energy than he or she produces. Team size beyond this point decreases productivity and increases time. (Law 3 is also known as Brooks’ law.)
Step Four: Software Sizing
Size is generally the most significant (but certainly not the only) cost and schedule driver. Overall scope of a software project is defined by identifying not only the amount of new software that must be developed, but also must include the amount of preexisting, COTS, and other software that will be integrated into the new system. In addition to estimating product size, you will need to estimate any rework that will be required to develop the product, which will generally be expressed as source lines of code (SLOC) or function points, although there are other possible units of measure. To help establish the overall uncertainty, the size estimate should be expressed as a least-likely-most range.
Predicting Size
Whenever possible, start the process of size estimation using formal descriptions of the requirements such as the customer’s request for proposal or a software requirements specification. You should reestimate the project as soon as more scope information is determined. The most widely used methods of estimating product size are:
Step Three: Collect Data
Any estimate, by definition, encompasses a range of uncertainty, so you should express estimate inputs as least, likely and most rather than characterizing them as single data points. Using ranges for inputs permits the development of a viable initial estimate even before you have defined fully the scope of the system you are estimating.
Certain core information must be obtained in order to ensure a consistent project estimate. Not all data will come from one source and it will not all be available at the same time, so a comprehensive data collection form will aid your efforts. As new information is collected, you will already have an organized and thorough system for documenting it.
A general form should be customized for each job to delete the parameters that are not relevant to the current estimate as well as parameters that may be gleaned from the provided documentation. Generally speaking, the fewer questions you need to ask your sources, the happier they will be to participate.
Collected data is grouped into relevant categories which are then assigned unique identifiers which describe the attribute (S=sizing, P=productivity, etc.). It is further identified in terms of its description, whether it is required at the start of the estimate or whether it will evolve as the estimate proceeds, and from whom the information must be gathered.
Step Two: Establish Technical Baseline, Ground Rules, and Assumptions
Establish Technical Baseline, Ground Rules, and Assumptions
To establish an accurate technical baseline, you must first define the functionality that must be estimated. Recognize the constraints associated with the application and the project, and determine what functionality must be developed versus what can be provided via COTS or reuse.
Ground rules are concise statements that describe the basis from which the estimate is made. “This estimate includes functions a, b, and c only; no costs associated with travel are included” is an example of a ground rule.
Assumptions are suppositions that describe unknown variables that will affect an estimate. “This estimate assumes the software developer will use development system X” is an example of an assumption. Later in the project when you confirm that “development system X” will be employed, this proven assumption would then be restated as a ground rule.
Ground rules and assumptions form the foundation of the estimate and, although in the early stages of the estimate they are preliminary and therefore rife with uncertainty, they must be credible and documented. Review and redefine these assumptions regularly as the estimate moves forward.
Previous:
Step One: Establish estimate scope
Coming Next:
Step Three: Collect Data
