Software Estimation Much More Complicated Than Some Perceive
How many times have we been asked to submit a 5 minute or off the cuff estimate to stakeholders… Then be held to that estimate even though we missed the complexity in our haste.. Of course if we didn’t prepare an estimate until everything was known the estimate would be of little value. That is why uncertainty must be part of the process and off the cuff, manual estimations should be avoided when possible.
I thought these diagrams showing the perceived simplicity of the software estimation process versus complexities in the software cost estimation process were very interesting.
Step Ten: Track Project throughout Development
Refining Estimates throughout Project
Estimating software size, cost, and schedule should be an ongoing process. Preliminary estimates may be required to bid a job or to initiate the development process, or you may need to conduct a cost-benefit or return-on-investment (ROI) analysis to evaluate a project’s feasibility.
Preliminary estimates are the hardest to develop and are the least accurate because of the incomplete nature of the information available and the other factors discussed.
You can improve the accuracy of a preliminary estimate by using the sizing methodology identified in Step 4 or by using two different estimation techniques and having your analysts normalize the differences. There will still be a significant risk in using the preliminary estimate to structure a project or to evaluate risk in the early stages of a project life cycle.
Once a project has started, you will need to complete more detailed estimates to accurately plan the project and throughout the conduct of the project you will need to monitor the actual effort and duration of tasks and/or phases against planned values to ensure you have the project under control.
Summary
Software cost estimation is a difficult process but a necessary part of a successful software development. You can help ensure useful results by adopting a process that is standardized and repeatable. Several of the steps we have discussed, particularly those that do not result directly in the production of the estimate (Steps 1, 6, and 7) are often deferred or, worse still, not performed at all, often for what appear to be good reasons such as a lack of adequate time or resources or a reluctance to face the need to devise a plan if a problem is detected. Sometimes you simply have more work than you can handle and such steps don’t seem absolutely necessary. Sometimes management is reluctant to take these steps, not because the resources are not available, but because managers do not want to really know what they may learn as a result of scoping their estimates, quantifying and analyzing risks, or validating their estimates. This can be a costly attitude, because in reality every shortcut results in dramatic increases in project risks.
Previous:
Step Nine: Document Estimate and Lessons Learned
Step Five: Prepare Baseline Estimate
Budget and schedule are derived from estimates, so if an estimate is not accurate, the resulting schedules and budgets are likely to be inaccurate also. Given the importance of the estimation task, developers who want to improve their software estimation skills should understand and embrace some basic practices. First, trained, experienced, and skilled people should be assigned to size the software and prepare the estimates. Second, it is critically important that they be given the proper technology and tools. And third, the project manager must define and implement a mature, documented, and repeatable estimation process.
To prepare the baseline estimate there are various approaches that can be used, including guessing (which is not recommended), using existing productivity data exclusively, the bottom-up approach, expert judgment, and cost models.
Software Productivity Laws
These laws of software productivity help explain the dynamics of an engineering development project, and they illustrate some of the reasons that just using productivity to estimate is inadequate.
Law 1 - Smaller teams are more efficient. The smaller the team, the higher the productivity of each individual person.
Law 2 - Some schedule compression can be bought. Adding people to a project, to a point, decreases the time and increases the cost as larger teams work together.
Law 3 - Every project has a minimum time. There is an incremental person who consumes more energy than he or she produces. Team size beyond this point decreases productivity and increases time. (Law 3 is also known as Brooks’ law.)
Estimating The Cost of Software Independent Verification & Validation (IV&V)
Since IV&V is a level of effort activity (purchase a much as you want) people can buy as much or as little as the want as a level of effort activity. IV&V is not estimated out of the box with SEER-SEM since it is not part of the development itself. Of course some people calibrate in IV&V practices from their organization.
Of course more critical systems would might have more IV&V than less critical ones (which need none) One customer with severe security issues spend about 40% on top of development for IV&V.
Full IV&V is not just independent testing but is an independent look at requirements sufficiency and traceability, design, coding, and testing as well as independent text.
A Titan briefing on the internet showed IV&V as 35 to 45% http://www.dtic.mil/ndia/systems/Rogers2.pdf
The book “Independent Verification and Validation By Robert O. Lewis”
shows IV&V being up to 18% of software development cost.
Wallace and Fuji (1989) report mixed results on the effectiveness of independent verification and validation (IV&V), largely due to the fact that IV&V adds 10 to 30 percent to development costs while saving 0 to 180% of IV&V costs.
