Step Ten: Track Project throughout Development
Refining Estimates throughout Project
Estimating software size, cost, and schedule should be an ongoing process. Preliminary estimates may be required to bid a job or to initiate the development process, or you may need to conduct a cost-benefit or return-on-investment (ROI) analysis to evaluate a project’s feasibility.
Preliminary estimates are the hardest to develop and are the least accurate because of the incomplete nature of the information available and the other factors discussed.
You can improve the accuracy of a preliminary estimate by using the sizing methodology identified in Step 4 or by using two different estimation techniques and having your analysts normalize the differences. There will still be a significant risk in using the preliminary estimate to structure a project or to evaluate risk in the early stages of a project life cycle.
Once a project has started, you will need to complete more detailed estimates to accurately plan the project and throughout the conduct of the project you will need to monitor the actual effort and duration of tasks and/or phases against planned values to ensure you have the project under control.
Summary
Software cost estimation is a difficult process but a necessary part of a successful software development. You can help ensure useful results by adopting a process that is standardized and repeatable. Several of the steps we have discussed, particularly those that do not result directly in the production of the estimate (Steps 1, 6, and 7) are often deferred or, worse still, not performed at all, often for what appear to be good reasons such as a lack of adequate time or resources or a reluctance to face the need to devise a plan if a problem is detected. Sometimes you simply have more work than you can handle and such steps don’t seem absolutely necessary. Sometimes management is reluctant to take these steps, not because the resources are not available, but because managers do not want to really know what they may learn as a result of scoping their estimates, quantifying and analyzing risks, or validating their estimates. This can be a costly attitude, because in reality every shortcut results in dramatic increases in project risks.
Previous:
Step Nine: Document Estimate and Lessons Learned
Bad Assumptions Invalidate Business Value and ROI Analysis
I recent email announced “ROI estimates in business fail primarily because managers give too much attention to the “payout” odds and too little attention to measuring and managing “probability” odds. A good risk and sensitivity analysis of the assumptions behind the predictions allows you to do both. ”
This is so true. Inappropriate or unlikely assumptions looking for payout based on the absurd (such as assuming that saving 1 minute per day of employees filling in their time card has a huge savings in a year) and a myriad of other traps cause business value and ROI to be appropriately produced using sound business case analysis.
In the same vein, absurd assumptions can cause cost and schedule estimates to be low. SEER evaluates the viability of assumptions, helping produce estimates that can actually be deployed.
Metrics and Estimation ROI
I was speaking with Ton Dekker and others from Galorath this morning abut the ROI from viable estimation. Ton pointed to the empirical evidence of a 10 to 15% saving in overall IT spending based on the implementation of a metrics program and estimation going along with it. Ton also spoke of MOUSE…. the “list of activities and services that need to be carried out to get a metrics program up and running.” The activity and service groups within MOUSE include:
|
Market View |
Operation |
Utilisation |
Service |
Exploitation |
|
Communication |
Application |
Training |
Helpdesk |
Registration |
|
Evaluation |
Review |
Procedures |
Guidelines |
|
|
Improvement |
Analysis |
Organisation |
Information |
|
|
Investigation |
Advice |
|
Promotion |
|
Why is all this important…. Well a viable metrics is not just collecting data… it is not just about checking a box. It is about communication, evaluation, and improvement. Looking at the number of metrics programs that fail, (the average life of a metrics program is 3 years according to Howard Rubin in 2006) It is about achieving business value with metrics. Programs that generate 10 or 15% cost savings generally don’t get killed. Programs that make more work and don’t have a payback can and should be killed.
