Reasons For Estimation Failure
Estimation is a core process that can be a root cause of project success or failure. A paper by Andy Nolan of Rolls Royce and published by INCOSE (I can’t locate the original paper on the web) identifies the 10 reasons for failure of estimation.. He also identified the allocation of estimation error. 19% being attribute for estimation failure:due to tools, 37% related to process issues, and 44% related to behavior issues.
Here are each of the 10 reasons:
1 Scope creep: Often the result of not having a clearly defined scope and/or not tracking and declaring changes to scope.
2 Poor input to estimate: This refers to poor quality requirements or objectives, leaving the estimate open to risk and scope variation. The estimate is “built on sand” but the estimator does not factor for this or clearly express the uncertainty as a 3-point estimate.
3 Failure to clearly define the initial scope: Items may be missed out or there may be a failure to declare assumptions. This will make the project vulnerable to scope creep or conflict later in the project as all parties argue over what is in scope and what is out of scope.
4 Unrealistic expectations and assumptions: Optimistic assumptions that it will go right this time. Claiming benefit from improvements before they have been demonstrated. Overruling the estimate.
5 Failure to declare, track and reduce risk and uncertainties. The project may then be vulnerable to unnecessary risk.
6 Lack of internal peer review: Failing to benefit from peers‟ experience. A review is a simple and cost effective way to improve estimate “Quality”
7 Lack of estimation experience: Estimates generated by a person who may not have experience in estimating or in the subject being estimated.
8 Failure to consider environmental factors: Undeclared and untracked elements of the environment can change – having unexpected and “dramatic” effects for examples, loss of key individuals, change of supplier, a more complex organisation, etc.
9 Failure in the estimation tool/process: The tool or process may be unable to deliver the desired accuracy. It is essential to measure and declare the estimate tolerance of any tools used.
10 Lack of estimation process/technique: Failing to apply a systematic estimation tool, method or processes. Without this, it is hard to defend the estimate.
Thank you for reading “Dan on Estimating”, if you would like more information about Galorath’s estimation models, please visit our contact page, call us at +1 310 414-3222 or click a button below to ask sales questions, sign up for our free library or schedule a demo.
Apple IOS 7: Adding People To a Late Project Makes It Later
I am an Apple IOS fan. I use an iPhone and am writing this on my iPad. And I am a geek so I wait anxiously for the next versions of iPhone, iPad, and IOS. I read the iPhone and iPad rumors at least weekly. I also get to see the results of staffing and other decisions based on staffing in numerous programs.
Recent batches of Apple rumors state that IOS 7, the new iPhone / iPad operating system is late and Apple may have to ship the newest hardware devices without IOS 7. Apple’s solution…. Take staff off the Macintosh operating system development and put them on IOS 7.
Basic software productivity laws (see law 3 below) state that adding people to a late project makes it later. This was first observed by Frederick Brooks and documented in his book “The Mythical Man month.” And taking people without domain knowledge just makes things worse.
Now there is some chance this might help. so long as their staff doesn’t increase to the point of Brooks Law comes into play. Additionally, absorbing people into the IOS project from outside will cause the IOS core team to slow down as entropy kicks in. We have developed a set of productivity laws that cover these phenomena. Some are based on Brooks Law while others are of our own discovery.
Software Productivity Laws:
Thank you for reading “Dan on Estimating”, if you would like more information about Galorath’s estimation models, please visit our contact page, call us at +1 310 414-3222 or click a button below to ask sales questions, sign up for our free library or schedule a demo.
Elminating Technical Debt: Easy?
An excellent article on eliminating technical debt… The Question “can we eliminate technical debt” the answer yes so long as we have all the time and resources to eliminate technical debt based on:
- Technical Debt Due To Lack of Analysis: Analyze and fully understand the process you need to computerize.
- Technical Debt Due To Lack of best Practices: Write code so that it is perfectly consistent with the best practices of the various technologies used.
- Technical Debt Due to Lack of Rework Integration: Change existing code so that it’s seamlessly integrated with the newly created code. This also requires a perfect understanding of the existing code.
- Technical Debt Due To Lack of A Test Suite: All the time and resources necessary to create a suite of automatic and integration tests for testing the various features of the written code one by one.
- Technical Debt Due To Lack of Technical Documentation: All the time and resources necessary to write the technical support documentation for future changes.
And of course I must add:
6. Technical Debt Due to Resource Constraints: Too few or too many people on the project causing issues as well as inadequate schedules.
7. Technical Debt Due To Poor Planning: Either lack of planning or not re-planning when things change.
8. Technical Debt Due To People Being Human: People are still human so there will be mistakes that don’t get caught. That is just life with software.
Thank you for reading “Dan on Estimating”, if you would like more information about Galorath’s estimation models, please visit our contact page, call us at +1 310 414-3222 or click a button below to ask sales questions, sign up for our free library or schedule a demo.
Standard Estimating Gates
A customer recently asked us for a standard set of estimating points. And among us we didn’t have an answer off the top of our heads… We looked at PMI, ITIL, COBIT, etc. and found no standard. We deal with estimating gates all the time… So how come we didn’t have any standards. It struck me that the estimating points match many of the decision gates, the development gates. So one of our PMI certified Project Managers called the PMI offices. They said they don’t mandate estimation points or gates since they are so diverse among organizations. And they suggested where there is a key deliverable, one the project cant live without, that is generally a gate.
PMI’s PMBOK states “cost estimates should be reviewed and refined during the course of the project to reflect additional detail as it becomes available and assumptions are tested. The accuracy of a project estimate will increase as the project progresses through the project life cycle. For example a project in the initiation phase may have a rough order of magnitude (ROM) estimate in the range of -25% to +75%. Later in the project, as more information is known, definitive estimates could narrow the range of accuracy to -5% to +10%. In some organizations, there are guidelines for when such refinements can be made and the degree fo confidence or accuracy that is expected” In other words PMI remains silent on standard estimation points.
So.. Here is a proposed set of standard estimation points. These gates are often tied to how an organization funds money for projects. So they need to be tailored to the governance approaches being used:
Concept Level Estimate
Just a finger in the wind to see if the idea is worth exploring or is so expensive as to be impractical.
Business Case Estimate
Estimate in the decision making phase when a project has not yet been authorized, often as part of the business case analysis.
Project Charter and Plan Estimate
Estimate that goes along with a project definition or project statement and estimates scope, objectives and participants in a project as well at the primary roles and responsibilities.
Detailed Plan & Function Spec Estimate
Project planning level where the project team prepares th estimate of how it will move forward. In Agile projects this may be the overall project estimate rather than individual sprint estimates.
Construction / Deployment Estimate
An estimate to complete of a project that is underway to help keep the project on schedule and cost.
Post Deployment Estimate
This estimate provides information for estimators and others to used such as lessons learned, actual data for calibration, etc.
Thank you for reading “Dan on Estimating”, if you would like more information about Galorath’s estimation models, please visit our contact page, call us at +1 310 414-3222 or click a button below to ask sales questions, sign up for our free library or schedule a demo.
Cloud Matters Canadian Cloud Council…. Costs & Benefits of Cloud
I had the privilege of presenting at the Canadian Cloud Council conference in Banff Canada. It was exciting to see and hear all the wonderful things people are doing or planning to do with cloud computing.
My talk touched on the analysis of the business case for cloud computing and pointed out that sometimes cost savings are attributed to the cloud when they are really do to factors such as development environment. The point was that each cloud development needs to estimate costs and business value and make the right decision for the business.
I have several more comprehensive briefings on cloud costing available if requested. And have included the summary of Cloud costs, benefits and ROI.
My key points were:
- Use an estimation process to identify costs, schedule, risk and benefits
- Make decisions based on value to business
- Attribute costs and cost savings to their root causes rather than just lumping them all to “cloud”
It is exciting to see the evolution of cloud computing.. And sometimes disturbing when organizations cast their distributed applications as cloud applications just to get on the cloud bandwagon.
Banff…. Doesn’t get much more beautiful than this..
Thank you for reading “Dan on Estimating”, if you would like more information about Galorath’s estimation models, please visit our contact page, call us at +1 310 414-3222 or click a button below to ask sales questions, sign up for our free library or schedule a demo.
Evidence of Cloud Development Saving Development Time
An Evans Data Cloud Development Survey found that developers believe they are getting an 11.6% reduction in development time by using public and private cloud development platforms such as Amazon WebServices, Windows Azure, HP Cloud Services, VMWare, Salesforce force.com, and Netsuite’s cloud services.
“While about 10 percent of developers cited no time savings in using cloud environments, an almost equal amount said they had experienced more than 30 percent time savings. About 38 percent cited savings in the 11 to 20 percent range.”
Obviously if we just take the same tools, practices and environment and put it in the cloud nothing magic happens… Perhaps response time is a bit slower. Looking at the impact of response time we see a potential 6% increase in schedule due to immediate response to a developer and an 18% increase in effort if every action has a terrible (3+ second) response:
So why should the cloud reduce development time?
- Better environment (e.g. force.com)
- Better development Tools (e.g force.com)
- Mobile making more work hours in a day (development during off-hours)
- Hawthorne Effect (People respond to being measured…or in this case estimate reductions because they hope there are some)
A Galorath study of salesforce.com’s force.com showed an over 40% decrease in development time. But this was attributed to the availability of reusable components and the powerful development language, not the fact that it was cloud based.
Thank you for reading “Dan on Estimating”, if you would like more information about Galorath’s estimation models, please visit our contact page, call us at +1 310 414-3222 or click a button below to ask sales questions, sign up for our free library or schedule a demo.
The Value of Combining Delphi and SEER
I am a huge believer in using Delphi techniques as part of the estimating process and have been the moderator in many programs.
The goal of a Delphi is to come to consensus among various expert stakeholders (generally engineering personnel) as to an estimate. Of course Delphi techniques can be used for other items as well.
Unfortunately, coming to consensus often yields compromises that blur risk and uncertainty. Using SEER and Galorath estimation processes can make Delphi more powerful, more accurate, and more able to quantify risk and uncertainty by estimating key characteristics (SEER inputs) and their range. Then using SEER to prepare the actual cost, schedule, risk, reliability estimate.
To summarize the Delphi process:
- Choose a team representing every engineering group involved
- Establish groundrules, assumptions, WBS and key drivers
- Individual evaluation of WBS items, groundrules and assumptions, and key performance parameters
- Group Estimation session for consensus. Iterative steps to gain consensus on key performance parameters (key SEER inputs) and the estimates. Then resolve differences with benchmarks and corporate history.
- Review results. The project manager reviews the final task list with the estimation team.
- Capture lessons learned
Delphi approaches are valuable for several reasons, the most important being the process step that defines groundrules and assumptions. Also being able to reconcile divergence in experts’ answers as real program risks are important.
Customers with critical programs can use the SEER inputs as the focus of the various Delphi approaches. And due to the ability to input a range, not just a single point, this approach allows the ability to capture and quantify uncertainty, factoring it into the estimate.
When the Delphi captures groundrules and assumptions and WBS together those may be created inside SEER. This provides both the basis for the next phase as well as ensures capturing those important items for future estimates and analysis. Usually in this case SEER is projected on a screen so all can view the work in progress. Then rather than just capture effort and schedule (which have been found to be more highly variant) for the WBS item(s) participants capture their beliefs of the key input parameters. This allows information to be saved and benchmarked as well as used for estimation. Thus the estimation process continues to improve.
Once the Delphi of the estimating inputs is completed experts can review both the likely effort and schedule as well as the risk and determine the criticality of the project and thus the probability required for the final estimate.
What-ifs can be performed to evaluate project alternatives or contingencies quickly and reliably.
Additionally, SEER’s Estimate By Comparison allows the ability to express size or any other SEER parameter in terms of both its relative weighting and to express uncertainty among experts. Estimate by Comparison is also used as a method of achieving agreement (of the range or the single point) by having the experts in the room together and eliciting agreement while viewing the choices together once the first passes are completed individually.
Finally, being able to see the estimate results, benchmark against industry and the organization itself complete the picture. The best thinking of the experts and comparison with reality.
Thank you for reading “Dan on Estimating”, if you would like more information about Galorath’s estimation models, please visit our contact page, call us at +1 310 414-3222 or click a button below to ask sales questions, sign up for our free library or schedule a demo.
Better Buying Power 2.0 Initiative: Better value to the warfighter and taxpayer
The Undersecretary of Defense for Acquisition, Technology, and Logistics AT&L (Frank Kendall) released the “Better Buying Power 2.0 memorandum and guidance”
The goal is “deliver better value to the taxpayer and Warfighter by improving the way the Department does business.”
1. Achieve Affordable Programs
- Mandate affordability as a requirement
- Institute a system of investment planning to derive affordability caps
- Enforce affordability caps
Read more
Thank you for reading “Dan on Estimating”, if you would like more information about Galorath’s estimation models, please visit our contact page, call us at +1 310 414-3222 or click a button below to ask sales questions, sign up for our free library or schedule a demo.
Estimating Cost From Model Based Systems Engineering (MBSE) Models
We at Galorath have been working on ways to get cost, schedule, risk, and reliability from systems engineering models for several years. INCOSE defines Model Based Systems Engineering as:
“formalized application of modeling to support system requirements, design, analysis, verification, and validation, beginning in the conceptual design phase and continuing throughout development and later life cycle phases”
- First, for software, Our work with Rational RSx was successful… That it: it extracts information from use case models, then uses that to run SEER for Software. This functionality is available as a product today.
- Our uses system models from IBM Rhapsody to generate costs for hardware intensive systems as well as other kinds of systems
- Our newest research, in concert with University of Arizona and University of South Australia, goes further in using system models to generate cost models.
The attached briefing Model Based Systems Engineering (MBSE) Cost Modeling is a simple introductions to the research and some of the successes. We will be exploring this more deeply and documenting some of our findings here.
Thank you for reading “Dan on Estimating”, if you would like more information about Galorath’s estimation models, please visit our contact page, call us at +1 310 414-3222 or click a button below to ask sales questions, sign up for our free library or schedule a demo.
Affordability Analysis: The Role of Process, Cost & ROI
This is the short version of my new brief on Systems and Software affordability and affordability process. This version is oriented towards Department of Defense software. I have another, more comprehensive version that covers commercial software. One key point is the use of parametrics in the affordability process to make lots of trades quickly… then after analysis of alternatives, drilling down on the chosen one or few alternatives.
Here is the abstract:
Affordability Analysis: The Role Process, Cost and ROI Modeling In Improved Program Performance
Affordability analysis as part of decision making may be the biggest edge of the decade for both commercial organizations and DoD / government organizations.
In an IT context companies struggling to increase profits often view IT as a necessary evil: one that consumes resources rather contributes to the bottom line. However, IT can be a significant contributor when IT decisions are made after modeling affordability in terms of the cost and return.
In a DoD context affordability as “should cost” and “will cost” are the bywords of the times: attempting to replace past cost / performance failure due to inflexible user requirements or over-specified contractor requirements is being replaced by realistic trades of cost, schedule, performance and other key performance parameters.
As people go forward in affordability analysis it is important to recognize that tools are important and that repeatable process is essential to success.
A complete affordability analysis determines the risk adjusted Total Cost of Ownership and return on IT investment along with its consistency with long-range investment and business strategy of an organization measured against risk and key technical and performance parameters.
Both existing systems and new developments will be addressed. Additionally risk and tradeoffs between functionality, quality, security and other system goals will be covered.
Process steps include: Read more
Thank you for reading “Dan on Estimating”, if you would like more information about Galorath’s estimation models, please visit our contact page, call us at +1 310 414-3222 or click a button below to ask sales questions, sign up for our free library or schedule a demo.
Project Prediction Infographic
