Standard Estimating Gates
A customer recently asked us for a standard set of estimating points. And among us we didn’t have an answer off the top of our heads… We looked at PMI, ITIL, COBIT, etc. and found no standard. We deal with estimating gates all the time… So how come we didn’t have any standards. It struck me that the estimating points match many of the decision gates, the development gates. So one of our PMI certified Project Managers called the PMI offices. They said they don’t mandate estimation points or gates since they are so diverse among organizations. And they suggested where there is a key deliverable, one the project cant live without, that is generally a gate.
PMI’s PMBOK states “cost estimates should be reviewed and refined during the course of the project to reflect additional detail as it becomes available and assumptions are tested. The accuracy of a project estimate will increase as the project progresses through the project life cycle. For example a project in the initiation phase may have a rough order of magnitude (ROM) estimate in the range of -25% to +75%. Later in the project, as more information is known, definitive estimates could narrow the range of accuracy to -5% to +10%. In some organizations, there are guidelines for when such refinements can be made and the degree fo confidence or accuracy that is expected” In other words PMI remains silent on standard estimation points.
So.. Here is a proposed set of standard estimation points. These gates are often tied to how an organization funds money for projects. So they need to be tailored to the governance approaches being used:
Concept Level Estimate
Just a finger in the wind to see if the idea is worth exploring or is so expensive as to be impractical.
Business Case Estimate
Estimate in the decision making phase when a project has not yet been authorized, often as part of the business case analysis.
Project Charter and Plan Estimate
Estimate that goes along with a project definition or project statement and estimates scope, objectives and participants in a project as well at the primary roles and responsibilities.
Detailed Plan & Function Spec Estimate
Project planning level where the project team prepares th estimate of how it will move forward. In Agile projects this may be the overall project estimate rather than individual sprint estimates.
Construction / Deployment Estimate
An estimate to complete of a project that is underway to help keep the project on schedule and cost.
Post Deployment Estimate
This estimate provides information for estimators and others to used such as lessons learned, actual data for calibration, etc.
Thank you for reading “Dan on Estimating”, if you would like more information about Galorath’s estimation models, please visit our contact page, call us at +1 310 414-3222 or click a button below to ask sales questions, sign up for our free library or schedule a demo.
Making The Best Decisions in IT and Cloud: Affordability Processes Applied to Cloud Computing
I have been spending a lot of time with cloud computing as well as with affordability processes. So it seemed a good match to do a presentation covering application of affordability processes to cloud computing. The bottom line is cloud computing can save a lot of money when used at in the right circumstances…. And cloud computing can be ineffective in terms of affordability in some situations.
This includes the 10 step affordability process as well as amplification of several process steps.
I go back and forth on combining steps 1 and 2… 1. Identifying Key Performance Parameters and 2. identifying figures of merit for affordability trade-offs but continue to keep the separate just to highlight the importance of knowing what the definition of success or failure are in step 1. In a few instances I used our SEER for IT model to illustrated trades between on-premises and cloud solutions. The is a quick and powerful way to determine affordability.
When I attended the recent cloud computing conference in Banff someone pointed out it is not cloud computing, but just computing… that we need not make the distinction so sharply but basically choose the computing approach that best fits the situation… I agree. Cloud is here to stay. It is not just a fad. And it can be extremely useful when appropriately applied.
One thing I didn’t point out in the briefing is some specific Galorath experience. For example, we hosted our SEER server on the Amazon cloud. Generally users host on their own internal servers for security reasons but this seemed like a good alternative. The process went well and we were happy… until we started performance benchmarking. We found the Amazon cloud IN THIS CASE performed an order of magnitude slower than VPN across half the world to corporate servers. Again this is just a single case… We could have tuned the Amazon Cloud or purchased more capacity. We will report performance improvements when we do. There are a myriad of options on the Amazon cloud. I hope we don’t just have to pick them by trail and error. We had an enterprise database running on the cloud, along with the application. Still, the good news.. We just need to move onto another cloud or beef up our Amazon cloud capability. Participants at this conference identified several that are more designed for rapid transaction processing. This is a whole lot better than the old days: buy a server(s); find out they aren’t fast enough; iterate. Long live the cloud ad affordability analysis.
PS the above graphic is a summary of a trade done between an on-premises and a cloud solution. While this shows a substantial cost reduction from the cloud THIS DOES NOT IMPLY ALL CLOUD SOLUTIONS ARE CHEAPER.
Thank you for reading “Dan on Estimating”, if you would like more information about Galorath’s estimation models, please visit our contact page, call us at +1 310 414-3222 or click a button below to ask sales questions, sign up for our free library or schedule a demo.
Cloud Matters Canadian Cloud Council…. Costs & Benefits of Cloud
I had the privilege of presenting at the Canadian Cloud Council conference in Banff Canada. It was exciting to see and hear all the wonderful things people are doing or planning to do with cloud computing.
My talk touched on the analysis of the business case for cloud computing and pointed out that sometimes cost savings are attributed to the cloud when they are really do to factors such as development environment. The point was that each cloud development needs to estimate costs and business value and make the right decision for the business.
I have several more comprehensive briefings on cloud costing available if requested. And have included the summary of Cloud costs, benefits and ROI.
My key points were:
- Use an estimation process to identify costs, schedule, risk and benefits
- Make decisions based on value to business
- Attribute costs and cost savings to their root causes rather than just lumping them all to “cloud”
It is exciting to see the evolution of cloud computing.. And sometimes disturbing when organizations cast their distributed applications as cloud applications just to get on the cloud bandwagon.
Banff…. Doesn’t get much more beautiful than this..
Thank you for reading “Dan on Estimating”, if you would like more information about Galorath’s estimation models, please visit our contact page, call us at +1 310 414-3222 or click a button below to ask sales questions, sign up for our free library or schedule a demo.
The Value of Combining Delphi and SEER
I am a huge believer in using Delphi techniques as part of the estimating process and have been the moderator in many programs.
The goal of a Delphi is to come to consensus among various expert stakeholders (generally engineering personnel) as to an estimate. Of course Delphi techniques can be used for other items as well.
Unfortunately, coming to consensus often yields compromises that blur risk and uncertainty. Using SEER and Galorath estimation processes can make Delphi more powerful, more accurate, and more able to quantify risk and uncertainty by estimating key characteristics (SEER inputs) and their range. Then using SEER to prepare the actual cost, schedule, risk, reliability estimate.
To summarize the Delphi process:
- Choose a team representing every engineering group involved
- Establish groundrules, assumptions, WBS and key drivers
- Individual evaluation of WBS items, groundrules and assumptions, and key performance parameters
- Group Estimation session for consensus. Iterative steps to gain consensus on key performance parameters (key SEER inputs) and the estimates. Then resolve differences with benchmarks and corporate history.
- Review results. The project manager reviews the final task list with the estimation team.
- Capture lessons learned
Delphi approaches are valuable for several reasons, the most important being the process step that defines groundrules and assumptions. Also being able to reconcile divergence in experts’ answers as real program risks are important.
Customers with critical programs can use the SEER inputs as the focus of the various Delphi approaches. And due to the ability to input a range, not just a single point, this approach allows the ability to capture and quantify uncertainty, factoring it into the estimate.
When the Delphi captures groundrules and assumptions and WBS together those may be created inside SEER. This provides both the basis for the next phase as well as ensures capturing those important items for future estimates and analysis. Usually in this case SEER is projected on a screen so all can view the work in progress. Then rather than just capture effort and schedule (which have been found to be more highly variant) for the WBS item(s) participants capture their beliefs of the key input parameters. This allows information to be saved and benchmarked as well as used for estimation. Thus the estimation process continues to improve.
Once the Delphi of the estimating inputs is completed experts can review both the likely effort and schedule as well as the risk and determine the criticality of the project and thus the probability required for the final estimate.
What-ifs can be performed to evaluate project alternatives or contingencies quickly and reliably.
Additionally, SEER’s Estimate By Comparison allows the ability to express size or any other SEER parameter in terms of both its relative weighting and to express uncertainty among experts. Estimate by Comparison is also used as a method of achieving agreement (of the range or the single point) by having the experts in the room together and eliciting agreement while viewing the choices together once the first passes are completed individually.
Finally, being able to see the estimate results, benchmark against industry and the organization itself complete the picture. The best thinking of the experts and comparison with reality.
Thank you for reading “Dan on Estimating”, if you would like more information about Galorath’s estimation models, please visit our contact page, call us at +1 310 414-3222 or click a button below to ask sales questions, sign up for our free library or schedule a demo.
New SEER Labor Rate Calculator: A Leap Forward in “Should Cost” & “Will Cost”
I saw a demo today of our new labor rate calculator. It takes in various labor rate drivers and computes a viable labor rate. It even evaluates the cost of equipment, electricity, floor space, insurance, etc.
This is a great step forward in the “should cost” and will cost for product manufacturing. Buying organizations can describe the problem and see what a fair labor rate for the region, country, machine, etc. Mixed currencies are supported as well.
The following is a small example of the kinds of information that can be specified In this case it is configured for manufacturing.
Thank you for reading “Dan on Estimating”, if you would like more information about Galorath’s estimation models, please visit our contact page, call us at +1 310 414-3222 or click a button below to ask sales questions, sign up for our free library or schedule a demo.
Why We Estimate Schedule & Cost
I stumbled across a working draft of an excellent document on my hard drive, produced, I believe, by a gentleman from Texas Instruments some years ago. I thought this list of why we estimate cost and schedule was excellent and still relevant.
Why We Estimate Schedule & Cost
- To scope proposed tasks
- To explore alternative system concepts
design to cost/budget
- To explore alternative design concepts
- To explore alternative proposals for enhancements and upgrades
- To identify key design elements
- To identify key process parameters
prioritize needs vs wants
- To identify key assumptions
- To identify and quantify uncertainties
- To identify tasks and their relationships
- To assess schedule feasibility
- To identify, allocate and schedule resources
- To assess an organization’s ability to perform within targeted costs
- To evaluate the consequences of internal and external constraints
- To establish achievable objectives
- To establish a basis for quality service
- To establish commitments
- To bound the risk against customer needs
- To balance levels of risk against customer needs
- To provide a basis of successful risk management
build vs buy analysis
- To prepare successful proposals
- To evaluate proposals from competing bidders
- To establish baselines for project tracking
enhance/reuse vs redesign analysis
- To predict life-cycle costs
- To predict returns on investments
- To provide information for establishing business and investment strategies
Thank you for reading “Dan on Estimating”, if you would like more information about Galorath’s estimation models, please visit our contact page, call us at +1 310 414-3222 or click a button below to ask sales questions, sign up for our free library or schedule a demo.
296 Billion Dollars in DOD Cost Overruns: 2009 GAO Weapons Systems Assessments
The full document GAO assessment of Weapons Systems is an update to the 2007 assessment.
It states “ Our review this year indicates that while the overall performance of weapon system programs is still poor; there have been some modest improvements in DOD’s acquisition outcomes: total cost growth on this year’s portfolio of 96 major defense acquisition programs has decreased marginally compared to the 2007 portfolio.”
Programs started in recent years have more knowledge about technology and design at key points in the acquisition process.
- Cumulative cost overruns are almost $296 billion in 2009 dollars
- 64 of 96 active defense programs reported increases in their projected cost since their initial cost estimate.
Sources of the problem:
While there are different ways to measure the extent and nature of cost growth, there is agreement between DOD and us on the sources of the problem:
(1) POOR REQUIREMENTS AND KNOWLEDGE: programs are started with poor foundations and inadequate knowledge for developing realistic cost estimates
(2) ARTIFICALLY LOW COST & SCHEDULE ESTIMATES WITH LOW TECHNOLOGY READINESS LEVELS: programs move forward with artificially low cost estimates, optimistic schedules and assumptions, immature technologies and designs, and fluid requirements
(3) REQUIREMENTS VOLITILITY: changing or excessive requirements cause cost growth
(4) REQUIREMENTS GROWTH and INADEQUATE VALUE CONSIDERATION OF CHANGES: an imbalance between wants and needs contributes to budget and program instability.
Thank you for reading “Dan on Estimating”, if you would like more information about Galorath’s estimation models, please visit our contact page, call us at +1 310 414-3222 or click a button below to ask sales questions, sign up for our free library or schedule a demo.
CostIQ Case-Based Reasoning Model For SEER Customer Order
This week we received our first order for the official release of CostIQ, the case-based reasoning estimation engine for SEER. It has been rewarding to see so many people using the pre-releases and lined up to procure CostIQ.
CostIQ is a Case-Based Reasoning system that can transform high level requirements and specifications into a cost modeling workup within a sophisticated cost estimating model (SEER).
CostIQ will generate robust estimates of effort, cost, schedule, risk and reliability for software, hardware, IT and manufacturing issues using the key functional and / or performance criteria as its inputs.
Case-based reasoning has been formalized as a four-step process:
- RETRIEVE the most similar case or cases
- REUSE the information and knowledge in that case to solve the problem
- REVISE the proposed solution
- RETAIN the parts of this experience likely to be useful for future problem solving
Thank you for reading “Dan on Estimating”, if you would like more information about Galorath’s estimation models, please visit our contact page, call us at +1 310 414-3222 or click a button below to ask sales questions, sign up for our free library or schedule a demo.
Estimating Best Practices
The following are the bullet points from Dan’s paper on estimating best practices. Using these best practices can increase project success dramatically.
- Decide Why You Want An Estimate
- Map Estimation Goals To Estimate Process Maturity & Develop Plan To Achieve The Maturity
- Have A Documented, Repeatable Estimation Process
- Evaluate Total Ownership Cost; Not Just Development
- Estimate A Range And Pick A Point For The Plan
- Re-estimate The Program When It Changes
- Avoid Death Marches: Programs With Unachievable Schedules Are Likely To Fail And Drain Morale
Thank you for reading “Dan on Estimating”, if you would like more information about Galorath’s estimation models, please visit our contact page, call us at +1 310 414-3222 or click a button below to ask sales questions, sign up for our free library or schedule a demo.
Early Estimating Is Like Martial Arts
I have been studying martial arts for the past year and a half. I am not great but I am committed.
I was taking a private lesson recently. I have been struggling during drills. While I execute correctly much of the time, each time I execute a move I start analyzing it, even while I am executing the following move.
This proves to be a counter-productive approach. I will often make mistakes in the following move because I am spending all my brain power thinking about what I just did right or wrong.
Estimating can have the same issues. Data from past programs is useful. But we must look forward as well. Differences in technology, requirements, program volatility, developers, and much more make just looking at the past a dangerous proposition.
I recall a magazine that once wrote of SEER, “The journey is its own reward” — meaning that the act of thinking about the program being estimated and understanding the range of issues and opportunities makes the program better and the estimates more viable.
Each estimate should look forward to what the program’s issues will be in addition to looking back to prior programs. Yes, data is important, as indicated in the 10 step process. And so is obtaining some understanding of the range of possibilities of the new system.
The following screenshot from SEER Metrics & Benchmarking illustrates the point. After some analysis, a new system is much more costly than nearly all the systems with similar characteristics. Had we just used data and not done the analysis we would have severely underestimated the new system. The journey is its own reward.
Thank you for reading “Dan on Estimating”, if you would like more information about Galorath’s estimation models, please visit our contact page, call us at +1 310 414-3222 or click a button below to ask sales questions, sign up for our free library or schedule a demo.
Project Prediction Infographic
