Standard Estimating Gates
A customer recently asked us for a standard set of estimating points. And among us we didn’t have an answer off the top of our heads… We looked at PMI, ITIL, COBIT, etc. and found no standard. We deal with estimating gates all the time… So how come we didn’t have any standards. It struck me that the estimating points match many of the decision gates, the development gates. So one of our PMI certified Project Managers called the PMI offices. They said they don’t mandate estimation points or gates since they are so diverse among organizations. And they suggested where there is a key deliverable, one the project cant live without, that is generally a gate.
PMI’s PMBOK states “cost estimates should be reviewed and refined during the course of the project to reflect additional detail as it becomes available and assumptions are tested. The accuracy of a project estimate will increase as the project progresses through the project life cycle. For example a project in the initiation phase may have a rough order of magnitude (ROM) estimate in the range of -25% to +75%. Later in the project, as more information is known, definitive estimates could narrow the range of accuracy to -5% to +10%. In some organizations, there are guidelines for when such refinements can be made and the degree fo confidence or accuracy that is expected” In other words PMI remains silent on standard estimation points.
So.. Here is a proposed set of standard estimation points. These gates are often tied to how an organization funds money for projects. So they need to be tailored to the governance approaches being used:
Concept Level Estimate
Just a finger in the wind to see if the idea is worth exploring or is so expensive as to be impractical.
Business Case Estimate
Estimate in the decision making phase when a project has not yet been authorized, often as part of the business case analysis.
Project Charter and Plan Estimate
Estimate that goes along with a project definition or project statement and estimates scope, objectives and participants in a project as well at the primary roles and responsibilities.
Detailed Plan & Function Spec Estimate
Project planning level where the project team prepares th estimate of how it will move forward. In Agile projects this may be the overall project estimate rather than individual sprint estimates.
Construction / Deployment Estimate
An estimate to complete of a project that is underway to help keep the project on schedule and cost.
Post Deployment Estimate
This estimate provides information for estimators and others to used such as lessons learned, actual data for calibration, etc.
Thank you for reading “Dan on Estimating”, if you would like more information about Galorath’s estimation models, please visit our contact page, call us at +1 310 414-3222 or click a button below to ask sales questions, sign up for our free library or schedule a demo.
Making The Best Decisions in IT and Cloud: Affordability Processes Applied to Cloud Computing
I have been spending a lot of time with cloud computing as well as with affordability processes. So it seemed a good match to do a presentation covering application of affordability processes to cloud computing. The bottom line is cloud computing can save a lot of money when used at in the right circumstances…. And cloud computing can be ineffective in terms of affordability in some situations.
This includes the 10 step affordability process as well as amplification of several process steps.
I go back and forth on combining steps 1 and 2… 1. Identifying Key Performance Parameters and 2. identifying figures of merit for affordability trade-offs but continue to keep the separate just to highlight the importance of knowing what the definition of success or failure are in step 1. In a few instances I used our SEER for IT model to illustrated trades between on-premises and cloud solutions. The is a quick and powerful way to determine affordability.
When I attended the recent cloud computing conference in Banff someone pointed out it is not cloud computing, but just computing… that we need not make the distinction so sharply but basically choose the computing approach that best fits the situation… I agree. Cloud is here to stay. It is not just a fad. And it can be extremely useful when appropriately applied.
One thing I didn’t point out in the briefing is some specific Galorath experience. For example, we hosted our SEER server on the Amazon cloud. Generally users host on their own internal servers for security reasons but this seemed like a good alternative. The process went well and we were happy… until we started performance benchmarking. We found the Amazon cloud IN THIS CASE performed an order of magnitude slower than VPN across half the world to corporate servers. Again this is just a single case… We could have tuned the Amazon Cloud or purchased more capacity. We will report performance improvements when we do. There are a myriad of options on the Amazon cloud. I hope we don’t just have to pick them by trail and error. We had an enterprise database running on the cloud, along with the application. Still, the good news.. We just need to move onto another cloud or beef up our Amazon cloud capability. Participants at this conference identified several that are more designed for rapid transaction processing. This is a whole lot better than the old days: buy a server(s); find out they aren’t fast enough; iterate. Long live the cloud ad affordability analysis.
PS the above graphic is a summary of a trade done between an on-premises and a cloud solution. While this shows a substantial cost reduction from the cloud THIS DOES NOT IMPLY ALL CLOUD SOLUTIONS ARE CHEAPER.
Thank you for reading “Dan on Estimating”, if you would like more information about Galorath’s estimation models, please visit our contact page, call us at +1 310 414-3222 or click a button below to ask sales questions, sign up for our free library or schedule a demo.
Cloud Matters Canadian Cloud Council…. Costs & Benefits of Cloud
I had the privilege of presenting at the Canadian Cloud Council conference in Banff Canada. It was exciting to see and hear all the wonderful things people are doing or planning to do with cloud computing.
My talk touched on the analysis of the business case for cloud computing and pointed out that sometimes cost savings are attributed to the cloud when they are really do to factors such as development environment. The point was that each cloud development needs to estimate costs and business value and make the right decision for the business.
I have several more comprehensive briefings on cloud costing available if requested. And have included the summary of Cloud costs, benefits and ROI.
My key points were:
- Use an estimation process to identify costs, schedule, risk and benefits
- Make decisions based on value to business
- Attribute costs and cost savings to their root causes rather than just lumping them all to “cloud”
It is exciting to see the evolution of cloud computing.. And sometimes disturbing when organizations cast their distributed applications as cloud applications just to get on the cloud bandwagon.
Banff…. Doesn’t get much more beautiful than this..
Thank you for reading “Dan on Estimating”, if you would like more information about Galorath’s estimation models, please visit our contact page, call us at +1 310 414-3222 or click a button below to ask sales questions, sign up for our free library or schedule a demo.
Evidence of Cloud Development Saving Development Time
An Evans Data Cloud Development Survey found that developers believe they are getting an 11.6% reduction in development time by using public and private cloud development platforms such as Amazon WebServices, Windows Azure, HP Cloud Services, VMWare, Salesforce force.com, and Netsuite’s cloud services.
“While about 10 percent of developers cited no time savings in using cloud environments, an almost equal amount said they had experienced more than 30 percent time savings. About 38 percent cited savings in the 11 to 20 percent range.”
Obviously if we just take the same tools, practices and environment and put it in the cloud nothing magic happens… Perhaps response time is a bit slower. Looking at the impact of response time we see a potential 6% increase in schedule due to immediate response to a developer and an 18% increase in effort if every action has a terrible (3+ second) response:
So why should the cloud reduce development time?
- Better environment (e.g. force.com)
- Better development Tools (e.g force.com)
- Mobile making more work hours in a day (development during off-hours)
- Hawthorne Effect (People respond to being measured…or in this case estimate reductions because they hope there are some)
A Galorath study of salesforce.com’s force.com showed an over 40% decrease in development time. But this was attributed to the availability of reusable components and the powerful development language, not the fact that it was cloud based.
Thank you for reading “Dan on Estimating”, if you would like more information about Galorath’s estimation models, please visit our contact page, call us at +1 310 414-3222 or click a button below to ask sales questions, sign up for our free library or schedule a demo.
The Value of Combining Delphi and SEER
I am a huge believer in using Delphi techniques as part of the estimating process and have been the moderator in many programs.
The goal of a Delphi is to come to consensus among various expert stakeholders (generally engineering personnel) as to an estimate. Of course Delphi techniques can be used for other items as well.
Unfortunately, coming to consensus often yields compromises that blur risk and uncertainty. Using SEER and Galorath estimation processes can make Delphi more powerful, more accurate, and more able to quantify risk and uncertainty by estimating key characteristics (SEER inputs) and their range. Then using SEER to prepare the actual cost, schedule, risk, reliability estimate.
To summarize the Delphi process:
- Choose a team representing every engineering group involved
- Establish groundrules, assumptions, WBS and key drivers
- Individual evaluation of WBS items, groundrules and assumptions, and key performance parameters
- Group Estimation session for consensus. Iterative steps to gain consensus on key performance parameters (key SEER inputs) and the estimates. Then resolve differences with benchmarks and corporate history.
- Review results. The project manager reviews the final task list with the estimation team.
- Capture lessons learned
Delphi approaches are valuable for several reasons, the most important being the process step that defines groundrules and assumptions. Also being able to reconcile divergence in experts’ answers as real program risks are important.
Customers with critical programs can use the SEER inputs as the focus of the various Delphi approaches. And due to the ability to input a range, not just a single point, this approach allows the ability to capture and quantify uncertainty, factoring it into the estimate.
When the Delphi captures groundrules and assumptions and WBS together those may be created inside SEER. This provides both the basis for the next phase as well as ensures capturing those important items for future estimates and analysis. Usually in this case SEER is projected on a screen so all can view the work in progress. Then rather than just capture effort and schedule (which have been found to be more highly variant) for the WBS item(s) participants capture their beliefs of the key input parameters. This allows information to be saved and benchmarked as well as used for estimation. Thus the estimation process continues to improve.
Once the Delphi of the estimating inputs is completed experts can review both the likely effort and schedule as well as the risk and determine the criticality of the project and thus the probability required for the final estimate.
What-ifs can be performed to evaluate project alternatives or contingencies quickly and reliably.
Additionally, SEER’s Estimate By Comparison allows the ability to express size or any other SEER parameter in terms of both its relative weighting and to express uncertainty among experts. Estimate by Comparison is also used as a method of achieving agreement (of the range or the single point) by having the experts in the room together and eliciting agreement while viewing the choices together once the first passes are completed individually.
Finally, being able to see the estimate results, benchmark against industry and the organization itself complete the picture. The best thinking of the experts and comparison with reality.
Thank you for reading “Dan on Estimating”, if you would like more information about Galorath’s estimation models, please visit our contact page, call us at +1 310 414-3222 or click a button below to ask sales questions, sign up for our free library or schedule a demo.
Affordability Analysis: The Role of Process, Cost & ROI
This is the short version of my new brief on Systems and Software affordability and affordability process. This version is oriented towards Department of Defense software. I have another, more comprehensive version that covers commercial software. One key point is the use of parametrics in the affordability process to make lots of trades quickly… then after analysis of alternatives, drilling down on the chosen one or few alternatives.
Here is the abstract:
Affordability Analysis: The Role Process, Cost and ROI Modeling In Improved Program Performance
Affordability analysis as part of decision making may be the biggest edge of the decade for both commercial organizations and DoD / government organizations.
In an IT context companies struggling to increase profits often view IT as a necessary evil: one that consumes resources rather contributes to the bottom line. However, IT can be a significant contributor when IT decisions are made after modeling affordability in terms of the cost and return.
In a DoD context affordability as “should cost” and “will cost” are the bywords of the times: attempting to replace past cost / performance failure due to inflexible user requirements or over-specified contractor requirements is being replaced by realistic trades of cost, schedule, performance and other key performance parameters.
As people go forward in affordability analysis it is important to recognize that tools are important and that repeatable process is essential to success.
A complete affordability analysis determines the risk adjusted Total Cost of Ownership and return on IT investment along with its consistency with long-range investment and business strategy of an organization measured against risk and key technical and performance parameters.
Both existing systems and new developments will be addressed. Additionally risk and tradeoffs between functionality, quality, security and other system goals will be covered.
Process steps include: Read more
Thank you for reading “Dan on Estimating”, if you would like more information about Galorath’s estimation models, please visit our contact page, call us at +1 310 414-3222 or click a button below to ask sales questions, sign up for our free library or schedule a demo.
The Risks of Risk Management
Computer Aid featured a thought provoking article on the risks of risk management. The article noted, for example, that people that are wearing seat belts take more risks (drive more risky) because they feel safer in their seat belt. And while they are actually safer the pedestrian, cyclist, and other drivers may be in more danger. And
The article referred to the work of Sam . Pelzman sowed that showed:
” What Peltzman identified is that behavior can change substantially in response to risk management policies. For example in the case of seat belts, people tend to drive faster and have more collisions when they use seat belts because of the sense of security it gives them. This is not to say that seat belts are ineffective, it seems that traffic safety overall (especially in terms of road deaths) has improved as a result of them, but not as much as you would have expected, because of this behavioral offset.”
Now how can this impact IT projects you might ask…. Well, if we have risk management policies in place organizations might
Then once people have identified risks and have put a risk management plan in place they may be more inclined to take more risk. Yikes.
“So the question for the project manager is are your risk management policies changing behavior?… Are people less cautious knowing there is a more robust monitoring process in place for their projects? Are team members less focused on escalating problems knowing that someone else is watching out for them?”
Risk of Unknown Unknowns
Even more important is that risk management doesn’t usually capture unknown unknowns.. That is risks we haven’t thought of. I have heard many people identify those risks that wernt thought of as being the things that caused project failures more than those that were thought of.
10 Step Process and Risk
The 10 step process in my book includes step 6 quantify risks and risk-analysis on dealing with risks and is worth reviewing.
Risk Process Failure
And as Douglas Hubbard points out: “The ultimate common mode failure would be a failure of risk management itself. A weak risk management approach is effectively the biggest risk in the organization” Hubbard, Douglas W. (2009-04-06). The Failure of Risk Management: Why It’s Broken and How to Fix It (Kindle Locations 236-237). John Wiley and Sons.
The point is if the initial risk analysis is not meaningful then the risk management methods are likely wrong and addressing the wrong problems.
“If risk assessment is a failure, then the best case is that the risk management effort is simply a waste of time and money because decisions are ultimately unimproved. In the worst case, the erroneous conclusions lead the organization down a more dangerous path that it would probably not have otherwise taken.”
Thank you for reading “Dan on Estimating”, if you would like more information about Galorath’s estimation models, please visit our contact page, call us at +1 310 414-3222 or click a button below to ask sales questions, sign up for our free library or schedule a demo.
IT Capability Maturity Framework (IT-CMF)
I am excited that IT business value is being addressed by The Innovation Value Institute (IVI) .
The Innovation Value Institute was co-founded in 2006 by the National University of Ireland Maynooth (NUIM) and Intel to help drive the transformation of IT management. Our goal is to create a global gold standard for IT management.
To achieve this goal, the Innovation Management Institute researches, develops, and disseminates empirically proven and industry validated IT best practice through a unique open collaboration between leading academic and industry practitioners. Through its consortium, IVI facilitates a collaborative community of like-minded peers committed to investigating, advancing and disseminating the frameworks, tools and best practices associated with managing IT Value and IT enabled Innovation. IVI is currently focused on extending the development and dissemination of the IT Capability Maturity Framework (IT-CMF).
Since its inception in 2006, the IT-CMF has been used by a variety of global organizations to solve a range of capability issues.
- IT capability measurement and improvement
- IT organizational design and capability management
- IT business alignment and leadership
- Organization benchmarking and best practice
- IT risk management
- Cloud computing (including services innovation and outsourcing
They have published an IT Capability Maturity Model that is very exciting in clarifying the whole issue of IT as business value. A graphic, copied from their web site follows:
The IT Capability Maturity Framework (IT-CMF) provides a capability maturity framework for Information technology including:
- Maps IT organizations onto a capability maturity model (33 capabilities within IT management)
- Provides practices, outcomes and metrics to improve capability maturity and consistency of results
- Enables assessment and benchmarking over time
- Encourages roadmaps with measures to improve maturity
- Offers best practice guidelines
- Comprises four macro-capabilities to emphasize their complexity and their importance in managing IT for business value.
Thank you for reading “Dan on Estimating”, if you would like more information about Galorath’s estimation models, please visit our contact page, call us at +1 310 414-3222 or click a button below to ask sales questions, sign up for our free library or schedule a demo.
Using Trends In Variance To Understand Trends in Risk
I always prefer leading indicators to lagging ones. When our CFO comes in at the end of April, for instance, and tells me how we performed through March I am always interested to know but…
I often say, “What do I do with this information–it is now ancient history.”
The paper “The Canary in the Mineshaft: Key Indicators for success in aerospace and defense programs” discusses trending based on leading indicators:
- Trend in estimated schedule to complete (ETC)… ranges in estimates of time to complete, then Monte Carlo analysis based on the uncertainty
- Trend in net present value (NPV)… Value provided by this program, again based on a range… This factors in the cost to complete, maintenance and ownership costs as well as revenues derived
Their point is if the trend in variance of range estimates of the above items is getting smaller, risk is being reduced and if the trend is getting larger risk is increasing.
They wisely use a definition of risk that equates it to uncertainty rather than discrete risks in this context.
Thank you for reading “Dan on Estimating”, if you would like more information about Galorath’s estimation models, please visit our contact page, call us at +1 310 414-3222 or click a button below to ask sales questions, sign up for our free library or schedule a demo.
Typical Agile Development Lifecycle
I ran across this nice graphic of an agile lifecycle produced by Galorath’s David Dewitt. I thought it was clear in its presentation of the lifecycle and how estimation and planning fit in.
I am often surprised when speaking to people who say, “We don’t need estimation or planning any longer because we are agile.” I suppose that may be true for very small projects. But for projects of any significance, it is not enough to tell stakeholders who need to budget and plan for delivered software that we will tell you we are done when we get there. Estimation at the macro level is required so the overall project scope is understood as well as at the lowest levels for developers to commit to their work in a sprint.
Thank you for reading “Dan on Estimating”, if you would like more information about Galorath’s estimation models, please visit our contact page, call us at +1 310 414-3222 or click a button below to ask sales questions, sign up for our free library or schedule a demo.
Project Prediction Infographic
