Estimating Project Risk Using Scenarios (Without Monte Carlo)
I was at a costing syposium today where Paul Garvey of Mitre gave a talk on doing risk analysis without using Monte Carlo or other statistical techniques. Paul, one of the gurus of statistical risk and the author on the definitive book on cost risk: “Probability Methods for Cost Uncertainty Analysis”, said after 25 years of dealing with Monte Carlo and all the mistakes the non-gurus make in setting up Monte Carlo analysis he had determined it is prudent to use a non statistical technique. Read more
The Risk of Statistical Risk
I was at the SCEA (Society of Cost Estimation and Analysis) conference this week. Some of the buzz was about risk, both talks given at the conference and the ongoing risk arguments. For several years the risk gurus have been lining up to show how to do more robust risk analysis. While I would not say they are getting carried away I would say i get concerned with the differences of opinion and the numerous options provided by smart people.
One of my heroes in risk, Dr. Steve Book or MCR points out that risk analysis should include correlation.
One of my other risk heroes, Evin Stump (of Galorath), points out that defining correlation properly for a work breakdown of any size can involve of thousands of correlation entries. For example a 500 element WBS has 124, 500 correlations and a 1000 element WBS has 499,000 correlations. Dr. Book doesn’t point that out but he does say “use .2″ That solves the hundreds of thousands of correlations issue. But according to Evin that doesn’t provide more accurate risk analysis. Evin points out “if two or more risky items are not statistically independent, a Monte Carlo simulation that fails to account for their correlation will underestimate their combined risk” He then asks “what if you overestimate correlation?” Hmmm could it be that .2 correlations could overestimate some systems. Evin also points out how difficult it is to actually determine correlation…. For example, what is the correlation between a light bulb and a light bulb on/off switch… Probably near zero but most people would be tempted to assign a high correlation. Read more
Estimation & Agile Development… Plan and Control Agile Projects
While at the IBM Rational user conference in Orlando I (Dan Galorath) attended a briefing by Scott Ambler, practice leader of Agile development for Rational. He had an interesting (and refreshing) perspective on Agile and Agile planning.
I have always cringed when I heard Agile people saying essentially.. “forget planning.. we are Agile… we will solve tomorrow’s problems tomorrow” That may be fine for tiny projects, but how do major projects get justified and funded without some kind of estimate and plan? And how do managers control them?
Scott spoke of satisfying governance requirements by generating estimates and plans within the agile environment. He also recognized that the business people need to understand business costs and business value so cost versus value tradeoffs can be properly made. He also pointed out that systems architectures and other constraints should be agreed upon up front. He said that architecture should be determined in the first couple weeks, not left for months.
Scott showed some interesting survey results showing that agile projects that are colocated are far more successful than those that are distributed.
This was timely since so many people are talking about Agile these days. Additionally Galorath Analyst David DeWitt has produced a paper on detailed Agile estimating with SEER and applied it with a customer. Send an email to info@galorath.com to receive a copy of David’s paper. And feel free to comment here about your Agile successes and failures and how project planning and control impacted them. And how SEER for Software has been involved.
And to those who have decided Agile means don’t plan or do any of the things that have helped to turn software from an art to an engineering discipline… Please plan and control your Agile projects. They will be far more successful if you do. Yes, the planning is somewhat different than Waterfall projects. But Agile doesn’t mean forget planning.
And to the question.. do I need SEER if I do Agile development…. a resounding yes.
Reducing Software Maintenance and Total Ownership Costs
Over 75% of the costs of software are in software maintenance. To address this I (Dan Galorath) participated in a best practices conference held by the IT Metrics & productivity Institute. I was fascinated by Herb Krasner’s presentation on software evolution versus software maintenance.
Herb defined software evolution as the set of activities, both technical and managerial, that ensures that software continues to meet organizational and business objectives in a cost effective way. He further defines evolutionary system types as either having an Imprecise statement of a real-world problem which generally changes over time or a system that becomes part of the world that it models/implement whose acceptance depends entirely on opinion and judgment. Evolutionary is in contrast with a maintenance system where the problem can be completely stated and where a change to the specification defines a new problem and a new system.
Herb Krasner , David Garmus, and Dan Galorath talks all illustrated how the use of metrics and management associated with measurement through both development and maintenance can reduce total ownership costs dramatically.
We then heard from Bob Lawhorn who spoke on how his company, Computer Aid Inc (CAI) actually saves organizations money and increases quality by taking over legacy system maintenance and support. It was one of the most inspiring illustrations on why we should do all this I ever saw. He spoke of taking over these systems with new hires, applying the CAI processes and measurement and reducing customer costs by around 30%. And they have been doing this for over 20 years.
Overall I came away inspired, seeing how software estimation, measurement, process, and real management of software like manufacturing is making a difference.
I will be presenting my talk on reducing total ownership costs several more times this year in partnership with CAI.
