Heuristics for Systems Engineering Cost Estimation
Dr. Ricardo Valerdi of MIT sent me a pre-publication copy of his upcoming IEEE article. Here is the abstract:
“Engineering cannot wait until all phenomena are explained. Engineers may work effectively, often for centuries, with heuristics. This paper provides thirty one heuristics that have been inspired by the development and application of a systems engineering cost estimation model. The objective of this paper is to present such heuristics in a simple manner so that they can benefit systems engineering researchers and practitioners that develop, calibrate, and use cost models.”
I enjoyed the article (as I do with pretty much everything Ricardo produces). Such simple truths. A few heuristics quoted from the paper follow:
“Don’t assume the original statement of the problem is necessarily the best, or even the right one.”
“Let the available data drive the application boundaries of the model.”
“Design the rating scale according to the phenomenon being modeled.”
The full article is available, as provided by Ricardo here: “Heuristics for Systems Engineering Cost Estimation.”
Thank you for reading “Dan on Estimating”, if you would like more information about Galorath’s estimation models, please visit our contact page or call us at +1 310 414-3222.
Related Posts Computer Generated
Considerations for CMMI Level 4 for Systems/Software product development and deployment
From Galorath’s Dr. Denton Tarbet:
CMMI is a model that can be applied to effect a change in the way an organization (or company) performs on specific projects. The Levels of CMMI:
- Process unpredictable, poorly controlled and reactive
- Process characterized for projects and is often reactive
- Process characterized for the organization.
- Process measured and statistically controlled
- Emphasis on continuous improvement in the process with a goal of more fully meeting the cost/performance objectives of the organization.
Key activities for CMMI Level 4 include:
- Set the organizational Process Performance goals and establish a Quantitative Project Management methods to ensure processes are measured and statistically controlled.
- Select processes that support the organizational goals. If as in most companies goals include developing products on time and on budget in order to increase customer satisfaction and improve profits
- You need a valid estimating process that provides a confidence based project plan
- You need to measure cost and schedule at team task level
- Predict future project performance (actual estimate based on past performance not a belief that the project performance will improve with no external action)
- Take “optimal” corrective actions early
- Develop organizational baselines that can be used to monitor project predictions based on past performance and enforce corrective action when baselines are breached, ie establish the upper and lower control limits that force project corrective action.
- Use a predictive model such as a Performance Based Earned Value Model to predict the project future performance
- Take corrective action to mitigate risks, improve performance, prevent issues, and optimize project performance.
The question regarding level 4 and the application of SEER models for Process Performance Modeling has been addressed in the SEER for Software suite with the addition of Project Monitoring & Control (PMC) option. PMC is a primary tool in support of step 2, 3, and 4 above. In a paper presented at the June 2009 ISPA conference, “Optimized Project Management of Systems and Software Projects” we discussed the concepts of developing an estimate using a parametric model such as SEER SEM so that the project has a confidence based project plan that can be executed. We demonstrated the use of the PMC option to SEER SEM as a tool to analyze project metrics indicating performance against project plan. PMC is a key tool for the Key Process Area “Quantitive Project Management (QPM) providing a method to support statistically controlling the project by using the process metrics to analyze and predict future project performance as a function of past project performance. When the predicted performance is not within an acceptable tolerance, process modifications should be made.
At that point we then use SEER-SEM to provide a quantitative assessment of the process alternatives possible to improve project performance. We have used the SEER-SEM tool in that mode for more than 5 years in support of a major DoD program providing on going assessment of project performance and recommendations of “best” process improvements from among the alternatives considered.
Applying a parametric model, reflecting the organization’s processes, you can develop an executable project plan to a defined confidence level . Continued use of the SEER parametric model’s PMC capability will provide a basis for an Organizational Process Performance (OPP) analysis process and the Quantitative Project Management QPM) approach to provide the statistical control of the project.
Thank you for reading “Dan on Estimating”, if you would like more information about Galorath’s estimation models, please visit our contact page or call us at +1 310 414-3222.
Related Posts Computer Generated
Brooks Law Is Applicable To Many Collaborative People Activities
Brooks Law is that there is an incremental person, when added to a software project, that makes it require more, not less time. And adding people to a late software project makes it later.
 I have been spouting Brooks Law for over 20 years for software projects. You know the drill. An important project is late. So the first inclination is to add more people, or put in a red team, or something to speed it up. Unfortunately when Brooks Law comes into play, those additional people make the project fall further behind. What happens is that for each additional person, there is some energy lost as they need to communicate with others. At some point the project has all the people it can handle. And adding people makes the project later. Even if the project could absorb more people, the reduction in productivity as the in-place team has to communicate with the additional staff on the late project causes the late project to get later.
Using this information we can determine where Brooks Law might kick in at any point in the project and can compute the minimum time – the point at which people have been added at the maximum rate that is still productive, and hence a minimum staffing time and numbers of staff. We can also determine the inefficiencies of overstaffing and the schedule penalty / cost improvement from using an optimal effort schedule as well. This is one of the core models within SEER for Software (SEER-SEM).
Although I have generally discussed this phenomenon in software projects, the same applies to hardware, IT or other engineering projects. And even on assembly projects.
I recall working at a cannery one evening. This was a facility that canned food for the needy. We had a great turnout, probably over a hundred people anxious to help. As the peaches came down the conveyor belt, the main job was to cut off any bad parts. Each person, wanting to help, picked up all the peaches they could and cut something off. I am sure the net pounds of peaches canned was well below what it would have been with the right number of people on the line.
And hardware development projects have nearly the same issues as software projects. Add too many people too early and there will be wasted energy. The project can take longer.
The opposite is also true, to a point. Assuming a project is planned with a longer schedule and fewer people from the start: Fewer people on a project can extend the schedule but fewer communication paths cause the total effort to be less.
The basic issue of Brooks Law can be expressed in the equation:
n(n − 1) / 2
This equation computes how many different communication paths exist between team members. Thus the larger the team, the more time people spend communicating rather than making progress. So adding people to a project that is already late increases those communication paths and takes more of the time of those who should be making progress.
 
Figure 2 illustrates that there is a minimum time for any software project to be completed (it can be shipped sooner but to be really ready to ship this minimum time comes into play.)Â Additionally, some cost maybe saved by a planned longer schedule (but not many projects in the US are willing to wait longer… In general, Europe is much more appreciative of the lower cost opportunities.) up to the optimal effort point where the cost begins to rise again.
Â
Â
Â
Â
PS: Use cautain when some claim Brooks law is no longer valid. Proper application of Brooks law should look at individual teams working on individual programs, not an entire large system all at once.
References:
Excellent Knol on Brooks law.
“The Mythical Man Month” is a classic in software engineering, written by Frederick Brooks and describes Brooks Law in detail.
Wikipedia, The Mythical Man Month
Software Sizing, Estimation and Risk Management: when Performance Is Measured Performance Improves“Â by D. Galorath and M. Evans.
Thank you for reading “Dan on Estimating”, if you would like more information about Galorath’s estimation models, please visit our contact page or call us at +1 310 414-3222.
Related Posts Computer Generated
Guidelines and Metrics for Assessing Space System Cost Estimates
I recently saw a presentation by Bernie Fox of Rand regarding guidelines and metrics for assessing space system cost estimates. Very interesting presentation (if you are interested in space systems)
The paper includes:
- Â Average costs and ranges for space vehicles, subsystems, and components for crosschecks.. Powerful Read more
Thank you for reading “Dan on Estimating”, if you would like more information about Galorath’s estimation models, please visit our contact page or call us at +1 310 414-3222.
Related Posts Computer Generated
Estimating United Conference Papers Available On the Web
Follow this link to see some of the estimating united papers. it was a great conference, full of useful information. Talks encompassing cost estimating (cost estimation), value engineering, product design, software, and more.
And thanks again to all the speakers and attendees as well as the hospitality of the Manchester United Football (soccer) club. And the Galorath staff whose diligent efforts made this a great success. Presentations will be available for all to download for the next few months, then will be available as part of the Galorath corporate library.
Thank you for reading “Dan on Estimating”, if you would like more information about Galorath’s estimation models, please visit our contact page or call us at +1 310 414-3222.
Related Posts Computer Generated
“Far Out” Estimating the Costs Of Satellites Far Into The Future
I had the opportunity to brief part of the space community on the latest release of the SEER Far Out model for cost estimating.  Far Out estimates the costs of satellite systems far out into the future. Why would anyone want such an estimate, you may ask. Estimates of systems in the future made today can change planning and approaches for future generations of technology.
This capability is of interest to:
- Military space asset planners
- Government agencies
- Commercial satellite producers
- Advanced concept designers
I have been excited about this project ever since its research began several years ago. It needed to deal with technology readiness levels (TRL), that is how available is the technology being used for systems developed in the future.
Technology readiness levels are interesting since, when they are low, estimates are extremely difficult and will, by nature, have a large range. For example, if you asked Thomas Edison in 1876  how much and how long it would take to develop the light bulb, he likely would have had no clue  before the basic carbon filament was invented. This was a TRL 1 problem.  He had no good way to know how long it would take him to get the most basic technology to a level that would be adequate for a product.
In 1979, once the carbon filament was developed to TRL 4, Mr. Edison would have had a much easier time providing development and production estimates. NASA’s technology readiness level scale follows:
Â
Â
Â
Â
Â
Â
Â
Â
Â
Â
Â
Â
Â
Â
Â
Â
Â
Â
Â
———-30————
Thank you for reading “Dan on Estimating”, if you would like more information about Galorath’s estimation models, please visit our contact page or call us at +1 310 414-3222.
Better Products At better Margins 20 to 40% Cost Price Reduction
I was so impressed with the presentation by one of our speakers at the recent user conference that I bought a copy of his book. In his presentation(and also included on our site, galorath.com) he showed how he saved millions of euros in product costs using target costing, value analysis, and viable cost estimating and SEER.Â
It is wonderful to see forward thinkers applying SEER to save big costs as well as to estimate costs. In his presentation he substantiated savings of 20 to 40% cost price reduction (value improvement) And he does this not be grinding contractors but by working with them to provide substantial cost savings without impinging on their margins.
His book is well worth a read. The abstract follows:
“Many companies experience price pressure as a result of the global marketplace. The advance of technology creates many opportunities adding more innovative new features. Yet these opportunities also seem to drive up costs. How to end the resulting profit squeeze? Many aim for low labor cost countries as a solution.
Our approach is: Use Target Costing and Value Analysis.
Target costing sets allowable costs, derived from the value as perceived by customers, instead of price as a result of cost. Value analysis shows the way where the value of a product is, or where to reduce the cost. One of the key success factors in this approach is Cost Engineering and Cost Estimation. Another is multi disciplinary teamwork between marketers, developers, operations and suppliers.
The structured approach and open communication help achieve astonishing results, typically in the range of 20 to 40% cost price reduction or value improvement.
This book, or better Manual, describes how to apply and implement these methods in an organization, and how to involve suppliers efficiently and effectively via so called Design-in Workshops.”
 It can be purchased for 69 Euros at www.orenda.nl
Thank you for reading “Dan on Estimating”, if you would like more information about Galorath’s estimation models, please visit our contact page or call us at +1 310 414-3222.
Related Posts Computer Generated
Risk / Uncertainty In SEER and Project Management
The question was asked “what probability should I estimate at” The answer follows: Risk is really uncertainty. So the question becomes how much uncertainty can or should I allow in my project plans. There are many approaches to this, depending on circumstances including:
What are the consequences of an overrun: Many large military type programs in the US budget at a 70% or 80% probability. These kinds of programs are generally large and not well defined at the early planning stages. They use the higher probability so they don’t have to go back for more budget or schedule as the program progresses. These programs will generally manage to a 50% probability, using the overage as a buffer for program growth.
Is The Project Fixed Price: Many programs that must bid on programs at fixed price will initially estimate at a probability like 80% so that they are covered if the project becomes more complicated than their initial looks. Of course competitive issues may cause them to bid lower.
Is this an in house Project: Many projects will plan at a 50% (most likely) probability. This allows them to have a tough but achievable plan and if the project runs into difficulty they can adjust.
I recommend managing to the 50% probability… Tough but achievable schedule and effort.
Some projects Estimate at the 20% Probability: This is a lower cost / schedule plan that can win a contract. Unfortunately these projects usually overrun. But they have taken calculated risks.
I once heard of a program that was thrilled that the 1% probability met with their hopes for the program. So they had a plan with a 99% probability of failure.
Thank you for reading “Dan on Estimating”, if you would like more information about Galorath’s estimation models, please visit our contact page or call us at +1 310 414-3222.
Related Posts Computer Generated
GAO Cost Estimating and Assessment Guide Released
Congratulations to the GAO team for releasing the “Cost Estimating and Assessment guide: Best Practices for developing and Managing Capital Program Costs”. This has been a work in progress for several years and represents a nearly superhuman effort on the part of the GAO team. Galorath is proud that is was able to assist in the guidebook development as well.
The guidebook may be downloaded here. And the press release from GAO follows:
Thank you for reading “Dan on Estimating”, if you would like more information about Galorath’s estimation models, please visit our contact page or call us at +1 310 414-3222.
Related Posts Computer Generated
Life Cycle Cost Percentages of Weapons Systems
While at a conference this week I heard the presentation of a paper by Quentin Redman, Andrew Crepea, and George Stratton, all of Raytheon, that summarized the life cycle costs of various weapons systems. This table is reproduced below:
Thank you for reading “Dan on Estimating”, if you would like more information about Galorath’s estimation models, please visit our contact page or call us at +1 310 414-3222.