Software Cost Estimation Is Like the BCS Ratings
Thanks to David DeWitt for this article. It is just disappointing that I sat outside at a football game in the freezing cold last week and they didn’t come up as BCS champions. 
Software Cost Estimation is like the BCS ratings…
Everyone that knows everything hates them!
David W. DeWitt, Galorath Incorporated
Know your college football? Hmmm, pick a year and name the NCAA football champion.
2008-Florida? 1986-Penn State? 1962-USC? Nope, Nope, and Nope!
No matter what year you selected you were wrong! The NCAA does not formally determine a champion for this category. But… since a gazillion dollars depends upon those bragging rights – the Bowl Champion Series (BCS) was invented. And like a software estimate – it’s been almost as popular!
The BCS computer solution was supposed to be so simple; let the computer pick the best option using statistics, add a dash of erudite pontification (pollsters), stir in a seasoned splash of professional gesticulation (coaches) and voila – an estimate of the two best teams in college football! Yes indeed, that sounds exactly like a software estimate!
The BCS scoring process uses a combination of computer tabulations and two disparate polls to determine team interim rankings, and then to identify the best two college football teams to play the “BCS” National Championship Game. The winner of this game is named the “BCS national champion.” (Not NCAA Champion!)
In BCS banter one will hear expressions like non-linear equations, standard deviation, and the Bayesian approach. Wow – that’s similar to software estimation speak. Hmm, maybe they are onto something. What if we did a software estimate like the BCS picks the best college football teams in the nation. Our premise for the typical corporate software estimate will be to determine the best cost and schedule. For the BCS – we pick the best gate attraction (income) and post bowl game extravaganza!
The Recipe:
| Software Estimation Inputs | BCS Calculation Inputs |
| How well did we build this the last time? Historical Data (though likely not collected)What are thoughts of the senior staff? Senior Management (1/2) What do the programmers think? Identify the Platform and Application, factor in the size, calculate industry averages, sieve through the parametric settings, insert Monte Carlo simulation, offset for risk confidence and schedule probability… |
What did the teams do last year? Previous year rankingsWhat do other pollsters think? Harris Interactive Poll (1/3rd) What do the people in the trenches think? Drop highest and lowest ranking of each team divide by 100, for maximum possible points |
That’s it for the BCS calculations – 1/3 * 1/3 * 1/3 = an answer. But alas – the BCS formula has come under dire – or dare I say – “political” scrutiny. In fact, in October of 2009, when his school was excluded from a previous year BCS bowl game, Senator Orrin Hatch of Utah sent a 10-page letter to President Obama calling for an antitrust probe of the BCS:
“Mr. President, as you have publicly stated on multiple occasions, the BCS system is in dire need of reform,”… “If the government can look at the concentration of money in railroads, telecommunications and software developers, then why not the big business of college sports in America?”
Software Developers??? – Yikes.
In a BCS system where you have nearly a full Delphi approach – from coaches and pollsters – and then add in a computer statistical model – the computer gets blamed! No wonder software estimates are so mistrusted. While everyone knows that a parametric model is essential to making a reasonable software cost estimate, it is nearly always often overridden by the objections of senior management who wish to also override those pesky “optimistic programmers.”
Has anyone considered this? The BCS by default was flawed. It excludes all but six original signers to the BCS covenant (plus Notre Dame and a few others). Without all the teams considered – without all the weightings of records and wins – and without all the scores somehow magically factored – no one will ever be happy. Kind of like a software estimate that ignores history and trends… No one ever thinks the project will take that long – until it does.
Until then, does anyone have Senator Hatch’s number? Note to non-football experts… Senator Hatch has objected to the BCS system as unfair. It would be nice if he would object to all the poorly crafted guesstimates and demand more rigor in software estimating too.
Thank you for reading “Dan on Estimating”, if you would like more information about Galorath’s estimation models, please visit our contact page or call us at +1 310 414-3222.
IT Costs Per Transaction Provide Valuable ROI Basis
Here was a recent post on the Harvard Business Blog on understanding the transaction costs of IT. According to the blog:
- the IT cost per equity trade is approximately $0.17
- the IT cost per hospital bed is $65 per patient
- the IT cost per trucking mile is $0.18
“…executives usually think about technology in terms of percentage of revenue, or percentage of operating cost — which contains almost no useful operating insight at all”
I thought this was an excellent point and looking at IT costs in this way can add reality to the cost of IT. What we need now is the business value of IT. For example, if having IT for a hospital bed includes automated monitoring and reduces staff or increases correct treatment (hospital error is shown to be the third largest cause of death in hospitals, according to some studies) we can make the most appropriate decisions.
Thank you for reading “Dan on Estimating”, if you would like more information about Galorath’s estimation models, please visit our contact page or call us at +1 310 414-3222.
FPGA Growth and Costing
FPGA Growth and Costing
by Sam Sanchez, Technical Director-Electronics
Field Programmable Gate Arrays are continuing to absorb more and more capabilities that used to reside elsewhere on electronic boards. In the past few years, FPGA devices have not only grown in size but also in the amount of embedded cores like processors within their fabric. As these devices have grown in capability, so has the magnitude of effort required to design and test them. This, in turn, has made it necessary to come up with techniques to cost them separately. These are no longer just expensive components within the board but are now systems in themselves. In order to properly cost these devices, the first thing to wrestle with is how to properly size the designs. In the past, some have used number of gates (similar to the way ASICs are sized), but this just isn’t standardized enough within the FPGA world to allow accurate and consistent sizing across various vendors or technologies. An alternate way to do a first order approximation of the design size is with VHDL although this also has its challenges. Yet another way is through the use of Logic Cells (LC) or Logic Elements (LE).
Although, the definition of a LC/LE varies across different technology nodes, it can be a very good sizing metric. We currently use this sizing method within our SEER-IC model. Another aspect of properly sizing a design is how to give credit for past design effort. Similar to the way that software designers can bring prior efforts into their current design through COTS software, the IC industry can also bring prior effort through Intellectual Property or simply IP logic. Increased use of well designed IP (emphasis on well designed) can reduce the amount of design effort. Of course, testing and verification are still significant tasks that must be done on the entire design (new logic and IP). Being able to settle on a standard sizing process will go a long way in helping to properly capture FPGA costs.
Thank you for reading “Dan on Estimating”, if you would like more information about Galorath’s estimation models, please visit our contact page or call us at +1 310 414-3222.
Controlling Software Projects: Development Is Only Job One: Chicago SPIN Nov 12 2009
Dan will be speaking at the Chicago SPIN on November 12, 2009 on the topic of controlling software projects. Estimation, planning, control, metrics, and maintenance for a total ownership cost view will be discussed.
The presentation is here: Chicago SPIN November 2009 Galorath Presentation Controlling Software Projects Development Is Only Job 1
PS Dan looks forward to his short visit to Chicago, his home town. And is going to carefully avoid pizza, hot dogs, and Italian beef while he is there.
The flyer follows:
Thank you for reading “Dan on Estimating”, if you would like more information about Galorath’s estimation models, please visit our contact page or call us at +1 310 414-3222.
Alpha Acquisition Process And Impact On Cost Analysis
The Alpha Acquisition process may provide a cost savings to both government and contractors. The following is from the Defense Acquisition University website:
The alpha contracting process involves many activities performed jointly by the Government and contractor teams.
This process innovation offers a number of advantages and performance enhancements, such as:
- Improving communications
- Decreasing the number of formal RFP iterations
- Lessening revisions and rework required to correct misunderstandings
- Reducing errors and mistakes
- Shortening the cycle time (procurement administrative lead time or PALT) required for contracting
- More
The benefits of Alpha Acquisition are not limited to reducing procurement acquisition lead times.
Contractor Reduced Proposal Preparation Costs
The contractor benefits by significantly reducing proposal preparation costs. Alpha Acquisition is a framework for expediting the acquisition process. The purpose is to eliminate any unnecessary processes and reviews, and to streamline and conduct in parallel the required ones. Nevertheless, the same issues addressed in standard procurements are addressed in Alpha Acquisition, the same questions asked, and the same support provided. However, it is all done much more quickly and started earlier in the process.
DCAA Cost Savings
One benefit includes the early involvement of DCAA personnel in the immediate utilization of rate recommendations, rather than at some later date when significant updates would have occurred. This results in a cost and time savings for DCAA by precluding subsequent reviews.
DCMC Savings
DCMC receives a cost and time savings by performing one review rather than several as a result of proposal updates.
Additional Government Program Office Costs
See the comment attached to this BLOG entry from one working in a program office.
Thank you for reading “Dan on Estimating”, if you would like more information about Galorath’s estimation models, please visit our contact page or call us at +1 310 414-3222.
New Code Counting Tool Made Available
A new version of the USC code counting tool has been made available to the public. Code counters are useful for understanding the volume of pre-existing software inherited by a project and to provide actual counts for organizations that use lines of code as a metric. Such counts can be put into cost estimating models such as SEER or USC’s COCOMO as well. I realize this can start significant arguments but lines of code are a valuable measure in some environments, just like function points or use cases are valuable in others. Beware, there are many buggy or malfunctioning code counting tools on the web. I know the USC team and those who assisted and this is likely to be more robust. USC’s announcement follows:
Dear Affiliates and CodeCount Users:
We are pleased to announce the release of an upgraded version of the CodeCount toolset called Unified CodeCount or UCC. The software source code and supporting materials are available to the public from http://sunset.usc.edu/research/CODECOUNT/
We would like to thank for the continued support of the CSSE Affiliates for the development and maintenance of the CodeCount toolset. The project has been a collaborative effort of the USC CSSE, the Northrop Grumman Mission Systems, the Aerospace Corporation, and NRO. We would like to thank Mathy Pandian of Aerospace Corporation; Lori Vaughan, Michael Klug, Aaron Shumate, Mike Wheeler of Northrop Grumman Mission Systems; Jill Dunn of NRO, and student contributors for contributions to this effort. We would also like to thank the users of the early versions of the tool for valuable feedback and bug reports.
Thank you for reading “Dan on Estimating”, if you would like more information about Galorath’s estimation models, please visit our contact page or call us at +1 310 414-3222.
Why Projects Fail From A Technical and Process Viewpoint
There was an interesting article in SDtimes
The article claims that “Most cases of failure that I have seen have been in two categories: imagination and process,” quoting Grady Booch, chief scientist of software engineering at IBM Research. These include:
- Requirements failures
- Failure to verify/validate requirements
- Failure to adhere to architecture
- Lack of risk management
- Lack of lessons learned
- Communication breakdown
The article has some examples of faulty requirements such as the Pennsylvania man who could not obtain driver’s license because the computer had classified him as dead and there was no way to correct that.
Thank you for reading “Dan on Estimating”, if you would like more information about Galorath’s estimation models, please visit our contact page or call us at +1 310 414-3222.
Dan Galorath Presentation at SEER / Tracer Seminar In Tysons Corner October 2009
It was exciting to participate in a seminar with Computer Aid and their ITMPI recently. Here we discussed concepts of software management and how Galorath’s SEER with project estimation, planning and control supports CAI’s Tracer product witch does root level process and task management.
Jim Ryan first spoke and discussed how the manufacturing revolution worked and how much of the success in manufacturing process and process measurement can be applied to software development. And excellent presentation that made a lot of points really apparent.
Next Dan Galorath went through software management techniques (click link to view presentation) that most likely yield successful projects, including estimation (cost estimating, schedule estimating, defect insertion/ removal estimation an more), planning, control, measurement, commitment and other items.
Thank you for reading “Dan on Estimating”, if you would like more information about Galorath’s estimation models, please visit our contact page or call us at +1 310 414-3222.
Project Prediction Infographic
