Have We Lost Our Ability to Estimate Software Size
I really enjoyed Galorath’s David DeWitt’s article “Have We Lost Our Ability to Estimate Software Size, AKA I’ve Been Slimed”, posted below:
David DeWitt,
“I’ve Been Slimed!” – Dr. Peter Venkman, Ghost Buster
I can clearly remember that day I arrived at work – towards the end of the year 2003 – it was easily before 6am. I was leading a small team tasked with prototyping a test environment for a NASA proposal. I stood there amazed as I watched my two programmers demonstrate a completely reengineered satellite simulation environment. Wait – let me be clear – within only a few days – they rewrote close to 30,000 lines of FORTRAN and another 4,000 lines of assembly code. How did they do it? They called it “goop.” “The hand cleaner?” I asked – rather befuddled. No, they were referring a new development language by LabVIEW (National Instruments) called “GOOP” – short for Graphical Object Oriented Programming.
That was the day I decided to stop being a programmer. I was, at my ‘age,” no longer really interested in keeping up with the latest programming paradigms (and vernacular – such as “paradigm”). I decided to abandon the past and embrace my role as a program manager. But now, looking back I wish I would have asked a few more questions.
Measuring Failure
In 1995 the Boston, Mass. – based IT project management research and consulting firm The Standish Group released their first CHAOS Summary report. The report quickly became an industry score card for measuring the success or failure of IT projects; due mostly in part to the astounding percentage of failed projects disclosed in the report. The report served as a wake-up call that appears to have been heard – the 10th anniversary CHAOS report announced that the percentage of failed projects had been reduced by more than half. But alas, within a mere five years, the number of failed projects is back on the rise; the 2009 Standish Group CHAOS report indicates that nearly 1 in 4 projects are doomed. But why?
According to the original 1995 CHAOS report, to improve the probability of success projects should be reduced in complexity and the software “grown.” The recommendation was to reduce software into smaller, more manageable segments, and develop it outward. If this reported reduction in failed projects is to be believed then it appears the software industry was diligent in “growing” projects using smaller elements[1]. However, what should also be understood is that those projects were comprised of many smaller pieces that were easier to “size.”
In 1995 the most common approach to sizing software was to count the source lines of code (SLOC) or count Function Points (though less prevalent). Software sizing was an established and mature methodology spanning over twenty years – with a myriad of tools available to automate the process and additional metrics available to measure software complexity and probability of defects (bugs). If the size of a project was understood, then the ability to estimate schedule and effort could easily be modeled by applying previous performance measures (and many other parameters). By forecasting a realistic estimate early in the development cycle there was a significantly higher probability of the project’s success (on time, within budget, at promised functionality) – and hence – less failure.
Back to the Future
It’s 2003 again, and I’ve just been told about a new methodology that allows anyone to build software using graphical components. This was not really all that new; in the mid-1990s through early 2000s “visual” and “portable” languages gained industry acceptance and began to dominate the development landscape. Within just a few years, languages that could be “produced” by an environment became the lingua du jour. After all, who could argue with the massive scale of economy that software manufacturing tools could generate using “Visual” programming? And for me in particular – after three days of watching my team crank out “GOOP” – I was a hero to my management.
But wait – notice the timeline in the chart above and the resurgence of software failures. While I am not a big proponent of causality – let’s at least take a few moments and explore this potential contributor to the trend in software failure. First, a picture from the TIOBE Programming Community[2] – the unofficial keepers of what is popular in programming languages.
The most predominant language for five years of the decline in programming has been: Java, followed by C#, JavaScript, and then Ruby. Take a moment to look at the other languages coming in vogue over the past five years. Why do you think Perl, C, and Visual Basic have actually grown in favor – after all, they’re old school languages (and I don’t find them sexy – but that’s me). Most all of these top-ten languages have become increasingly more sophisticated, are wrapped in integrated development environments, and are positioned with the sole purpose of increasing productivity. But there is little consideration for how to measure the amount of code being generated. In fact, the environment builders boast that one barely needs to fiddle under the hood; Draw, click, and Poof – instant code that runs.
Size Matters
The most significant driver to how much time, cost and effort it takes to build software is the scope (or size) of what is to be built and therefore one of the biggest factors in accurate estimation. As the Godfather of software estimation has warned us (Barry W. Boehm) – “The biggest difficulty in using today’s algorithmic software cost models is the problem of providing sound sizing estimates”[3] How does an estimator measure “GOOP” and how many lines of code that a code generator inserts are really needed? What percentage of a C++ template can we remove (if we dare) and keep the Class fundamentally stable – yet concise. As Mark Twain once said – “the hardest part about writing is removing all the extra words.”
Is it possible to count software lines of code anymore? Even using the best code counting tools available – aren’t they really just counting lots of lines of code that may be unnecessary? Or in inverse – how much time did it take the programmer to remove all that code that should not have been counted – and was not? My suspicion is that all the code stays in (unless a standard with high rigor like FAA DO-178B verified the system).
Since I’m on the topic of counting code – what happened to Ada and FORTRAN; those stalwart languages of the 80′s and 90′s that were easy to count? Alas, they are now number 24 and 25; again, no assumption of causality. (But yes my tongue is planted firmly in cheek). Estimates seemed so much easier then. Cue the music.
Hmm, there is something becoming more clear in the Standish report – assuming I am not making what statisticians would call an “error of confirmation”, or falling into a “narrative fallacy” I would propose that perhaps we have made the business of sizing software far too complicated – and by subjection, our ability to accurately estimate software size (and a good cost estimate). After all – just how long does it take to make GOOP?
PS From Dan… Functional sizing methods such as Function Points, SEER Function Based Sizing, Use Case Points, COSMIC Function Points, etc. can be entered into a model like SEER-for Software (SEER-SEM) to get an accurate estimate of effort, schedule, cost, etc. since SEER understands the amount of work required to develop software, even in a visual interface. That is why SEER uses effective effort units for functional estimation.
[1] Jim Johnson, chairman of The Standish Group, says he was so surprised to observe a dip in IT project success rates that he waited an extra four months before publishing the CHAOS report to make sure its findings were accurate. He attributes the increase in IT project failures to the recession, which according to economists began in December, 2007, and subsequent budget cuts.
[2]http://www.tiobe.com/index.php/content/paperinfo/tpci/index.html
[3] Software Engineering- Barry W. Boehm’s Lifetime Contributions to Software Development, Management and Research., Edited by Richard W. Selby , Wiley-IEEE Computer Society Pr; Reprint edition (June 4, 2007)
Thank you for reading “Dan on Estimating”, if you would like more information about Galorath’s estimation models, please visit our contact page or call us at +1 310 414-3222.
Related posts:
- Have We Lost Our Ability to Estimate Software Size? Following is an update to Galorath’s David DeWitt’s article on software sizing: Have We Lost Our Ability to Estimate Software Size?...
- Estimate Packaged Software Or Suffer the Consequences I am excited about the Galorath Webinar covering packaged software estimation. It always amazes me when I hear people say...
- Software Staff Size Still Impacts Productivity: Brooks Law Lives! When drawing conclusions from data it is important to analyse all the relevant considerations and not just one variable. For...
- Software Code Counter Review and Recommendations Galorath’s Mike Churchman did a comprehensive evaluation of numerous code counting tools and provided recommendations. While some people may wonder...
- Step Nine: Document Estimate and Lessons Learned Each time you complete an estimate and again at the end of the software development, you should document the pertinent...
Comments
3 Responses to “Have We Lost Our Ability to Estimate Software Size”
Leave a Reply
Project Prediction Infographic
The way this post has been displayed is very good and impressive. The key things mentioned in this blog are very important if one focuses.
Wow I am really impressed by the way this post has been presented. It shows how much work the writer has done for this post. Well done.
The way that the author explained everything and by representing them in charts is really commendable.