IEEE Publishes New Software Failure Study: Less than 20% Failed
This new article , “A Replicated Survey of IT Software Project Failures” in IEEE Software Sept 2008 suggests that the percent of canceled projects is less than 20% and that it is not appropriate to call software a crisis. “There’s a general decreasing trend over time, with the most recent estimates mostly below the 20-percent level.” I recommend the article and IEEE software in general. The article sites many studies of cancellation:
| Cancellation / Abandonment % | |
| Standish 1994 | 31% |
| Standish 1996 | 40% |
| Standish 1998 | 28% |
| Jones 1998 systems | 14% |
| Jones 1998 military | 19% |
| Jones 1998 other | >24% |
| Standish 2000 | 23% |
| Standish 2002 | 15% |
| Computer Weekly 2003 | 9% |
| UJ 2003 | 22% |
| Standish 2004 | 18% |
| Standish 2006 | 19% |
Government IT Project Woes and Estimating Total Ownership Costs
A recent government report showed 81% of budget or $57 billion in IT projects in danger of failing. Detailed reports on the hearings can be found here. Of 413 IT projects identified by OMB and federal agencies NEARLY 80% OF THEM WERE IDENTIFIED AS HAVING BEEN POORLY PLANNED. The scorecard for IT projects shows much progress but much work left to do.
Some of our recent analysis of IT costs seems to indicate that software development may be the smallest of four components:
1. Software Development
2. Software Maintenance
3. IT Infrastructure
4. IT Services
In this example sanitized system we see software development costs (the costs many track the most.. and for good reason.. these are the highest risk items) were about 10% of the total cost of ownership. Meta analysis shows IT Infrastructure & services can be 60% or more of the system’s total ownership cost.
Forrester’s Five Essential Metrics for Managing IT
A recent report by Craig Symons of Forrester identified 5 core metrics for managing IT. This paper is full of wisdom such as “metrics for its performance must measure relevance and business impact”… effectiveness metrics rather than operationally focused metrics such as downtime.
1. Alignment of IT Investments to Business Strategy (shows how IT projects meet business strategy goals & how not all projects are mandatory))
2. Cumulative Business Value of IT Investment (shows not all projects are equal)
3. IT Spend Ratio - New Versus Maintenance
4. Critical Business Service Availability (Linked to applications or services, not generic technology)
5. Operational Health (Reliability, Safety & Security, Quality Project Execution)
This research note is currently available for free download as a special promotion (requiring registration) from a third party company.
SEER for Information Technology provides the ability to answer most of these metrics questions in quantified, risk adjusted manner.
CECIM: A Capability Improvement Model For Cost Engineering
Being a major fan of repeatable process for cost and cost engineering, I cant help but admire the work of CECIM (Capability Improvement Model for Cost Engineering.) CECIM was an excellent piece of work in establishing estimation maturity and repeatable process. The major document is a white paper written by H. Pickerin and D. Lewis. Carl Dalton of Galorath International teaches CECIM along with Hugh Pickerin in the UK. Read more
How Hard Is A Hardware Model
We caution organizations with trying to accomplish this challenging endeavor with spreadsheets or simple CER type models as those approaches can be wrought with risk. And a poor estimate is worse than none at all.
Performance Based Earned Value
It has been very interesting reviewing Paul Solomon’s Performance Based Earned Value. It is in many ways consistent with SEER’s monitoring and control. It preaches looking at quality in addition to quantity of work performed as well as capturing the issues of deferred functionality. I should point out that we sometimes called SEER’s Monitoring & Control Performance Based Earned Value before we realized it was trademaked. Since we have stuck with 4 dimensional earned value for SEER.
PBEV has 16 guidelines in addition to the 32 included in the EIA 748 earned value standard. Read more
Why SEER Got Started
Many years ago a project was never developed because of my realistic estimate. I wondered if I had done the right thing, which was try to provide an achievable project plan and a realistic estimate, even though that estimate was longer than the company desired. This experience made me wonder if I had failed as a manager. However, some years later I tried to reproduce this estimate using SEER-SEM and discovered that I had significantly underestimated the project. SEER-SEM enabled me to understand that I had not failed and that my refusal to give in to the division head’s pressure had been the best thing for that company. Read more
The Total Ownership Cost of IT Systems
It is interesting to see how the much of the industry speaks of IT system cost problems while ignoring a large part of the problem. Software maintenance can be 75+% of software total ownership costs but IT Infrastructure and the cost of IT systems and services can be 60% of the total ownership cost itself. Even if we use cloud computing most of those IT service costs don’t go away. Help desk, data conversion, etc. is still a problem, even in the cloud. Read more
Estimating Is Not A Problem
I spoke with someone recently who had seen a survey that showed estimating was not a problem. I was interested in that. With cost and schedule overruns being the norm, and deferred functionality so often being the means of delivery, why people would not think estimating was a problem.
It occurs to me that those persons may not have gone thru the root cause analysis of their project difficulties. Estimation is the basis of an achievable plan. And that achievable plan, factoring in risk should yield more successful projects. However if success is defined as fact free planning, then estimation is not a problem, the entire process is.
GAO Cost Assessment Guidebook
The GAO cost assessment Guidebook is something you should keep an eye on if you work in the government environment at all. And if you are in a commercial business there are some good tidbits in there for you as well. As I understand it the guidebook was originally intended to support auditors. Then its scope was expanded to provide a consistent methodology and identification of best practices for both government and industry for developing and managing program cost estimates. I, along with a cadre of others have been providing review and feedback.
The guide lists a 12 step estimating process and guidance in numerous areas. While it is still a work in process I am generally pleased with it at this point. The July 2008 draft can be found at GAO Cost draft.
One area I was especially interested in included, a 5 level scale of estimating maturity. They originally had three levels. That just didn’t seem consistant with machuriy scales. I send them the CECIM work done in the UK on estimation process. And I believe a 5 level scale has now been Incorporated.
