CMMI Challenges Being Addressed: Process Is As Process Does
SEI people are battling the “Process for process sake” syndrome. CMMI is really trying to invoke a set of best practices. The level is not the goal.
When CMM (CMMI’s predecessor) was first developed the Government promised that the CMM level would not be used to judge organization in contract award. This was forgotten almost immediately.
DCMA released a study in 2007 that found for ACAT-1 (Really big programs) CMMI level was:
- About 83% mostly adhere to CMMI processes
- About 44% somewhat adhere to processes
Another study found that the 6 major reasons for program failure were:
- Business practices
- Culture
- External influences: I what it now
- Enabling Infrastructure
- Joint Integrated Processes
- Acquisition Reform (loss of system engineering capability within government)
NOTE engineering processes were not a root cause of program failure
Many Observed Incorrect Implementations of CMMI High Maturity
At the CMMI conference a number of difficulties with high maturity, CMMI levels 4 & 5 quantified management. Some included incorrect use of standard statistical techniques for example:
- Too few data points
- Updating control charts too infrequently
- Bad formulas in control level calculations
- Trying to statistically manage inappropriately
- Using aggregated data
- Presenting baselines as stable when they were not
- Lack of quantification of quality and process performance objectives
- Lack of management use of information
- Using trivial cases to show CMMI levels 4 & 5
They also discussed how many organizations just wanted a number.. that they didnt really have a commitment to process. And how that is (finally) changing.
I was thrilled to hear that practice is changing. Over the years we have been disturbed by the trend to get a small project or organization certified, then do business as usual throughout the company.. that is process not being used.
This is the reason SEER for Software provides approximate CMMI level as an output rather than an input. When CMM first came out people asked for knowledge bases or parameters the woudl change the estimate just based on the CMM level. We actually built some draft knowledge bases based on the characteristics of organizations with higher maturity but never released them because we realized that organization CMM or CMMI level does not generally impact the individual projects on a per sey basis. We realized that people, process and technology inputs to SEER actually predict the effort, schedule and effective CMMI level best.
SEER estimates any CMMI level well. It used the root causes of improvement (people, process, technology, reuse, etc.) to generate such estimates. The other reason this is so powerful is because management and other stakeholders can see and understand the specifics of why their estimates change.
Mapping Agile Principals to SEER for Software (SEER-SEM)
I thought these slides from Galorath’s David Dewitt covering estimating agile systems in SEER were interesting, showing which SEER parameters deal with which Agile characteristics. SEER handles agile developments out of the box. David used SEER to estimate and manage Agile projects as a software project manager before he joined Galorath. David also did an entire webinar on Agile.
Dan’s 2 Papers At CMMI Conference In Denver Nov 18 & 19
Dan will be giving two papers at the CMMI conference in Denver and will be available for consultation during the Tuesday and Wednesday Nov 18 and 19.
1. Improving Project Planning and Control: A 10-Step Process Within CMMI or other Process Orientations Click Here
2. CMMI’s Role in Reducing Total Cost of Ownership: Measuring and Managing New and Legacy Software Click Here
2009 IT Spending: Generally Flat
According to research published by Computer Economics, Inc. on 2009 IT spending:
about 34% of IT organizations have cut 2009 budget
About 11% have increased IT budgets
About 54% have no change in budget from 2008
Doing more with the same or less requires better project and operations planning. That is why SEER for Software and SEER for IT are used by organizations looking to optimize their productivity within scarce budgets.
How Galorath Quantified the Salesforce.com Platform With SEER For Software (SEER-SEM)
The following summarizes the report prepared by Galorath’s Dr. Tarbet on the process of building specific Knowledge Bases and validation of SEER for Software (SEER-SEM) for the Salesforce.com platform (Force.com). Force.com enables you to deliver enterprise-class Web applications on demand—without the cost of deploying infrastructure, supporting the software-as-a-service paradigm (SaaS). This was customer funded (by a potential user of salesforce.com).
Galorath established a technical interface with internal Salesforce experts, consultants with extensive experience developing software in the Salesforce.com Platform, and with current users of the on-demand SaaS capabilities provided by the platform. From the technical discussions, preliminary analyses of expected effort impacts on the software development effort for an IT project were derived. A Salesforce.com Platform Knowledge Base and a language definition for the APEX language has been developed for SEER-SEM from the technical inputs.
Bottom Line 30% to 40% Savings: The model indicates that effort can be expected to be reduced from 30% to 40% over developing the same project in JAVA for projects that are aligned to the Customer Relations Management model, which serves as the basis for the Salesforce.com Platform. Read more
Some Gottchas of Estimating Service Oriented Architecture Systems (SOA)
As Fredrick Brooks said “there are no silver bullets.” Unfortunately the software industry continues to tout every incremental step as that silver bullet which will solve the problems of software development. Don’t get me wrong. SOA when applied in the appropriate environment can improve productivity. Galorath analysts have been using SEER to estimate SOA systems for some years now. The basic conclusions are 1) SOA can be the best approach to achieving interoperability and reusability and 2) SEER accurately models the effort, schedule, risk, cost of software development of an SOA system as well as the cost of the IT services and infrastructure to support them. But the reality must be modeled, not just vendor claims.
Grace Lewis, Et Al. of the Software Engineering Institute did a good job of identifying some of the misconceptions of SOA as follow:
SOA Does Not Provide a Complete Architecture: SOA provides a pattern for an architecture, not the whole system architecture itself. Hence you can’t really buy SOA off the shelf. You can buy SOA products to help.
Dan Woods of Forbes points out: “The problem is that most IT departments are used to running applications. We can think of them as automakers assembling a car from many different parts. The auto manufacturer makes sure that all the parts fit together and work as they should so the car runs well. The world of SOA lifts the hood of the car and breaks the rest of the car into parts, that is, individual services that can be reassembled. “
Migrating Legacy Systems Is Nether Automatic Nor Easy… There Are Significant Costs
The Use of Standards Does Not Guarantee Interoperability In An SOA Environment
Testing SOA Systems Is Harder: “Because an SOA environment is distributed, loosely coupled, and asynchronous, testing can be significantly more complex than simply testing a set of known paths in a single system.”
Gartner also raised up the caution flag stating that SOA deployments were slowing down as people begin to understand the real costs.
Also, Forbes Dan Woods post provides a sobering observation: with SOA, IT departments end up bringing much of the work formerly handled by software vendors in house. So for estimation purposes plan on doing more IT development and services in house… And plan on training, retaining, and equipping those personnel.
SEER Quantifies Savings On Military FPGAs (Custom Chips)
Altera used SEER for Hardware Electronics and Systems to quantify the business case for FPGAs The results were published in ECN Asia.
Part of the case study follows:
“In order to illustrate potential design productivity savings in an optimized design flow, a sample FPGA-based design was developed and measured with a third-party design estimation tool. This design example was co-developed with engineers at Galorath, creators of SEER-SW, SEER-H, and SEER-IC software. These tools allow users to enter parametric descriptions of their equipment designs, and to develop cost and schedule estimates for both proposal development and resource planning. The estimates are based on knowledge databases developed by Galorath and various users of the SEER-series software tools.
The military design example shown in Table 1 takes advantage of the 40nm Stratix FPGAs. This cost estimate focuses on the FPGA design only, and includes design, verification, place-and-route, and simulation, but not board design. The SEER-IC estimation tool allows the user to enter lower and upper requirement and parameter limits to measure cost and schedule risk, as well. This design example is assumed to be a very large, multi-channel sensor design. Logic element (LE) cost will likely put it in the Stratix IV class of FPGAs. There is a large number of I/O pins and new design (50 percent). The complexity of both the interfaces (front end) and processing (back end) is relatively high. Current development tools and practices are estimated to be low quality, and the (average) capability of the designers to be nominal. Requirements volatility is assumed to be normal for military programs. Read more
Effectiveness Measurement At Galorath Still Focusing On the Positive
I was in an ISO review today at Galorath Inc. I saw customer satisfaction scores that were through the roof (in a good way.) I saw compliments (from the ISO compliments file) that were flattering. I asked about the complaints file (also an ISO requirement) I was told we had no complaints since the past review. I personally appreciate the compliments but learn more from complaints. When pressed I found nothing was reported as complaints. But there certainly had been some areas where we could have done better.
Process improvement is about finding where things could be better, not about painting rosy pictures. We are in the business and even we like to concentrate on the good and minimize the bad.
We are probably better than most in this regard. Process improvement and improvements in customer satisfaction is important. And to achieve these we have to identify and stress where we can do better.
The Mistake of Measuring Everything
I recently participated in a seminar with Larry Dribin of Pearl Street Group. Larry did a beautiful job with a talk called something like “You Get What You measure” Afterwards I added the corollary “If you measure everything you get nothing”
I have seen a number of organizations that just start measuring everything. They figure they will do something with the measurements sometime. Measurement becomes a burden with no ROI. And people can’t respond by doing the best job on what they are measured against. Read more
