The 6 Characteristics of Effective Metrics
I had the pleasure of meeting and working with Bob Lewis of Infoworld and CIO magazines recently. In looking through his many excellent blogs I noticed this one today. It sumarizes a viable metrics program very well and is compatible with the goal, question, metric approach. He calls it the 6 C's of effective metrics:
- "Connected to Organizational Goals: Good metrics are connected to important goals. In fact, they begin as important goals, stated in English.
- Consistent: Consistent metrics always go in one direction when the situation improves and the other direction when it deteriorates. If good doesn't always point in one direction and bad in the other, your metric will drive organizational dysfunction.
- Calibrated: Calibration means you get the same value in the same situation no matter who records it. It also means the data are free from sample bias and other quality problems.
- Complete: Anything you don't measure you don't get, so any useful system of measures must include all factors that are important to achieving the goal.
- Communicated: The purpose of metrics is to drive behavior. If you don't communicate their purpose, they won't drive behavior.
- Current: Goals change. Keep the old measures and you'll achieve your old goals, not your new ones."
Thank you for reading "Dan on Estimating", if you would like more information about Galorath's estimation models, please visit our contact page or call us at +1 310 414-3222.
The Future of Software Analysis and Measurement : Expert Panel Questions & Answers
Here are some of the questions that were answered after the webinar with Bill Curtis, David Herron, and Dan Galorath. They were answered on Cast software’s Facebook page.
What about aerospace component level software and how would you apply a cost factor their complexity factors?
What is IFPUG Back Fired Points, how does it help?
Daniel Galorath Backfiring means counting lines of code then using a number of lines per function point to approximate function points. I think it is much better than nothing for a finger in the wind. Many others in the industry object strongly to it.
Another question – What type of information is needed to start using a software analysis and measurement tool?
Bill Curtis First you need to know how you want to use the results. That will help you establish criteria for evaluating which of the various tools best meets your needs. The Goal-Question-Metric paradigm is a good guide for determining what your measurement needs are.
Here’s another question from the webinar – Would like to understand how we can improve our ability to capture metrics (defect density either using effective loc and or functional point) for applications that rely heavily on database’s where the logic resides – e.g to map biz rules existing in table rows/columns etc?
Daniel Galorath Generally you would count the work involved in building the database, not the entry of the data into the database itself. SEER will provide estimates of defect density, etc. From my understanding CAST can provide measurement of this.
What is the panel’s opinion on LLOC to FP translation such as QSM language by language table which they developed from their database.
Do you have to have a technology inventory?
How do you extend the IT governance to software suppliers? what requirements to pose to them?
Bill Curtis Customers are starting to write measurable quality targets such as robustness or security targets into their outsourcing contracts as the equivalent of service level agreements. They then establish a Quality Gate where all software received is measured and evaluated before being put in operation. If the supplier’s software falls below the quality target they must remediate the code or face a financial penalty.
Thank you for reading “Dan on Estimating”, if you would like more information about Galorath’s estimation models, please visit our contact page or call us at +1 310 414-3222.
The Future of Software Analysis Measurement Webinar Featuring Bill Curtis, David Herron and Dan Galorath
We have some exciting events coming up this month with the first: The Future of Software Analysis and Measurement on October 12, 2011. I am really excited to share the virtual podium with Bill Curtis, Senior VP and Chief Scientist of CAST Corporation and with David Herron, David Consulting Groups VP of Knowledge Solution Services as moderator.
Bill and I both spoke at a software engineering conference last year and I got very excited about his work in software analysis with CAST. Besides Bill being an engaging speaker his content was very illuminating, covering issues of existing software, its complexity and reliability. He ev!–more–
en showed the number of latent defects in software of various languages. I have greatly summarized some of the a href=http://www.galorath.com/wp/software-defects-in-fielded-software-cast-analysis.phpsoftware defect conclusions /aelsewhere on this BLOG and CAST has been instrumental in recognizing and quantifying the a href=http://www.galorath.com/wp/500-billion-it-debt-for-deferred-maintenance.phptechnical debt/a
And with David Herron, one of the most knowledgeable people in the measurement community, this should be a do not miss event.
Details of the event follow as does a link to signup. Hope you can make it.
blockquotestrongThe Future of Software Analysis and Measurement/strong
October 12, 2011 8:00am Pacific, 11:00am Eastern, 4:00pm London
a href=http://www.castsoftware.com/news-events/event/future-of-sam?gad=glrClick this Webinar Link to sign up/a
Join us on October 12th to hear from an exciting lineup of experts on the Future of Software Analysis and Measurement: Dan Galorath, President CEO of Galorath Inc and Bill Curtis, SVP Chief Scientist, CAST will have an engaging discussion moderated by David Herron, VP, Knowledge Solution Services, David Consulting Group.
These industry veterans will share experiences with their client’s software development processes and discuss how Software Analysis and Measurement tools coupled with Parametric Estimation models can impact organizational performance through increased ROI, customer satisfaction and business value.
The panel will provide insightful and actionable steps that will make an immediate impact on your strategy including how to:
• Drive organization value by fueling Estimate and Measurement practices within an enterprise
• Build the funding rationale through proven economic impact models
• Establish the ROI from Estimate and Measurement practices and process/blockquote
Thank you for reading “Dan on Estimating”, if you would like more information about Galorath’s estimation models, please visit our contact page or call us at +1 310 414-3222.
Measurement and Providing Value to the Business ISMA Keynote 2011
Here is a copy of my keynote talk at the 2011 ISMA measurement conference, Measurement and Management and Business Value. The real point is the IT and measurement personnel have the information, knowledge and skill set to provide value to the business, far beyond the costs generate. But the information needs to be communicated in terms leaders can understand rather than the techie language we all think and speak in… AND if IT starts showing how it contributes to the business and becomes a profit center instead of a cost center IT will get more money to apply to more valuable things.
UPDATE: Someone also asked for a reference communicating the language of management. I believe this publication on business case analysis to be helpful in that regard.
PS In my talk I referred to software and measurement people as geeks. To many the term geek is a source of pride. One person in the talk was offended by being called a geek. My sincere apologies. However modern vernacular often defines a geek as a lover of technology, a software developer or someone with an intense love of mathematics. I consider myself a geek. And my friend Paul Glen makes his living by “leading geeks.”
Thank you for reading “Dan on Estimating”, if you would like more information about Galorath’s estimation models, please visit our contact page or call us at +1 310 414-3222.
Estimating Telecommuting Costs & Benefits
Today I received the results of a study on the benefits of telecommuting, prepared for Citrix, which offers remote collaboration technology. The paper discusses the many benefits of telecommuting for those jobs where telecommuting is feasible.
According to the paper, half-time telecommuting is the average for those who telecommute. The article points out potential productivity increases from telecommuting as well as a number of employee specific benefits such as time, gas, accident risks, etc.
It is interesting that they point out ”fewer interruptions” as a benefit. While I am sure some interruptions are non-productive, I have seen some great productivity increases when people are together and can discuss difficult problems “over the water cooler.”
SEER’s multi-site parameter has been proven over time for projects that are geographically diverse AND where people are not immediately accessible. Yet VoiP, Webex, Citrix, and other collaboration technologies have made great strides in bringing people together virtually. Businesses report losses of $600 billion annually in workplace distractions. But the study did not identify how much productivity gains from being together.
SEER-SEM Multi-Site Development sensitivity
I personally work at home for part of each day… answering emails and working on items where I need quiet time before I head to the office. And I do work from home all day a few times per month, usually when I have a paper due or some other deliverable.
Funny, as I was writing this blog post (from my home office…today is a telecommuting day) I received a call from our CFO. We are having a board meeting tomorrow. And I had planned to have conversations with the board group who will be in two time zones and pull in a person who is three time zones earlier… I ended up planning to have this gentleman on the phone at 5:30am…oops). That is really not so much a telecommuting issue as a true multiple site issue… It is more difficult to coordinate with people scattered across the globe.
Thank you for reading “Dan on Estimating”, if you would like more information about Galorath’s estimation models, please visit our contact page or call us at +1 310 414-3222.
Data Driven Estimating Part 1: How Data Feeds The Estimation Modeling Paradigm at Galorath / SEER
Lee Fischman’s article on data driven estimating follows:
Summary
- Galorath constantly collects data from many sources, both public and private.
- Data must be processed to be useful, using methods that Galorath routinely and openly discusses.
- In addition to data, numerous other sources play an important role in maintaining the accuracy of SEER for Software estimates.
- As important as data is, innovation is equally so. The core SEER model has been improved and extended over time, and new features have been added, to make estimating ever easier, more applicable, and more accurate.
- Data Driven Estimating Part 2: Data Driven Estimating Features in SEER for Software is available upon request.
What Does It Mean To Be Data Driven?
These days software estimation vendors are competing to have the largest repositories of completed software projects, and the customer is encouraging this competition, which is fundamentally good. However, there is more to insuring the accuracy of an estimation model than just having a lot of data points sitting on the proverbial shelf.
Where Data Comes From
The first question asked of a vendor is, where does your data on completed software projects come from? Early on, much of it came from Government agencies, who in turn collected from contractors. Over time, public sources have emerged that contain voluntarily submitted information from private companies worldwide; the prime example of this being the International Software Benchmark Standards Group (ISBSG). Galorath has obtained software project data over the years through numerous private and public sources. The data comprises many thousands of total observations that have passed data quality tests. Most observations contain size and effort information, thousands more do not contain all the desired fields.
What is Done with Data
While plain-vanilla data can reveal a lot, it has its limitations. For this reason, Galorath maintains extensive surveillance of industry trends, including third-party analyses. These can reveal insight into changes in modern practices such as Agile development, the productivity gained by the latest IDEs, and many other ongoing evolutions. The company is a member of numerous industry consortiums – in part to obtain access to the latest research available.
At Galorath, once data is acquired, it is processed into a form that is usable for analysis. This involves normalization so that the data points are comparable, i.e., include the same activities from early requirements through testing and the same types of labor, including programmers, testers, management, etc… We also try to find and understand outliers those projects that are so different that they are not useful. At numerous conferences and in webinars, we have described our normalization process and compared our results against other methods.
Using the collected data we update SEER for Software in several ways. A key method is to run our model against various stratifications (specific subsets such as Business and Client-Server) that are defined by SEER for Software’s knowledge bases. Simply put, we compare the models estimates to observed outcomes. Based on these results, knowledge bases are re-calibrated when necessary. In fact, data sets are not uniform in terms of the information observed. Some completed project records may include peak staff, development activities, and software language used, while others don’t. We account for this by performing a separate analysis of various factors: language productivity, development proportions, productivity variation by application or development method, and so on. These analyses are done first, and the model is adjusted, before gross analysis is begun.
SEER for Software’s core model is configured to a particular circumstance by a set of knowledge bases, and its these knowledge bases that are calibrated based on new industry information and trends. Each knowledge base is defined in terms of a set of parameters, some visible to users, and others normally hidden. When a knowledge base is updated the visible parameters, such as Modern Development Practices, may be modified and some underlying calibration factors may be adjusted. These knowledge base adjustments occur every few years as evidence warrants.
Innovative Features Support the Data Driven Approach
The overall evolution of SEER for Software is best called innovation-driven as well as data-driven. While data analysis is a very important part of how we maintain SEER for Software, we also continually enhance the model’s ability to estimate real world projects. These enhancements have often been industry firsts: flexible project staffing, off-the-shelf (COTS) integration modeling, translation of estimates into detailed project plans having intricate interdependencies, extended schedule and small schedule estimating, cloud computing solutions, to name only a few. All these innovations, alongside data-driven updates, serve an important role in insuring the model’s precision.
Note Data Driven Estimating Part 2: Data Driven Estimating Features in SEER for Software is available upon request via info@galorath.com Additional information may be found at www.galorath.com
Thank you for reading “Dan on Estimating”, if you would like more information about Galorath’s estimation models, please visit our contact page or call us at +1 310 414-3222.
Issues in Software Measurement & Estimation ISMA 2010
Measurement is a wonderful thing. However measurement without standards and definition can be worse than no measurement at all. This paper which I will be presenting at the 2010 ISMA conference begins the attack, highlighting the need and proposing that additional work commences in standards for estimation and measurement. Software Estimation and Measurement 2010
Thank you for reading “Dan on Estimating”, if you would like more information about Galorath’s estimation models, please visit our contact page or call us at +1 310 414-3222.
Project Prediction Infographic
