Here are the slides from the Galorath / David Consulting Group estimating clinic  held in New York City. There were interesting discussions encompassing dealing with impossible demands, understanding risk and risk management, sizing, and a number of other topics.  Note: The Galorath slides and the DCG slides are combined in the one PDF file.

Concepts of IT providing business value to the organization were also discussed in detail.

Thanks to all who attended and especially to Mike Harris and David Herren for their insights.

Related posts:

1. Galorath / Reifer Team To Provide Large Project Success Via Risk Assessments We are excited to announce a new addition to the...
2. Another View of Project Success From Dr. Dobbs While the Standish Group’s surveys are the most widely known...
3. The Business Value Of IT Book I opened my copy of this book, The Business Value of...

Lee Fischman’s article on data driven estimating follows:

Summary

• Galorath constantly collects data from many sources, both public and private.
• Data must be processed to be useful, using methods that Galorath routinely and openly discusses.
• In addition to data, numerous other sources play an important role in maintaining the accuracy of SEER for Software estimates.
• As important as data is, innovation is equally so.  The core SEER model has been improved and extended over time, and new features have been added, to make estimating ever easier, more applicable, and more accurate.
• Data Driven Estimating Part 2: Data Driven Estimating Features in SEER for Software is available upon request.

What Does It Mean To Be Data Driven?

These days software estimation vendors are competing to have the largest repositories of completed software projects, and the customer is encouraging this competition, which is fundamentally good.  However, there is more to insuring the accuracy of an estimation model than just having a lot of data points sitting on the proverbial shelf.

Where Data Comes From

The first question asked of a vendor is, where does your data on completed software projects come from?  Early on, much of it came from Government agencies, who in turn collected from contractors.  Over time, public sources have emerged that contain voluntarily submitted information from private companies worldwide; the prime example of this being the International Software Benchmark Standards Group (ISBSG).  Galorath has obtained software project data over the years through numerous private and public sources.  The data comprises many thousands of total observations that have passed data quality tests.  Most observations contain size and effort information, thousands more do not contain all the desired fields.

What is Done with Data

While plain-vanilla data can reveal a lot, it has its limitations.  For this reason, Galorath maintains extensive surveillance of industry trends, including third-party analyses.  These can reveal insight into changes in modern practices such as Agile development, the productivity gained by the latest IDEs, and many other ongoing evolutions.  The company is a member of numerous industry consortiums – in part to obtain access to the latest research available.

At Galorath, once data is acquired, it is processed into a form that is usable for analysis.  This involves normalization so that the data points are comparable, i.e., include the same activities from early requirements through testing and the same types of labor, including programmers, testers, management, etc… We also try to find and understand outliers those projects that are so different that they are not useful.  At numerous conferences and in webinars, we have described our normalization process and compared our results against other methods.

Using the collected data we update SEER for Software in several ways.  A key method is to run our model against various stratifications (specific subsets such as Business and Client-Server) that are defined by SEER for Software’s knowledge bases.  Simply put, we compare the models estimates to observed outcomes.  Based on these results, knowledge bases are re-calibrated when necessary.  In fact, data sets are not uniform in terms of the information observed.  Some completed project records may include peak staff, development activities, and software language used, while others don’t.  We account for this by performing a separate analysis of various factors: language productivity, development proportions, productivity variation by application or development method, and so on.  These analyses are done first, and the model is adjusted, before gross analysis is begun.

SEER for Software’s core model is configured to a particular circumstance by a set of knowledge bases, and its these knowledge bases that are calibrated based on new industry information and trends.  Each knowledge base is defined in terms of a set of parameters, some visible to users, and others normally hidden.  When a knowledge base is updated the visible parameters, such as Modern Development Practices, may be modified and some underlying calibration factors may be adjusted.  These knowledge base adjustments occur every few years as evidence warrants.

Innovative Features Support the Data Driven Approach

The overall evolution of SEER for Software is best called innovation-driven as well as data-driven.  While data analysis is a very important part of how we maintain SEER for Software, we also continually enhance the model’s ability to estimate real world projects.  These enhancements have often been industry firsts: flexible project staffing, off-the-shelf (COTS) integration modeling, translation of estimates into detailed project plans having intricate interdependencies, extended schedule and small schedule estimating, cloud computing solutions, to name only a few.  All these innovations, alongside data-driven updates, serve an important role in insuring the model’s precision.

Note Data Driven Estimating Part 2: Data Driven Estimating Features in SEER for Software is available upon request via info@galorath.com Additional information may be found at www.galorath.com

Related posts:

1. Software Estimating: Sources and Uses of Data and Data Driven Estimates Intro These days software estimation vendors are competing to have the...
2. Viable Software Estimation Modeling: A Key Component Of Software Risk Management I spoke with someone recently who explained the reason they...
3. Estimation Plus Process and Measurement Rigor Yields More Successful Projects: SEER and Tracer It has been rewarding working with the Computer Aid  (CAI)...

Dan Galorath briefed the Galorath estimate maturity model, a method of evaluating where the organization is in estimation practice and where it wants / needs to go.

This paper discusses poor estimates and their impact on projects as well as how to improve estimate maturity as a means to more successful projects.

The presentation concludes:

• Estimation and estimation process are core for successful software projects
• Improving estimate maturity can improve your project success rate
• Gartner says even SEER use improves estimate vs. actual variance by 50%
• A good guess is not a substitute for a viable estimate
• Looking at total ownership cost can change project prospective
• Much of the industry is at level 1 in estimation maturity
• Process initiatives like CMMI require:
• Estimation & planning, monitoring & control, measurement & analysis
• IT Infrastructure and IT Services are significant costs and must be estimated in addition to software development and maintenance

Related posts:

1. CECIM: A Capability Improvement Model For Cost Engineering Being a major fan of repeatable process for cost and...
2. Many Observed Incorrect Implementations of CMMI High Maturity At the CMMI conference a number of difficulties with high...
3. Another View of Project Success From Dr. Dobbs While the Standish Group’s surveys are the most widely known...

Dan will be speaking at the Chicago SPIN on November 12, 2009 on the topic of controlling software projects.  Estimation, planning, control, metrics, and maintenance for a total ownership cost view will be discussed.

The presentation is here: Chicago SPIN November 2009 Galorath Presentation Controlling Software Projects Development Is Only Job 1

PS Dan looks forward to his short visit to Chicago, his home town.  And is going to carefully avoid pizza, hot dogs, and Italian beef while he is there.

The flyer follows:

Chicago Software Process Improvement Network

(C-SPIN) Meeting

AT&T Center Campus

AT&T Institute 2501 W. Eagle Way at

Lakewood Blvd and Eagle Way, Hoffman Estates, IL 60192

Be sure to check the C-SPIN website (http://c-spin.net/) to confirm details.

SPECIAL DATE: Thursday, November 12, 2009

6:00 – 7:00 PM – Atrium: Registration, Networking, & Light Snacks

6:20 – 6:40 PM – Birds-of-a-Feather Topic – TBD

7:00 – 8:30 PM – Auditorium: Presentation

8:30 – 9:00 PM – Auditorium: Additional Q&A and discussion

Abstract:

Planning software development projects is never easy. Requirements, time-to-market, architectural and quality issues, staffing, risks and many other factors must be carefully considered. Development tradeoffs impact software maintainability andcan significantly impact total cost of ownership. Many factors other complicate software planning.  Software planners tend to underestimate the degree to which code degrades over time. Software development costs are only a portion of the total cost of ownership. Software maintenance can often be up to four times the initial development cost.

This session will provide a systematic approach to addressing total cost of ownership across the software lifecycle, including design for maintainability, development of measurement criteria, collection of metrics, and industry standards, guidelines, and best practice options. Parametric modeling will be discussed using the SEER platform as a specific example. Estimating block changes and their potential interdependencies and impacts will also be covered.

Biography

Over a period of three decades, Dan Galorath has established a reputation as a leading expert in project estimation, analysis, management and control. He has literally written the book on the subject (Software Sizing, Estimation and Risk Management, Auerbach Publishing). Dan realized early that there was a better way to develop accurate project estimates. Today SEER ® analysis tools from Galorath Inc. are saving millions of dollars for organizations worldwide. Galorath’s customer base reads like a who’s who of American and European industry: Airbus, Bank of America, Boeing, Ford Motor Company, Lockheed Martin, NASA, Raytheon, Siemens, and the U.S. Army, Navy, and Air Force, etc.

Dan lectures internationally and has published numerous articles and papers on software cost modeling, testing, software lifecycle error prediction and requirements definition. Dan was named winner of the 2001 International Society of Parametric Analysts (ISPA) Freiman Award presented to individuals who have made outstanding contributions to the theoretical or applied aspects of parametric modeling. Dan also received the 2009 lifetime achievement award for his contributions to estimation, planning and control from The Society of Cost Estimation and Analysis (SCEA)

Driving and Parking Directions

From I-90 Northwest Tollway, exit North on Barrington Road; take Barrington Road to Lakewood Blvd (2nd light) and turn East (right) onto Lakewood Blvd.  Turn right into the AT&T Campus Center’s West Employee Entrance (the first entrance past Eagle Way) and follow the signs to the West Parking Structures.  Parking in covered lot W3 is recommended.  Look for parking spaces near the west entrance/exit of parking structure W3. Then walk west from the parking structure to the Institute Building to the main door, which has a big ENTRANCE sign over the door.  The Institute is approximately 100 yards from the parking structure–back across the road you came on.  Visit http://www.c-spin.net for a map of the AT&T Campus Center.  (Additional parking is available in covered lot W2 as well as upper level parking lot W1.)

About C-SPIN

C-SPIN is a leadership forum for the free and open exchange of software process improvement experiences and practical ideas. We promote achieving higher levels of process maturity, software quality, and mutual respect. Companies, academic institutions, government organizations, and individuals are invited. For information about C-SPIN or the steering committee, contact Karen Mermel at kmermel@yahoo.com.

To receive future announcements electronically, register your e-mail address (include name, personal email address, company, and phone number) http://c-spin.net/newsletter/

Related posts:

1. Software Measurement and Total Ownership Costs… Dan’s Presentation at the NYC SPIN I had the opportunity to present at the New York...
2. Dan Galorath Presentation at SEER / Tracer Seminar In Tysons Corner October 2009 It was exciting to participate in a seminar with Computer...
3. Optimizing Development Projects With Multi-Vendor Approaches Using multiple vendors to develop software, each with their own...

I was asked today for a SEER-SEM 10 Step estimate review checklist.  Of course there are checklists in my book:

105 – 107; Step Seven: Estimate Validation and Review

108 – 109; Estimate Review Activities

117 – 128; Estimate Process Questionnaire

432 – 434; SEER-SEM Estimation Process Step 7: Review, Verify and Validate Estimate

And a blog that includes some of the checklists

There are also a number of other useful checklists:

• Bob Park, while with the SEI, developed this checklist.  It remains useful even today.
• Galorath has an estimate assessment document based on a prerelease of the 10 Step book.
• Galorath has an additional document available to SEER users upon request.

Related posts:

1. Step Seven: Estimate Validation and Review Step Seven: Estimate Validation and Review At this point in...
2. Step One: Establish Estimate Scope and Purpose Establish Estimate Scope and Purpose Define and document expectations. When...
3. Step Nine: Document Estimate and Lessons Learned Each time you complete an estimate and again at the...

It has been rewarding working with the Computer Aid  (CAI) people, the people who sponsor the ITMPI in tying together best practice approaches (SEER for Estimating, Planning and Control and Computer Aid’s Tracer for metrics, reporting and root level management).  I am always inspired when I hear the  CAI people explain their process and success in fixed price software development and maintenance.  They do it by a number of methods and processes.  But the one that always makes me the most excited is their Tracer software.  Tracer allows root level tracking of tasks and progress, providing near real time feedback regarding project health and status and individual task performance.  I think it is wonderful that they have made their solution available to the industry at large.   It is much like the factory floor real time feedback in manufacturing, applied to software.  From CAI’s web site is the following description which describes estimating effort, cost, and process tracking.:

The SEER Tracer alliance is a fusion of Galorath’s SEER for Software application suite – the world’s premier project estimation and planning application – and CAI’s Tracer application – the world’s premier operational IT control device. SEER optimizes project estimates and project planning while Tracer tracks real time effort and progress and feeds such operational data back into SEER for project tracking and forecasting.

About SEER:

 SEER is a complete CMMI Project Management solution that establishes estimates and automatically generates project plans based on those estimates, monitors projects against the plan, and aligns measurement and analysis activities. SEER for Software uses a combination of sizing metrics and techniques, parametric algorithms, knowledge bases (industry data), simulation-based probability analysis techniques, and user supplied data to estimate and analyze the development and maintenance of software projects with respect to effort, cost, schedule, staffing, risk (probability of success), and reliability (defects).

SEER provides a systematic approach for estimating the resources and scheduling that software development requires, and an unparalleled capability for trade-off and risk analysis.

About Tracer:

 Tracer was designed for use within the day-to-day IT operating work environment of IT Application Support personnel or Application Development teams with low administrative overhead. Tracer’s table driven design for all major components helps focus IT professionals on “work” not “data entry.” The flexibility of Tracer’s configuration tables allows easy adaptability to virtually any IT operating environment. The real-time customizable dashboards display all assigned work tasks to developers and support personnel, using color coding to highlight tasks approaching or exceeding service level goal thresholds. Resource/Task time tracking is integrated within all work processes and brings a dramatic increase in the accuracy of time and work related data reporting. Accurate and timely data collection and reporting are necessary to effectively manage the IT organizations work and resources.

Many IT software tools have some of the components, functions and features that are standard with Tracer. However, only Tracer has all these critical IT process management and work tracking components built-in:

• Defined process templates for application support and development
• Process enforcement based on configurable process rules
• Work, resource and time tracking
• Managing by Metrics – SLA based metrics graphs, trends and work / resource status reporting

About SEER / Tracer

• SEER / Tracer helps you estimate the effort and cost of IT and/or Software projects and their ongoing support
• SEER / Tracer helps you build realistic project plans, increasing the probability of project success
• SEER / Tracer drives a standardized and repeatable process, from initial requirements through development and maintenance
• SEER / Tracer functions seamlessly within a single integrated operating environment and historical data repository
• SEER / Tracer provides a comprehensive framework for planning, monitoring, control, and compliance and a comprehensive system for IT project governance

 The SEER Tracer Alliance represents a unique end to end solution that institutionalizes best practices throughout the entire IT project lifecycle, from requirements gathering, through estimation, project planning, measurement and analysis, development, monitoring/control, and maintenance.

 The SEER Tracer alliance is a fusion of Galorath’s SEER for Software application suite – the world’s premier project estimation and planning application – and CAI’s Tracer application – the world’s premier operational IT control device. SEER optimizes project estimates and project planning while Tracer tracks real time effort and progress and feeds such operational data back into SEER for project tracking and forecasting.

Related posts:

1. Process Compliance Saves 15% of Development or Maintenance Costs I had the opportunity to hear a brief on Computer...
2. Dan Galorath Presentation at SEER / Tracer Seminar In Tysons Corner October 2009 It was exciting to participate in a seminar with Computer...
3. Ten-step Project Estimation Process Software Estimation Concepts Many project managers and project management offices...

It appears that this is the final version of the GAO cost estimating guide which provides guidance on preparing viable cost estimates both early in the process and throughout the life cycle. Congratulations to the team.  This is a great contribution to the industry and can, if used as intended create more successful projects.  I especially appreciate the focus on preparing a viable estimate of cost, schedule, etc. then applying earned value management (EVM) to that. So often, in the past we have seen a chasm between those that generated estimates and the EVM people.  These are two sides of the same coin.  The introduction follows:

Because federal guidelines are limited on processes, procedures, and practices for ensuring credible cost estimates, the Cost Guide is intended to fill that gap. Its purpose is twofold—to address generally accepted best practices for ensuring credible program cost estimates (applicable across government and industry) and to provide a detailed link between cost estimating and EVM. Providing that link is especially critical, because it demonstrates how both elements are needed for setting realistic program baselines and managing risk. As a result, government managers and auditors should find in the Cost Guide principles to guide them as they assess (1) the credibility of a program’s cost estimate for budget and decision making purposes and (2) the program’s status using EVM. Throughout this guide, we refer to program cost estimates that encompass major system acquisitions, as well as government in-house development efforts for which a cost estimate must be developed to support a budget request. The basic information in the Cost Guide includes the purpose, scope, and schedule of a cost estimate; a technical baseline description; a work breakdown structure (WBS); ground rules and assumptions; how to collect data; estimation methodologies; software cost estimating; sensitivity and risk analysis; validating a cost estimate; documenting and briefing results; updating estimates with actual costs; EVM; and the composition of a competent cost estimating team.6 The guide discusses pitfalls associated with cost estimating and EVM that can lead government agencies to accept unrealistic budget requests—as when risks are embedded in an otherwise logical approach to estimating costs. Since the Department of Defense (DOD) is considered the leader in government cost estimating, the guide relies heavily on DOD for terminology and examples that may not be used by, or even apply to, other federal agencies. Chapters 1–17 of the Cost Guide discuss the importance of cost estimating and best practices associated with creating credible cost estimates. They describe how cost estimates predict, analyze, and evaluate a program’s cost and schedule and serve as a critical program control planning tool. Once cost estimates have been presented to and approved by management, the chapters also establish the basis for measuring actual performance against the approved baseline plan using an EVM system. Those chapters explain how EVM, if it is to work, must have a cost estimate that identifies the effort that is needed—the work breakdown structure—and the period of time over which the work is to be performed—the program schedule.7 In essence, the cost estimate is the basis for establishing the program’s detailed schedule, and it identifies the bounds for how much program costs can be expected to vary, depending on the uncertainty analysis. When all these tasks are complete, the cost estimate can be used to lay the foundation for the performance measurement baseline (PMB), which will measure actual program performance. Since sound acquisition management requires more than just a reliable cost estimate at a project’s outset, chapters 18–20 provide guidance on converting the cost estimate into an executable program and a means for managing program costs. Our program assessments have too often revealed that not integrating cost estimation, system development oversight, and risk management—three key disciplines, interrelated and essential to effective acquisition management—has resulted in programs costing more than planned and delivering less than promised. Therefore, chapters 18–20 address best practices in implementing and integrating these disciplines and using them to manage costs throughout the life of a program. OMB has set the expectation that programs will maintain current estimates of cost. This requires rigorous performance-based program management, which can be satisfied with EVM. Chapters 18–20 address the details of EVM, which is designed to integrate cost estimation, system development oversight, and risk management. Additionally, for programs classified as major acquisitions—regardless of whether the development work is completed in-house or under contract—the use of EVM is a requirement for development, as specified by OMB.8 The government may also require the use of EVM for other acquisitions, in accordance with agency procedures. Since linking cost estimating and EVM results in a better view of a program and allows for greater understanding of program risks, cost estimators and EVM analysts who join forces can use each other’s data to update program costs and examine differences between estimated and actual costs. This way, scope changes, risks, and other opportunities can be presented to management in time to plan for and mitigate their impact. In addition, program status can be compared to historical data to better understand variances. Finally, cost estimators can help EVM analysts calculate a cumulative probability distribution to determine the level of confidence in the baseline. But bringing a program to successful completion requires knowing potential risks and identifying ways to respond to them before they happen—using risk management to identify, mitigate, and assign resources to manage risks so that their impact can be minimized. This requires the support of many program management and engineering staff and it results in better performance and more reliable predictions of program outcomes. By integrating EVM data and risk management, program managers can develop current estimates at completion (EAC) for all levels of management, including OMB reporting requirements. Therefore, chapters 18–20 expand on these concepts by examining program cost planning, execution, and updating.

Related posts:

1. GAO Cost Estimating and Assessment Guide Released Congratulations to the GAO team for releasing the “Cost Estimating...
2. GAO Cost Assessment Guidebook The GAO cost assessment Guidebook is something you should keep...
3. Galorath Article “Applying Earned Value Management To Software Intensive Programs Part 2 For paper with figures see Applying Earned Value Management To...

I will be speaking on estimating process yielding project success at the Computer Aid Seminar in Detroit On April 30, 2009.

My message will be that spending time on estimation and planning yields more successful projects and fewer project surprises.  And I will go through the 10 step process.  Additionally I will introduce Galorath’s estimation maturity model.

I am excited to speak along with Ed Yourdon, Larry Dribin, and CAI’s Bob Longhorn.

It should be a great event.  I have participated in these with Larry and Bob in the past, and I am a lifetime fan of Ed.

I believe we will be doing the same genre in Toronto later in the year.

Related posts:

1. Dan Galorath Presentation at SEER / Tracer Seminar In Tysons Corner October 2009 It was exciting to participate in a seminar with Computer...
2. SEER Users Seminar In EL Segundo, October 8, 2009 I just had the opportunity to review the agenda for...
3. Controlling Software Projects: Development Is Only Job One: Chicago SPIN Nov 12 2009 Dan will be speaking at the Chicago SPIN on November...

I have had a good time developing a webinar with Geoff Hewson of the Software Productivity Center.  It should be very informative and hopefully entertaining as well.

Register by clicking

Date and time: Tuesday, March 10, 2009 8:30 am
Pacific Daylight Time (GMT -07:00, San Francisco)

Description: Software estimation, planning and control can be keystones to successful projects. Companies that estimate strategically are able to zero in their projects so they deliver to business targets. Despite this many treat estimation as a black art or worse – a guess.

Dan Galorath and Geoff Hewson will explain the key concepts that drive successful software estimation in support of planning and managing successful software development and maintenance projects:
1. Decisions and deliverables needed to design estimation procedures tuned to your organization.
2. Guidance in setting up the infrastructure you need to realize the most benefit from your new estimation capabilities.
3. Establishing a “negotiating culture” to allow you to deal effectively with the results of your estimates and focus projects on business success.

SEER for Software concepts will support the discussion.

Related posts:

1. Galorath / DCG IT Estimation For Business Value and Project Success Clinic Here are the slides from the Galorath / David Consulting...
2. The Future of Software Analysis Measurement Webinar Featuring Bill Curtis, David Herron and Dan Galorath We have some exciting events coming up this month with...
3. Ten-step Project Estimation Process Software Estimation Concepts Many project managers and project management offices...

How many times have we been asked to submit a 5 minute or off the cuff estimate to stakeholders… Then be held to that estimate even though we missed the complexity in our haste.. Of course if we didn’t prepare an estimate until everything was known the estimate would be of little value.  That is why uncertainty must be part of the process and off the cuff, manual estimations should be avoided when possible.

I thought these diagrams showing the perceived simplicity of the software estimation process versus complexities in the software cost estimation process were very interesting.

Perceived (Classical) Source: Vigder/Kark

More Real World Cost Estimation Process: Source Vigder/Kark

Related posts:

1. Ten-step Project Estimation Process Software Estimation Concepts Many project managers and project management offices...
2. Book “Software Sizing, Estimation & Risk Management” Is Now Available Electronically Dan’s book, Software Sizing, Estimation & Risk Management, is now...
3. 10 Deadly Sins of Software Estimation Whenever I open my book and see the quotes from Steve...