Fool the Guesser – Applying Reference Class Forecasting at the State Fair

August 21, 2014 · Filed Under Estimating, General · Comment 

credit: tmnews.com

My favorite thing to do at the state fair is eating smoked turkey legs. I normally do this while walking around the various booths and barkers listening to pitches for the newest ShamWow® or Veg-o-Matic®. This year I found myself standing in front of a large red and white weighing scale.

“Guess your age, guess your weight – or you win a prize, How about you sir – you’re a handsome young fellow,” shouted the colorful bald headed barker.

I declined, but found myself sitting on a nearby bench – turkey in hand –enjoying the fellow estimator.   For a mere five-dollars he would promise to guess your age within two years, your birth month within three months or your weight within two pounds – or you won your choice of many wonderful prizes.   I watched numerous people walk away with soft fuzzy animals, but amazingly the “Guesser” was close to 80-85% accurate. “How does he do that,” I asked myself.

Naturally, I quietly tried to spontaneously guess the weights and ages myself, but I was not nearly as accurate as the “Guesser.” This reminded me of the countless high level or Rough Order Magnitude (ROM) cost estimates I’ve witnessed with an even worse record using spontaneous guessing.  At one organization, their guess was an over/under ratio of an amazing +100%/-300%. They were taking millions of dollars away from other potential projects “just in case.”

I watched the “Guesser” and was surprised at how much time he spent talking with the “rube.” I couldn’t understand why he didn’t just guess right or wrong and then move them along – after all – they were paying five-dollars for a prize that cost under a buck.   It was a pure profit deal. Turns out, he really wanted to get it right! Like all estimators, we want to think we are pretty good at our jobs.   So he took his time and applied Reference Class Forecasting, although he had a different name for it.

In 2002 Daniel Kahnemann shared the Nobel Prize for Economics “for having integrated insights from psychological research into economic science, especially concerning human judgment and decision-making under uncertainty.”1 A few years ago I read his best-selling book, Thinking Fast and Slow.2 In this text Kanhemann describes how planning typically falls into one of two perspectives – an Inside View and an Outside View.

Those that prepare estimates using an “Inside View” quickly focus on specific circumstances and search for evidence using their own experiences (“Expert Judgment?”). This is a significant contributor to what he calls a “Planning Fallacy” (a bad estimate). For example, I am responsible for cooking the holiday turkey every year. I‘ve done this for many years when I lived in Florida that I simply used the same time and temperature when I moved high up into the Blue Mountains. My “Inside View” made my cooking time and temperature estimate woefully optimistic. Dinner was late!

What I should have done was use an “Outside View.” Kahnemann explains that preparing an estimate using an Outside View requires the estimator to use a reference class; some empirical evidence from a similar activity. This use of the Outside View has become known as “Reference Class Forecasting.” For an IT estimate, this Outside View may take the form of a large/international database with plans and outcomes from a myriad of projects – provided they can provide statistical information about the likely outcome. The selection of a reference class is simple: (paraphrased from Thinking Fast and Slow):

  1. Identify an appropriate reference class (IT projects, painting a house, building a race car, etc.).
  2. Obtain the statistics of the reference class (in terms of number of servers, square meters of walls, go-cart or formula-one race car). Statistics are used to generate a baseline estimate.
  3. Use specific information about the case to adjust the baseline prediction, if there are particular reasons to expect an optimistic bias to be more or less pronounced in this project than in others of the same type.

In applying Reference Class Forecasting at the state fair I started to understand how the “Guesser” was so good. He was using his internal database – built from countless years on the job, and selected the proper class for the individual he was attempting to keep from a prize. I started to do the same. Instead of quickly looking at the face or girth I considered other factors, braces, athletic wear, color of hair (for men), jewelry, companions, age-spots, and so on and so forth. In a very short period of time I was getting nearly as close, and a few times, even more accurate estimates than the Guesser.

Unfortunately, the “Inside View” is generally the first choice selected when making an estimate. There is ample evidence3 that reveals that most estimators (of anything) are overly optimistic of their capabilities and hence make bad estimates – in other words they are comfortable with their “Inside View.” The “Guesser” was unknowingly using an “Outside View” with each person he sized up. He considered his vast mental database of classes, selected the proper category, and challenged any biases (i.e., a woman with an apparently young man does not mean she’s a young woman).

With my turkey leg long gone I took the opportunity to ask the “Guesser” a few questions. He told me he had been on the carnival and state fair circuit for over twenty years and had easily guessed the age or weight of over ten-thousand people.

When I asked him his secret he answered, “Like I said – I’ve been doing this for over twenty years and easily over 10,000 people. Ya learn how to make your hunches.”

That is quite a reference database he has up in that bald head and I have a hunch I’ll use an Outside View when I’m roasting turkey at altitude.

 

David W. DeWitt
Director of Software and IT Consulting, Galorath Incorporated

 

  1. Daniel Kahneman – Facts, http://www.nobelprize.org/nobel_prizes/economic-sciences/laureates/2002/kahneman-facts.html
  2. Daniel Kahneman (25 October 2011). Thinking, Fast and Slow. Macmillan. ISBN 978-1-4299-6935-2.
  3. Optimism: The Enemy Of Successful Acquisitions, Dan Galorath on Estimating, http://www.galorath.com/wp/optimism-the-enemy-of-successful-acquisitions.php

Thank you for reading “Dan on Estimating”, if you would like more information about Galorath’s estimation models, please visit our contact page, call us at +1 310 414-3222 or click a button below to ask sales questions, sign up for our free library or schedule a demo.




Can You Really Trust Your Project Plan to Tell You the Truth, Even If You’re Already Knee-Deep in the Project?

June 13, 2014 · Filed Under Project Management, Risk · Comment 

Find Renewed Confidence In Your Project by Implementing the Best Project Estimation Software

So you dove head first into a new project that seemed to be an excellent next step for your business. You and your team had a tremendous amount of enthusiasm for the project at the start, but just a basic idea of your production costs, labor, materials, tooling, set-up, and rework numbers. Now that you are knee-deep in the project, your team and the stakeholders’ enthusiasm for the project is rapidly waning because you are not hitting the numbers you expected, and unanticipated problems seem to be cropping up left and right.

best project estimation software

What to do? Perhaps the answer is a new project estimation system. But even when you identify the best project estimation software for your industry, can you depend on its output to give you feedback that’s valid?
The answer is yes, the best project estimation software will give you honest and thorough answers about your project. You might be at a point of panic because you know you’re already months into a project, or you might still be just considering the feasibility of the project, two different emotional states that the software is immune to – all it knows to do is weigh inputs and history, and report back impartially.

Here are four big reasons why now is a good time to consider employing project planning software:
1. It Helps You Build a Complete Picture of the Project

The best project estimation software available allows you to build an entire manufacturing project plan from templates, or from scratch. Its user-friendly interface allows for quick and easy entry of information regarding every single detail of the project, including progress already made, if applicable. It asks intuitive questions to help you consider possible alternative materials, tooling, manpower, and more to plan for the best possible outcomes.

 

2. It Serves as Your Virtual Expert

A built-in and constantly updated library of industry-specific knowledge will help you anticipate more potential stumbling blocks that could come your way, which will help you to plan accordingly for continued successful progress. Sector-specific models derived from extensive project histories, behavioral models, and metrics give you the opportunity to make unlimited hypothetical trade-offs to analyze endless scenarios in order to pinpoint the best plan of action to ensure good outcomes for the project. Pop-up windows ask intuitive questions to guide you through the process of defining the project’s scope, complexity, and technologies. Software at this level should pose questions that may bring topics, options, or potential problems to light that your team never even considered.

 

3. Its Cost Analysis Capabilities Makes Budgeting a Breeze

Top-notch project estimation software takes into consideration costs already incurred, and runs it against the project’s full budget. It can quickly and accurately generate labor and machine cost rates for use in determining facilities costs on a global scale. It contains a regularly updated, comprehensive database which includes worldwide labor cost data, default technology costs, average costs for many regions (including low-cost centers), default labor costs, and regional currencies to make your budgeting tasks so much easier. The best project estimation software on the market right now makes it easy to produce accurate figures and estimates, and to actually meet those numbers with a solid management plan.

 

4. It Offers a Variety of Reporting Options

Customizable charts, graphs, and reports allow you to visually present project outcomes and alternatives, as well as work-in-progress. Analyzing these reports can help you identify weak points in your current manufacturing systems and assets, and determine better alternatives to improve quality and output. You may run numerous simulations and generate reports for each scenario to compare outcomes and decide on the best assets and systems for your project. The effects of your improvements will be evident when you compare reports from before and after the change over to new planning software. Your new project estimation software should also allow users to export all of this information to other software programs for even wider reporting capabilities.
To tell you the truth about a project’s status or feasibility, regardless of what stage of the project you are in, the best project estimation software combines traditional project estimating software and cost estimating software solutions into one seamless package. Learning the truth about your project may sting a little, but in the end, you need to know all of your viable options – and you want to give your stakeholders the best, most accurate information possible. The best project estimation software can tell you the optimal way to complete the project, or whether the project should be scrapped.

 

Want to get a live demo of what our clients say is the best project estimation software they’ve ever used? Click here or call 310-414-3222 in the U.S. or +44 (0) 207 788 9042 internationally.

Thank you for reading “Dan on Estimating”, if you would like more information about Galorath’s estimation models, please visit our contact page, call us at +1 310 414-3222 or click a button below to ask sales questions, sign up for our free library or schedule a demo.




Managing and Eliminating Estimation Bias

thinking manI have been studying the art and science of estimating for many decades now.  Some of the observations I have made include:

  1. Most people don’t like to estimate
  2. Most people don’t know how to estimate
  3. Those that estimate are almost always wildly optimistic
  4. Viable estimates can make projects successful, outsourcing more cost effective, and help businesses make the most informed decisions

That is why models like SEER are essential to organizations, providing the tempering with that outside view of reality that is recommended by Daniel Kahneman.  Dr. Kehnaman received a Nobel prize for his work in estimation bias and strategic mis-estimation.   This briefing covering estimation bias reviews some of the work of Dr. Kehnaman and others in helping stomp out bad estimates.  I believe this is scheduled to be a webinar in the near future.  Stand by.  Here is the presentation: Managing and Reducing Estimation Bias and Strategic Estimation

Thank you for reading “Dan on Estimating”, if you would like more information about Galorath’s estimation models, please visit our contact page, call us at +1 310 414-3222 or click a button below to ask sales questions, sign up for our free library or schedule a demo.




How to Shun Public Mortification by Setting Realistic Project Expectations

May 15, 2014 · Filed Under Estimating, Project Management, Risk · Comment 
Dollarphotoclub_58650737

The key is to set realistic expectations.

Use these tips to avoid project estimation disasters.

Grossly over-budget. Famously broken. Publicly acclaimed as sketchy. Most certainly this is not how any sane software developer, project manager, or estimator wants their next project to be classified. In the face of sky-high hopes, tremendous political pressure, and a nation waiting to see how the pieces all fall into place, perhaps one of the first ways many projects veer off-track is in a failure to assess and set realistic expectations.

Whether you’ll report to an internal client, a buyer you’ve hotly pursued, or even an entire nation, learning to manage expectations in advance is a critical skill that can preserve a delicate relationship.

Here are 7 tips to help you provide confident assurance that your customer or stakeholders’ needs will be met.

1.    Know your limits, or risk overstepping them.
What is your bandwidth? How much time can you allot to the project with your existing resources? What stream of preexisting obligations, snags, and demands might overflow the banks and flood this new project before it even starts?

2.    Balance promises with reality.
While the adage “under-promise and over-deliver” has its share of pitfalls in that doing so in the extreme guarantees your competitor will win the project by being on-target with promises and delivery, it is still better than its opposite, where you make lavish promises and paltry deliveries.

3.    Disclose limits beforehand to prevent trouble escalation.
Most project obstacles rear their heads with some measure of predictability. The sound of the train precedes its arrival with at least a bit of warning. Rather than pretending the path ahead is irreversibly smooth, be upfront with your end customer about potential pitfalls before the project begins. This helps you avoid wasting time in damage-control mode later when one of these blockades pops up.

4.    Allow for wiggle room.
When possible in your negotiations, adopt the time urgency of your customer while also demonstrating your understanding of the need for the project to be done right. The phrase “Done Fast, Done Cheap, Done Right… Choose Two” is simplistic but true. In most projects, one of these goals must be sacrificed in order to deliver the other two. Disappointment can be avoided when time, budget, and quality realities are transparent from the start.

5.    Keep the lines open.
Sometimes a project reaches a fork in the road with no clear indicator of which way to turn. Uncertainty can freeze progress in its tracks, and can be made worse when there’s a sense the team “should have known” the answer to the perplexing dilemma and rather than coming clean with it to the customer, they send a hurried scout down each path to see which way to go. Don’t allow uncertainty to cause your project to come to a grinding halt. Before the project begins, explore various “what-ifs”, alternatives and trade-offs – this will keep forward momentum safe and ease potential constriction of communication lines.

6.    In the loop is always preferable to in the dark.
It’s a sure bet your customer has someone breathing down their collective necks as impatiently as a first-grader on a long car drive. “Are we there YET?” is the constant refrain, and it needs to be answered with a credible answer to avoid small-scale rioting. If the timeline morphs mid-project, prompt and honest communication is the best policy. Agreeing to making this sort of update if needed affords your customers the intel needed to manage expectations and communicate intelligently.

7.    Present projections backed by history.
Every project you undertake is both unique and yet based on precedent. Ideally, your organization documents processes, timelines, and the progression of projects. This enables you to learn from history, to plan well for new opportunities, and to more firmly tether promises made by the sales department with the realities of operations. The documentation process yields information that can be aggregated into a sort of internal knowledge base that can be used to create and sanity-check estimates for future projects.

While humans are naturally wired to be optimistic, the hope that everything will go as planned must often be tempered with caution – especially when setting expectations with stakeholders. With realistic expectations in place from the start, before any promises are made, the likelihood of thrilling your customers is much higher – and the risk of a crash and burn disaster in the public eye shrinks.

Aggregating historical project data to create realistic expectations bears its own set of challenges, especially in the absence of access to directly-relevant prior history. SEER project estimation software features continually updated industry-specific knowledge bases which allow users to glean reliable insights from other projects as they plan for their own.

For more information on how project estimation software could be your secret weapon, download our whitepaper “On Time, On Schedule, and a Huge Success”.

Thank you for reading “Dan on Estimating”, if you would like more information about Galorath’s estimation models, please visit our contact page, call us at +1 310 414-3222 or click a button below to ask sales questions, sign up for our free library or schedule a demo.




SEER Composites Brings Cost To Designers Using CATIA

March 20, 2014 · Filed Under Design for Manufacturing Estimating, Estimating, General · Comment 

Galorath and Dassault Systems showed the opportunities to reduce cost and make the best design decisions via the integration of SEER and CATIA (Dassault’s widely used CAD System) at the Paris  JEC show.  Visitors were excited by the ability to view a composites part within CATIA and with a few clicks get an analysis of the costs of the alternate. This allows designers to make the most cost effective decisions and makes cost an engineering variable.   A story of this solutions was discussed on several engineering web sites such as engineering.com.  Dassuult’s CEO

Philippe Laufer CEO of CATIA also says:

 “This [the SEER Composites integration] leads to finding out the most efficient way of manufacturing a product while meeting cost, performance, functionality, and appearance requirements.”

 

 

Thank you for reading “Dan on Estimating”, if you would like more information about Galorath’s estimation models, please visit our contact page, call us at +1 310 414-3222 or click a button below to ask sales questions, sign up for our free library or schedule a demo.




Removing Estimation Bias: ICEAA Keynote March 2014

March 18, 2014 · Filed Under Conferences, Estimating, General · Comment 

Estimation is challenging.  And humans appear to be hardwired to be optimists.  THis briefing covers some of the work in removing estimate bias and strategic misrepresentation. Additionally it illustrates some of the ways SEER models and models in general can help.  ICEAA Removing Estimation Bias.  Key points include:

Experts are likely providing biased estimates

Poor estimates are a root cause of project failure

Estimates can be better, squelching bias & strategic misestimation… Parametrics help.

 

Thank you for reading “Dan on Estimating”, if you would like more information about Galorath’s estimation models, please visit our contact page, call us at +1 310 414-3222 or click a button below to ask sales questions, sign up for our free library or schedule a demo.




Galorath / Dassault Advance the State of the Practice on Composites

March 4, 2014 · Filed Under Design for Manufacturing Estimating, Estimating, General · Comment 

Source: CompositesWorld

Galorath Inc. (Los Angeles, Calif., USA) reported on Feb. 25 that it has reached an agreement with Dassault Systèmes to integrate Galorath’s SEER for Manufacturing cost-estimating software with Dassault’s CATIA product design software. Galorath says the inclusion of SEER’s Cost Estimator in CATIA will allow manufacturers to initially model and test composite manufacturing processes and tradeoffs during the earliest and most preliminary stages of design.

catia to seer

Galorath’s SEER software has been assisting large-scale project estimation and planning for 25 years. Galorath says a key feature of SEER is that it is based on parametric modeling and incorporates industry data and best practices. Using analysis and simulation, SEER helps companies estimate and control project costs, risks, quality and duration.

“We know that many of our SEER for Manufacturing users, particularly in aerospace, use both CATIA and SEER,” says Dan Galorath, CEO of Galorath Inc. “With the creation of SEER’s Cost Estimator in CATIA, designers will be able to get a detailed cost estimate of a composite design. As the design changes shape, size, or materials, new estimates can be quickly rerun to understand the cost impact of those changes. Not only does it reduce duplicate effort of re-entering data into cost models, users will know in advance what it will cost to produce that part.”

“Dassault Systemes 3D EXPERIENCE platform connects data and people. Offering Galorath’s unparalleled, cost-estimating capabilities within CATIA 3DEXPERIENCE Composites processes helps users effectively predict the cost of composite use in product manufacturing and is another demonstration of the power of the 3DEXPERIENCE platform,” says Philippe Laufer, CEO CATIA, Dassault Systemes. “Using Galorath’s SEER for Manufacturing within CATIA 3DEXPERIENCE processes will help companies perform early trade-off analysis on the use of various materials and composites processes before manufacturing even takes place. This leads to finding out the most efficient way of manufacturing a product while meeting cost, performance, functionality and appearance requirements,” added Laufer.

This interface is being delivered initially for CATIA V5 and will allow users to create and save multiple estimate scenarios for each part to compare and trade options. It includes fully customizable knowledge base templates and rules to create detail composite part cost estimates for labor, material and tooling costs.

http://www.compositesworld.com/news/galorath-cost-estimating-software-integrated-into-dassault-systmes-platform

Thank you for reading “Dan on Estimating”, if you would like more information about Galorath’s estimation models, please visit our contact page, call us at +1 310 414-3222 or click a button below to ask sales questions, sign up for our free library or schedule a demo.




Introduction To Software Productivity Causes, Effects and Measurement

February 13, 2014 · Filed Under General, Software Estimating · Comment 

I did the attached briefing on Introduction to software productivity some time ago and forgot about it. And while talking to Capers Jones today about productivity I found it and thought it was pretty good.

It talks about different productivity measures and how to get one that is consistent. In fact I believe the four main criteria are

  • consistent
  • repeatable
  • auditable
  • available

It cautions ” Quantifying Productivity is useful for comparison and estimate sanity checks but steps should be taken to ensure consistency”

 

Also, productivity, software estimation, and software measurement are discussed as follow:

Productivity: how to measure productivity, what to do with it, the pitfalls of productivity management, lessons learned and best practices.

Software Estimation: Making software estimation work for internal, external, fixed price and other acquisition scenarios as well as software estimation lessons learned and best practices

Software Measurement and actions based on software measurement: Including what to measure and how to apply measurement to more successful software projects.  Industry experience and best practices will be covered.

Thank you for reading “Dan on Estimating”, if you would like more information about Galorath’s estimation models, please visit our contact page, call us at +1 310 414-3222 or click a button below to ask sales questions, sign up for our free library or schedule a demo.




Dan’s Affordability Keynote at the SCAF Conference

It was interesting giving a talk on affordability to this group in London.  While the UK has affordability as a goal (in fact the conference theme was affordability) they look at it a bit differently.  It was interesting talking about price versus cost and the UK prospective.

Another thing that was interesting is the prospective that for IT systems software development is more like 6 to 10 percent of the total ownership cost.  But it is the bulk of the risk.

And I got out just before the tube strike began in earnest.  The next day I had to walk 7 miles from Paddington to get to a meeting.  The Bakerloo like was completely shut down, there was a 200 meter line for taxis and the buses were impossible to get on due to the masses of people.  I would love to see a cost analysis of this event.  The saddest part is, from what I understand, the strike was due to obsolete jobs being eliminated... But the people were not eliminated, just retrained.  Go figure?

Here are my slides: Affordability Analysis: The role of process, cost and ROI modeling in improved program management and performance.

Thank you for reading "Dan on Estimating", if you would like more information about Galorath's estimation models, please visit our contact page, call us at +1 310 414-3222 or click a button below to ask sales questions, sign up for our free library or schedule a demo.




Learning from healthcare.gov Challenges Webinar

December 6, 2013 · Filed Under General, Software Estimating · Comment 

Galorath Incorporated and CAST Software bring you a webinar….   This is not the typical finger pointing but summarizing the IT challenges and looking at what we can learn from them.

Join us for an expert examination of HealthCare.gov from Dan Galorath (Galorath Inc.), Lev Lesokhin (CAST), and Lee Fischman (Galorath Inc.).

Galorath and CAST each study and assist organizations to achieve more success in their software and IT systems.  Each examined HealthCare.gov’s troubled launch and its implications for the IT industry.  Galorath’s findings are summarized in Understanding HealthCare.gov’s Rocky Rollout“.  CAST’s Lev Lesokhin has discussed HealthCare.gov’s issues with the CBS Evening News, Wall Street Journal, and other media outlets.  

In this webinar we will reveal  more of what we’ve learned including the following:

1.   Findings described in “Understanding HealthCare.gov’s Rocky Rollout“.  What led to the conclusions behind some of its 13 points?

2.   Lessons learned from HealthCare.gov.  What went wrong, when and why?  What can be learned from the experience?

3.   Bestpractices for avoiding failure on future projects.  How can we plan, control and manage projects for success?

 

CAST is the leader in Software Analysis and Measurement, providing visibility into root causes of cost and risk in enterprise software systems.

Galorath’s SEER Software is the leading solution for project planning, estimating and tracking of software and IT projects.

Register Now:  Click here

Date & Time:  December 17, 2013  8:30am Pacific (11:30am Eastern, 4:30pm London)

 

Duration: 60 Minutes

 

Presenters:


Lev Lesokhin,
EVP of Strategy and Market Development at CAST


Dan Galorath,
Founder & CEO of Galorath Incorporated

Lee Fischman, Sr. Director of Galorath Incorporated 

 

 

Register: Click here 

Click “Register”

On the registration form, enter your information and then click “Submit”

Once the host approves your enrollment, you will receive a confirmation email message with instructions on how to join the event.

 

 

For Assistance: Contact Kelly Timko at 310-414-3222 x632 or ktimko@galorath.com

 

Thank you for reading “Dan on Estimating”, if you would like more information about Galorath’s estimation models, please visit our contact page, call us at +1 310 414-3222 or click a button below to ask sales questions, sign up for our free library or schedule a demo.




Next Page »