RSS

Category Archives: Industrial

Software continues to eat the world

Software continues to eat the world

One year ago Marc Andreessen, co-founder of Netscape and venture capital firm Andreessen-Horowitz, wrote an essay for the Wall Street Journal titled “Why Software Is Eating The World“. It is interesting to reflect back to this piece and some of the predictions made back at a time when Internet company LinkedIn had just gone public and Groupon was just filing for an IPO.

Andreessen’s observation was simply this: Software has become so powerful and computer infrastructure so cheap and ubiquitous that many industries are being disrupted by new business models enabled by that software. Examples listed were books (Amazon disrupting Borders), movie rental (NetFlix disrupting Blockbuster), music industry (Pandora, iTunes), animation movies (Pixar), photo-sharing services (disrupting Kodak), job recruiting (LinkedIn), telecommunication (Skype), video-gaming (Zynga) and others.

On the infrastructure side one can bolster this argument by pointing at the rapid development of new technologies such as cloud computing or big data analytics. Andreessen gave one example of the cost of running an Internet application in the cloud dropping by a factor of 100x in the last decade (from $150,000 / month in 2000 using LoudCloud to about $1500 / month in 2011 using Amazon Web Services). Microsoft now has infrastructure with Windows Azure where procuring an instance of a modern server at one (or even multiple) data center(s) takes only minutes and costs you less than $1 per CPU hour.

Likewise, the number of Internet users has grown from some 50 million around 2000 to more than 2 billion with broadband access in 2011. This is certainly one aspect fueling the enormous growth of social media companies like Facebook and Twitter. To be sure, not every high-flying startup goes on to be as successful after its IPO. Facebook trades at half the value of opening day after three months. Groupon trades at less than 20% of its IPO value some 9 months ago. But LinkedIn has sustained and even modestly grown its market capitalization. And Google and Apple both trade near or at their all-time high, with Apple today at $621b becoming the most valuable company of all time (non inflation-adjusted).

The growing dominance and ubiquitous reach of software shows in other areas as well. Take automobiles. Software is increasingly been used for comfort and safety in modern cars. In fact, self-driving cars – once the realm of science fiction such as flying hover cars – are now technically feasible and knocking on the door of broad industrial adoption. After driving 300.000 miles in test Google is now deploying its fleet of self-driving cars for the benefit of its employees. Engineers even take self-driving cars to the racetracks, such as up on Pikes Peak or the Thunderhill raceway. Performance is now at the level of very good drivers, with the benefit of not having the human flaws (drinking, falling asleep, texting, showing off, etc.) which cause so many accidents. Expert drivers still outperform the computer-driven cars. (That said, even human experts sometimes make mistakes with terrible consequences, such as this crash on Pikes Peak this year.) The situation is similar to how computers got so proficient at chess in the mid-nineties that finally even the world champion was defeated.

In this post I want to look at some other areas specifically impacting my own life, such as digital photography. I am not a professional photographer, but over the years my wife and I have owned dozens of cameras and have followed the evolution of digital photography and its software for many years. Of course, there is an ongoing development towards chips with higher resolution and lenses with better optic and faster controls. But the major innovation comes from better software. Things like High Dynamic Range (HDR) to compensate for stark contrast in lighting such as a portrait photo against a bright background. Or stitching multiple photos together to a panorama, with Microsoft’s PhotoSynth taking this to a new level by building 3D models from multiple shots of a scene.

One recent innovation comes in the form of the new Sony RX100 camera, which science writer David Pogue raved about in the New York Times as “the best pocket camera ever made”. My wife bought one a few weeks ago and we both have been learning all it can do ever since. Despite the many impressive features and specifications about lens, optics, chip, controls, etc. what I find most interesting is the software running on such a small device. The intelligent Automatic setting will decide most settings for your everyday use, while one can always direct priorities (aperture, shutter, program) or manually override most aspects. There are a great many menus and it is not trivial to get to use all capabilities of this camera, as it’s extremely feature-rich. Some examples of the more creative software come in modes such as ‘water color’ or ‘illustration’. The original image is processed right then and there to generate effects as if it was a painting or a drawing. Both original and processed photo are stored on the mini-SD card.

Flower close-up in ‘illustration’ mode

One interesting effect is to filter to just the main colors (Yellow, Red, Green, Blue). Many of these effects are shown on the display, with the aperture ring serving as a flexible multi-functional dial for more convenient handling with two hands. (Actually, the camera body is so small that it is a challenge to use all dials while holding the device; just like the BlackBerry keyboard made us write with two thumbs instead of ten fingers.) The point of such software features is not so much that they are radically new; you could do so with a good photo editing software for many years. The point is that with the ease and integration of having them at your fingertips you are much more likely to use them.

Example of suppressing all colors except yellow

The camera will allow registering of faces and detect those in images. You can set it up such that it will take a picture only when it detects a small/medium/large smile on the subject being photographed. One setting allows you to take self-portrait, with the timer starting to count down as soon as the camera detects one (or two) faces in the picture! It is an eerie experience when the camera starts to “understand” what is happening in the image!

There is an automatic panorama stitching mode where you just hold the button and swipe the camera left-right or up-down while the camera takes multiple shots. It automatically stitches them into one composite, so no more uploading of the individual photos and stitching on the computer required.

Beach panorama stitched on the camera using swipe-&-shoot

I have been experimenting with panorama photos since 2005 (see my collection or my Panoramas from the Panamerican Peaks adventure). It’s always been somewhat tedious and results were often mixed (lens distortions, lighting changes sun vs. cloud or objects moving during the individual frames, not holding the camera level, skipping a part of the horizon, etc.) despite crafty post-processing on the computer with image software. I have read about special 360 degree lenses to take high-end panoramas, but who wants to go to those lengths just for the occasional panorama photo? From my experience, nothing moves the needle as much as the ease and integration of taking panoramas right in the camera as the RX100 does.

Or take the field of healthcare. Big Data, Mobility and Cloud Computing make possible entirely new business models. Let’s just look at mobility. The smartphone is evolving into a universal healthcare device for measuring, tracking and visualizing medical information. Since many people have their smartphone with them at almost all times, one can start tracking and analyzing personal medical data over time. And for almost any medical measurement, “there is an app for that”. One interesting example is this optical heart-rate monitor app Cardiio for the iPhone. (Cardio + IO ?)

Screenshots of Cardiio iPhone app to optically track heart rate

It is amazing that this app can track your heart rate just by analyzing the changes of light reflected from your face with its built-in camera. Not even a plug-in required!

Another system comes from Withings, this one designed to turn the iPhone into a blood pressure monitor. A velcro sleeve with battery mount and cable plugs into the iPhone and an app controls the inflation of the sleeve, the measurement and some simple statistics.

Blood pressure monitor system from Withings for iPhone

Again, it’s fairly simple to just put the sleeve around one upper arm and push the button on the iPhone app. The results are systolic and diastolic blood pressure readings and heart rate.

Sample blood pressure and pulse measurement

Like many other monitoring apps this one also keeps track of the readings and does some simple form of visual plotting and averaging.

Plot of several blood pressure readings

There is also a separate app which will allow you to upload your data and create a more comprehensive record of your own health over time. Withings provides a few other medical devices such as scales to add body weight and body fat readings. The company tagline is “smart and connected things”.

One final example is an award-winning contribution from a student team from Australia called Stethocloud. This system is aimed at diagnosing pneumonia. It is comprised of an app for the iPhone, a simple stethoscope plug-in for the iPhone and on the back-end some server-based software analyzing the measurements in the Windows Azure cloud according to standards defined by the World Health Organization. The winning team (in Microsoft’s 2012 Imagine Cup) built a prototype in only 2 weeks and had only minimal upfront investments.

StethoCloud system for iPhone to diagnose pneumonia

This last example perhaps illustrates best the opportunities of new software technologies to bring unprecedented advances to healthcare – and to many other fields and industries. I think Marc Andreessen was spot on with his observation that software is eating the world. It certainly does in my world.

Advertisements
 
Leave a comment

Posted by on August 20, 2012 in Industrial, Medical, Socioeconomic

 

Tags: , , , , ,

Sankey Diagrams

Sankey Diagrams

Whenever you want to show the flow of a quantity (such as energy or money) through a network of nodes you can use Sankey diagrams:

“A Sankey diagram is a directional flow chart where the width of the streams is proportional to the quantity of flow, and where the flows can be combined, split and traced through a series of events or stages.”
(source: CHEMICAL ENGINEERING Blog)

One area where this can be applied very well is that of costing. By modeling the flow of cost through a company one can analyze the aggregated cost and thus determine the profitability of individual products, customers or channels. Using the principles of activity-based costing one can create a cost-assignment network linking cost pools or accounts (as tracked in the General Ledger) via the employees and their activities to the products and customers. Such a Cost Flow can then be visualized using a Sankey diagram:

Cost Flow from Accounts via Expenses and Activities to Products

The direction of flow (here from left to right) is indicated by the color assignment from nodes to its outflowing streams. Note also the intuitive notion of zero-loss assignment: For each node the sum of the in- and outflowing streams (= height of that node) remains the same. Hence all the cost is accounted for, nothing is lost. If you stacked all nodes on top of one another they would rise to the same height. (Random data for illustration purposes only.)

The above diagram was created in Mathematica using modified source code originally from Sam Calisch who had posted it in 2011 here. Sam also included a “SankeyNotes.pdf” document explaining the details of the algorithms encoded in the source, such as how to arrange the node lists and how to draw the streams.

I find these a perfect example of how a manual drawing can go a long ways to illustrate the ideas behind an algorithm, which makes it a lot easier to understand and reuse the source code. Thanks to Sam for this code and documentation. Sam by the way used the code to illustrate the efficiency of energy use (vs. waste) in Australia:

Energy Flow comparison between New South Wales and Australia (Sam Calisch)

Note the sub-flows within each stream to compare a part (New South Wales) against the whole (Australia).

Another interesting use of Sankey Diagrams has been published a few weeks ago on ProPublica about campaign finance flow. This is particularly useful as it is interactive (click on image to get to interactive version).

Tangled Web of Campaign Finance Flow

Note the campaigns in green and the Super-PACs in brown color. The data is sourced from FEC and the New York Times Campaign Finance API. Note that in the interactive version you can click on any source on the left or any destination on the right to see the outgoing and incoming streams.

Finance Flow From Obama-For-America

Finance Flow to American Express

Here are some more examples. Sankey diagrams are also used in Google Flow Analytics (called Event Flow, Goal Flow, Visitor Flow). I wouldn’t be surprised to see Sankey Diagrams make their way into modern data visualization tools such as Tableau or QlikView, perhaps even into Excel some day… Here are some Visio shapes and links to other resources.

 
3 Comments

Posted by on May 14, 2012 in Financial, Industrial

 

Tags: , , ,

Quarterly Comparison: Apple, Microsoft, Google, Amazon

Quarterly Comparison: Apple, Microsoft, Google, Amazon

Last quarter we looked at the financials and underlying product & service portfolios of four of the biggest technology companies in the post “Side by Side: Apple, Microsoft, Google, Amazon“. With the recent reporting of results for Q1 2012 it is a good time to revisit this subject.

Comparison of Financials Q4 2011 and Q1 2012 for Apple, Microsoft, Google, and Amazon.

Market cap has grown roughly by 25% for both Apple and Amazon, whereas Microsoft and Google only added 5% or less. A sequential quarter comparison can be misleading due to seasonal changes, which impact different industries and business models in a different way. For example, Google’s ad revenue is somewhat less impacted by seasonal shopping than the other companies.

Sequential quarter comparison of financials

Apple and Microsoft seem to be impacted in a similar way by seasonal changes. For Amazon, which already has by far the lowest margin of all four companies, operating income decreased by 40% while it increased its headcount by 17%. This leads to much lower income per employee and with increased stock price to a doubling of its already very high P/E ratio. I’m not a stock market analyst, but Amazon’s P/E ratio of now near 200 seems extraordinarily high. By comparison, the other companies look downright cheap: Apple 8.8, Microsoft 10.5, Google 14.5

Horace Dediu from asymco.com has also revisited this topic in his post “Which is best: hardware, software, or services?“. What’s striking is that all three companies (except Amazon) now have operating margins between 30-40% – very high for such large businesses – with Apple taking the top near 40%. Over the last 5 years, Apple has doubled it’s margin (20% to 40%), whereas Microsoft (35-40%) and Google (30-35%) remained near their levels.

(Source: Asymco.com)

Long term the most important aspect of a business is not how big it has become, but how profitable it is. In that regard Amazon is the odd one out. Their operating income last quarter was about 1% of revenue. Amazon needs to move $100 worth of goods to earn $1. They employ 65,000 people and had revenue of $13.2b last quarter, yet only earned $130m during that time! Apple earns more money just with their iPad covers! Amazon’s strategy is to subsidize the initial Kindle Fire sale and hoping to make money on the additional purchases over the lifetime of the product. In light of these numbers, do you think Amazon has a future with it’s Kindle Fire tablet against the iPad?

But what really struck me about the extreme differences in profitability is this comparison of Apple and Microsoft product lines (source: @asymco twitter):

(source: @asymco twitter)

This shows what an impressive and sustained success the iPhone has been. And the iPad is on track to grow even faster. Horace Dediu guesses that Apple’s iPad will be a bigger profit generator than Windows in one quarter, and a bigger profit generator than Google (yes, all of Google) in three quarters. We will check on those predictions when the time comes…

 
3 Comments

Posted by on May 2, 2012 in Financial, Industrial

 

Tags: , , , , , ,

Tube Maps

Tube Maps

I just got back from a combined business and vacation trip around Easter to Germany and Austria. In Europe, public transportation is an important part of the infrastructure. Especially in the big cities many people commute daily by train or subway, some even live without a car.

One of the most important pieces of information for train and subway systems is the tube map. It is a schematic transit map showing the lines, stations and connections of the train or subway system. It’s main element is that it abstracts away geographical detail (where is what) and focuses on topological aspects: How do I need to transit to which other line to get to a particular station?

London Tube Map (Source: Wikipedia)

The Wikipedia tube map article details the origins around the London subway system which was called the tube (hence the name for this type of map) dating back to first schematic maps in 1931 by Harry Beck:

“Beck was a London Underground employee who realised that because the railway ran mostly underground, the physical locations of the stations were irrelevant to the traveller wanting to know how to get to one station from another — only the topology of the railway mattered.”

This style of map has been widely adopted and successively refined. Having grown up in Munich and having used its train (S-Bahn) and subway (U-Bahn) system for some 25 years, I came to realize that it is not only a convenient tool for the traveller. It can form the basis of mental models of the topology of a city. The first lines of the Munich S- and U-Bahn system were built for the Olympic Games in 1972. The history and evolution of the train and subway system over the 40 years since has been documented on this website. Let’s look at the tube maps and their evolution in roughly 10 year time intervals.

Munich Tube Map 1971

1971: Note the basic shape of a central track West-East shared by all S-Bahn lines which then fan out radially to the suburbs. The 45° angles help with the text labels and add simplicity to the layout. This simplicity is one key element for such tube maps to become a mental model of the city topology, i.e. of knowing what is where and how to get to it. Note that initially there are only two U-Bahn lines sharing most of their underground tracks.

Munich tube map 1980

1980: The design of the map evolves to “stretch” out the line-graph to both fill out the entire available rectangular space and to free up some more space in the center; here two additional U-Bahn lines require more space, also due to the fact that U-Bahn stations are closer together than S-Bahn stations in the periphery. The Text label “P+R” is introduced to designate Park & Ride facilities at the stations for commuters.

Munich tube map 1992

1992: Some additional U-Bahn lines and stations fill in the center. One of the S-Bahn lines is renamed (S3 -> S8) and extended to the North to connect to the new Munich airport (Erding). Also a few minor map changes (new color scheme, font and legend).

Munich tube map 2001

2001: S1 now also reaches the new airport, which simplifies travel from the Western part of the city and effectively creates a Northern loop. The map changes in the top section to reflect this new topology; this graphically compresses the U-Bahn system in the upper half. A new color (blue) for the stations represents he inner zone. This together with the new text label “XXL” represents tariff boundaries. (A similar approach with blue font color for inner zone station names was dropped after a brief version in 1997; it looked confusing.)

Munich tube map 2012

2012: The current map adds several graphical aspects such as the concentric rings of background color for tariff boundaries, a new font for cleaner look and less line breaks as well as icons for long distance train connections. It also shows some geographic features such as the Isar river and the two lakes in the South-West as well as icons for tourist attractions or land-marks such as the new soccer stadium, the ‘Deutsches Museum’ or the Zoo. For a hi-res map see this pdf file.

Such a sequence shows the evolution of schematic concepts and visual representations over the decades. When you take away some of the simplifying tube map abstractions such as the 45° angle, you get topographical maps like this:

Topographical map of Munich U-Bahn 2010

While such a map gives you a more precise idea of where you are at any given station in the city, it is much harder to remember and to reconstruct in your head. I believe that this simplicity-by-design of modern tube maps makes it such a strong candidate for forming the basis of mental models of city topology.

Here is an interesting variation of the Munich transit system in a so called isochrone map using colors to display transit times say from the center to other city destinations. Robin Clarke created the following map and describes in this post how he did it.

Munich transit system Isochrone Map (Source: Robin Clarke)

A final example of using tube maps in an interactive graphic comes from Tom Carden. He created an applet that lets you click on any of the 200 London subway stations and get a isochronic map showing transit times from that origin to any other station. While not laid out as clean as the Beck-style tube maps, this interactive graphic represents 200 different maps all in one! (Click on the image to get to the interactive version.)

Interactive London Tube Map (Source: Tom Carden)

See also the more recent Blog post London Tube Map for additional examples of graph visualizations using the London underground as illustration object.

As a traveller arriving in an unknown city we often tend to take such subway infrastructure and its documentation for granted. What amazes me is to think about the amount of cumulative work – plan, design, construction, logistics, etc. – that has gone into building such an infrastructure. A few interesting facts about the Munich U-Bahn (subway) system: 6 lines, 100 stations, 103 km, ~ 1 million passengers /day. (Source: Wikipedia). Building a subway costs in the order of $100 million/km, so this represents an investment of about $10 billion! Think about that the next time you try to find your way through a new city…

 
2 Comments

Posted by on April 20, 2012 in Industrial, Recreational

 

Tags: ,

Interactive Documents – Roambi Flow

Interactive Documents – Roambi Flow

One year ago I purchased my own iPad 2. When using it in meetings, it quickly became apparent how much potential there is to make presented information much more interactive. I posted last June about Interactive and Visual Information. In the meantime, more and more software is aiming at making documents more interactive, especially on the iPad to leverage mobility and touch.

In this post we will look at Roambi Flow, a product that lets you compose documents with interactive elements. Roambi is a set of business intelligence products by San Diego based company MeLLmo which has been designed from the ground up to take advantage of iOS features such as rich graphics and touch interface. On Roambi’s product website you will find detailed descriptions of each of these products.

Roambi Analytics Views

Roambi Analytics introduced a series of so called Views. Each of these views is interesting in its own and warrants a more in-depth coverage; I’ll just enumerate them briefly.

Blink gives you cube analytics displaying various measures in selected dimensions, swiping and scrolling through a data set.

Cardex is a visual metaphor for organizing sets of elementary reports and visually comparing them side-by-side like a mini comparison dashboard.

CataList lets you browse top-level lists and drill into a detailed view with sliders to see data points over time and display highlighted information.

Elements allows you to compose dashboards of connected, basic chart elements to explore multi-dimensional data.

Layers specializes on the display and navigation of hierarchically grouped data sets – such as continent, country, city – through the use of scroll, pinch and zoom gestures.

PieView is a variation of the Piechart theme. It’s main innovation is to allow the rotation of the entire piechart similar to the original Apple iPod click wheel. (It doesn’t eliminate the shortcomings of piecharts per se, but it makes them a little easier to live with and a lot more fun to explore.)

Squares is using the heat map concept in a very intuitive and easy to use way to display data organized along two main axes – such as the global sales performance of various products in various countries. Dragging along rows or columns highlights them one at a time, tapping on a row or column “explodes” its content to a matrix with more detail – in which one can again navigate, sort, etc.. Tap & Hold on the heat map generates a Fish-Eye view with more detail of the tapped element maximized. Moving while holding will move the fish-eye to areas of interest. (see image below)

SuperList is a generic view for lists with numeric information that allows to sort, filter, toggle between bars and numbers etc. Think of it as a starting point for tabular data display on the iPad.

Fish-Eye view in Squares, one of the Roambi Analytics views

Each view has a Help-style description with a short 1 min video overview in it. This goes to show that seeing these views in narrated action is much more intuitive and easier to understand than just reading about them. It’s literally leveraging some “show & tell”. The best way to explore these views is to download the free Roambi Viewer apps on the iPad and play with them. They come with stored sample data sets so you can visually explore the views even while you are offline. Roambi also features brief videos and tutorials on their website.

But back to Roambi Flow: You want your data to tell stories. This is best done through a combination of text explaining the context, perhaps some multimedia demonstrating the highlights and some interactive elements allowing the reader to visually explore on her own. This is where Roambi Flow comes in. It’s a publishing container that allows you to embed the above views (and other multi-media content) into regular text documents. The reader navigates the content at the top-level like a traditional book, either by clicking on the table of contents or by literally flipping through the pages. The app will even simulate the page turning like we are used to from Apple’s iBooks.

Page transition in Roambi Flow; Note the embedded, interactive element on the next page.

The individual elements can be double-tapped, which expands them to full-screen and then support their full visual exploration capabilities. The views can be linked to backend data sources to automatically stay in sync with up-to-date information. View displays can be bookmarked and shared with others. But the main point really is the fact that the reader does not only see a static image, but can interact and manipulate the views to obtain a richer understanding of the underlying data sets.

Roambi Flow page with two interactive view elements.

Given the rapid adoption of iPads in corporate environments it is straightforward to see such interactive documents spreading both within a company as well as in its external communications. Imagine reading the annual report, the sales pitch or the research paper when you can interact with the financials, the offered product or the proposed scientific model! With interactive content, reading will never be the same.

 
Leave a comment

Posted by on March 12, 2012 in Industrial

 

Tags: , , ,

Mobile Business Intelligence Market Study

Mobile Business Intelligence Market Study

Dresner Advisory Services publishes an annual study on Mobile Business Intelligence vendors, the latest in October 2011. It focuses on the mobile capabilities of BI platform vendors similar to those in the Gartner Magic Quadrant for Business Intelligence we recently looked at.

The ~50 page document has a good executive summary and provides insight from industry surveys and changes between 2010 and 2011. In terms of data visualizations, it generally does a poor job of conveying the study findings. There is an abundance of pie charts and stacked bar charts with often very confusing color codes. For example, consider this chart on BI vendor mobile platform priority:

Mobile BI Vendor Platform Priority (source: DAS)

Rank information shouldn’t be conveyed by color (better by vertical position). It is very confusing to see which platforms gained or lost in the ranking. A data visualization should first and foremost make it easy to spot patterns and thus provide insight. Not every dataset makes for a good Excel bar chart.

All that said, I found one very useful chart which shows all vendor Mobile BI capabilities at a glance:

Mobile BI Vendor Scores (source: DAS)

Regarding the vendor scoring, from the study:

Using the data that was provided by twenty-four different BI vendors, we constructed a model which scores them based on mobile platform support, platform integration and numbers of supported BI features (Figure 33).

Please carefully review the detailed vendor and product profiles on pages 47 – 52 and to consider both dimensions (i.e., platform and features) independent of each other.

It should be noted that this model reflects only two dimensions of a BI vendor’s product capability and is not intended to indicate “market leadership” only a convergence of capabilities for Mobile BI. Readers are encouraged to use other tools to understand the many other dimensions of vendor capability, such as our own Wisdom of Crowds Business Intelligence Market Study ®.

The full report can be downloaded from the Yellowfin website here.

 
Leave a comment

Posted by on February 29, 2012 in Industrial

 

Tags: , ,

Probabilistic Project Management at NASA with Joint Confidence Level (JCL-PC)

Probabilistic Project Management at NASA with Joint Confidence Level (JCL-PC)

On the Strategic Project and Portfolio Management Blog by Simon Moore one can find many fascinating stories about project failures as well as a related collection of project management case studies. One entry there links to a project management method NASA is mandating internally since 2009 to estimate costs and schedule of their various aerospace projects. The method is called Joint Confidence Level – Probabilistic Calculator (JCL-PC). It’s a sophisticated method using historical data and insight into estimation psychology (like optimism bias) to arrive at corrective multipliers for project estimates based on project completion percentages with required confidence level. It’s also using Monte Carlo simulations to determine outcomes, leading to scatterplots of the simulated project runs on a Cost-vs.-Schedule plane. From there one can determine estimates with for example 70% confidence levels for what the cost and schedule overruns will likely be.

If you’re either already familiar with the method or if you are very good at abstract thinking the above paragraph will have meant something to you. If it didn’t, bear with me. In this post I make a brief attempt to explain what I understood about the method using the data visualizations from two sources (a 100+ page report and a 12 page FAQ). The report is fascinating on many levels, as it deals with the history of high-profile project overruns (Apollo program, Space Shuttle, Space Station) and the pervasive culture of under-estimation (optimism bias) through not accounting for project risks that are unknown, but historically evident.

JCL starts with historical observations of similar projects with regards to cost and schedule overruns. For example, the above cited report contains best fit histogram distributions for robotic missions.

Overrun Distributions of Cost and Schedule for Robotic Missions (Soucre: NASA)

The idea is to use a set of such distributions for probabilistic estimates of cost and schedule. The set of distributions needs to account for the fact that in the early stages of a project there are more unknowns and as such higher risk of overruns. From the report:

The JCL-PC estimating method is based on the hypotheses that in the beginning phases of a project there are many unknown risks – and over time the project will have a high probability of exceeding estimated costs and scheduled duration. … Work as it was initially planned will inevitably change. Quantifiable risks become clearer and NASA’s S-Curves will tend to lay down as the work goes forward. Keep in mind that it’s not the project that is becoming inherently riskier. It’s a matter of participants fully identifying the real work that was “out there” all along. Even though the scope of the work wasn’t fully perceived “back when” – progress has continued to identify the risks and quantify the corrective actions. History is written in real time and that history differs to a greater or lesser degree from what was anticipated. The JCL-PC helps us better plan for and manage that difference.

The JCL-PC method strikes a needed balance between subjectivity and anticipated risk variability leaving only one remaining probability influence factor to deal with. – namely, assigning the percentage complete of the subject project. This % complete factor includes both subjective and objective elements.

One of the key elements is the notion of a multiplier which implements this reduced-uncertainty-over-time as well as a so called optimism corrector and other project risk in line with historical aerospace project overruns. The multiplier is plotted below as a function of the project % complete parameter for different confidence levels:

Multiplier as function of project % complete for various confidence levels

The concept is illustrated via two charts of a fictitious $1m project (applied here to cost overruns, but equally applicable to schedule overruns): The first shows a point estimate and it’s S-curves (confidence bands) per project % complete.

The second shows the S-Curves after applying “the optimism corrector and some minor project risk, through a more typical project life cycle with project scope creep … As the project evolves the S-Curve moves slightly to the right and becomes more and more vertical.”

It would be great to have an interactive graphic where the S-Cruves are plotted in response to sliding the project % complete between 0% and 100%. The report lists the above multipliers in a numerical table spanning project % complete (in 1% increments) and four confidence levels (50%, 60%, 70%, 80%). Rather than copying the entire table I filtered this down to just 10% increments in project % complete. This table tells NASA officials at various confidence levels, how much money they will have to spend for a $1m project as a function of project % complete:

Cost Estimate Table with project % complete and confidence levels

The data point highlighted in yellow is described as follows:

When the project is 50% complete, you’ll notice that a 50% confidence level suggests that the project can be completed for the anticipated $1,000,000. However, if we adhere to the NASA standard of a 70% confidence level, we see that another $400,000+ will likely be needed to complete the project. No matter how well a project is managed, it rarely compensates for ultra- optimistic budget estimates that sooner or later return with a vengeance and overcome the most skillful leaders.

As a final illustration the FAQ document includes this scatterplot as JCL-PC output:

Scatter Plot of Monte Carlo simulation with JCL-PC

A Frontier Curve represents all possible combinations of cost and schedule that will give you a percent JCL. The plot shows the Frontier Curve for a 70% JCL in yellow. The green dots are simulated runs with outcomes below the selected cost and schedule (blue cross-hair, yellow labels). White dots have either cost or schedule overruns, red dots have both.

The report makes bold claims about the potential of JCL-PC, but also about the challenges inherent in attempting to change an entire management culture. I am not qualified to comment on these claims, but my impression is that such probabilistic project management methods will raise the bar in the field and should lead to more accurate estimates.

The more I think about such abstract concepts, the more I’m convinced that mental models are inherently visual. We remember some key visualizations or charts and anchor our understanding of the concept around those visual images. We also use them to communicate or teach the concepts to each other – hence the value of the whiteboard or even the napkin drawing. As such, the increasing computational ability to produce such visual images and ideally even interactive graphics is an important element of academic and scientific endeavors.

 
Leave a comment

Posted by on February 25, 2012 in Financial, Industrial, Scientific

 

Tags: , , ,

 
%d bloggers like this: