This informationisbeautiful graphic is pretty, but I don’t find it informative. The y scale is nonlinear, and I don’t know if the x scale conveys anything. It’s hard to work out the timing of inundation, which is really the key. The focus on the low points of big cities in developed countries is misleading, because those will be defended for a long time. Ho Chi Minh city should be on there, as well as the US gulf coast. USA Today would love this.

(Click the image to try it).

With this experiment, I was striving for a couple things:

• A reduced-form version of the climate model, with “good enough” accuracy and interactive speed, as in Vensim’s Synthesim mode (no client-server latency).
• Tufte-like simplicity of the UI (no grids or axis labels to waste electrons). Moving the mouse around changes the emissions trajectory, and sweeps an indicator line that gives the scale of input and outputs.
• Pervasive representation of uncertainty (indicated by shading on temperature as a start).

This is just a prototype, but it’s already more fun than models with traditional interfaces.

I wanted to run it on the iPhone, but was stymied by problems translating the model to Processing.js (javascript) and had to set it aside. Recently Travis Franck stepped in and did a manual translation, proving the concept, so I took another look at the problem. In the meantime, a neat export tool has made it easy. It turns out that my code problem was as simple as replacing “float []” with “float[]” so now I have a javascript version here. It runs well in Firefox, but there are a few glitches on Safari and iPhones – text doesn’t render properly, and I don’t quite understand the event model. Still, it’s cool that modest dynamic models can run realtime on the iPhone. [Update: forgot to mention that I sued Michael Schieben's touchmove function modification to processing.js.]

The learning curve for all of this is remarkably short. If you’re familiar with Java, it’s very easy to pick up Processing (it’s probably easy coming from other languages as well). I spent just a few days fooling around before I had the hang of building this app. The core model is just standard Euler ODE code:

initialize parameters

initialize levels

do while time < final time

compute rates & auxiliaries

compute levels

The only hassle is that equations have to be ordered manually. I built a Vensim prototype of the model halfway through, in order to stay clear on the structure as I flew seat-of-the pants.

With the latest Processing.js tools, it’s very easy to port to javascript, which runs on nearly everything. Getting it running on the iPhone (almost) was just a matter of discovering viewport meta tags and a line of CSS to set zero margins. The total codebase for my most complicated version so far is only 500 lines. I think there’s a lot of potential for sharing model insights through simple, appealing browser tools and handheld platforms.

As an aside, I always wondered why javascript didn’t seem to have much to do with Java. The answer is in this funny programming timeline. It’s basically false advertising.

The tradition, one might call it the orthodoxy, in system dynamics is that a problem can only be analysed, and policy guidance given, through the aegis of a fully quantified model. In the last 15 years, however, a number of purely qualitative models have been described, and have been criticised, in the literature. This article briefly reviews that debate and then discusses some of the problems and risks sometimes involved in quantification. Those problems are exemplified by an analysis of a particular model, which turns out to bear little relation to the real problem it purported to analyse. Some qualitative models are then reviewed to show that they can, indeed, lead to policy insights and five roles for qualitative models are identified. Finally, a research agenda is proposed to determine the wise balance between qualitative and quantitative models.

… In none of this work was it stated or implied that dynamic behaviour can reliably be inferred from a complex diagram; it has simply been argued that describing a system is, in itself, a useful thing to do and may lead to better understanding of the problem in question. It has, on the other hand, been implied that, in some cases, quantification might be fraught with so many uncertainties that the model’s outputs could be so misleading that the policy inferences drawn from them might be illusory. The research issue is whether or not there are circumstances in which the uncertainties of simulation may be so large that the results are seriously misleading to the analyst and the client. … This stream of work has attracted some adverse comment. Lane has gone so far as to assert that system dynamics without quantified simulation is an oxymoron and has called it ‘system dynamics lite (sic)’. …

Coyle (2000) Qualitative and quantitative modelling in system dynamics: some research questions

Jack Homer and Rogelio Oliva aren’t buying it:

Geoff Coyle has recently posed the question as to whether or not there may be situations in which computer simulation adds no value beyond that gained from qualitative causal-loop mapping. We argue that simulation nearly always adds value, even in the face of significant uncertainties about data and the formulation of soft variables. This value derives from the fact that simulation models are formally testable, making it possible to draw behavioral and policy inferences reliably through simulation in a way that is rarely possible with maps alone. Even in those cases in which the uncertainties are too great to reach firm conclusions from a model, simulation can provide value by indicating which pieces of information would be required in order to make firm conclusions possible. Though qualitative mapping is useful for describing a problem situation and its possible causes and solutions, the added value of simulation modeling suggests that it should be used for dynamic analysis whenever the stakes are significant and time and budget permit.

Homer & Oliva (2001) Maps and models in system dynamics: a response to Coyle

Coyle rejoins:

This rejoinder clarifies that there is significant agreement between my position and that of Homer and Oliva as elaborated in their response. Where we differ is largely to the extent that quantification offers worthwhile benefit over and above analysis from qualitative analysis (diagrams and discourse) alone. Quantification may indeed offer potential value in many cases, though even here it may not actually represent ‘‘value for money’’. However, even more concerning is that in other cases the risks associated with attempting to quantify multiple and poorly understood soft relationships are likely to outweigh whatever potential benefit there might be. To support these propositions I add further citations to published work that recount effective qualitative-only based studies, and I offer a further real-world example where any attempts to quantify ‘‘multiple softness’’ could have lead to confusion rather than enlightenment. My proposition remains that this is an issue that deserves real research to test the positions of Homer and Oliva, myself, and no doubt others, which are at this stage largely based on personal experiences and anecdotal evidence.

Coyle (2001) Rejoinder to Homer and Oliva

My take: I agree with Coyle that qualitative models can often lead to insight. However, I don’t buy the argument that the risks of quantification of poorly understood soft variables exceeds the benefits. First, if the variables in question are really too squishy to get a grip on, that part of the modeling effort will fail. Even so, the modeler will have some other working pieces that are more physical or certain, providing insight into the context in which the soft variables operate. Second, as long as the modeler is doing things right, which means spending ample effort on validation and sensitivity analysis, the danger of dodgy quantification will reveal itself as large uncertainties in behavior subject to the assumptions in question. Third, the mere attempt  to quantify the qualitative is likely to yield some insight into the uncertain variables, which exceeds that derived from the purely qualitative approach. In fact, I would argue that the greater danger lies in the qualitative approach, because it is quite likely that plausible-looking constructs on a diagram will go unchallenged, yet harbor deep conceptual problems that would be revealed by modeling.

I see this as a cost-benefit question. With infinite resources, a model always beats a diagram. The trouble is that in many cases time, money and the will of participants are in short supply, or can’t be justified given the small scale of a problem. Often in those cases a qualitative approach is justified, and diagramming or other elicitation of structure is likely to yield a better outcome than pure talk. Also, where resources are limited, an overzealous modeling attempt could lead to narrow focus, overemphasis on easily quantifiable concepts, and implementation failure due to too much model and not enough process. If there’s a risk to modeling, that’s it – but that’s a risk of bad modeling, and there are many of those.

Here’s a CLD:

And here’s it’s stock-flow sibling:

My bottom line is:

• CLDs are very useful, if developed and presented with a little care.
• It’s often clearer to use a hybrid diagram that includes stock-flow “main chains”. However, that also involves a higher burden of explanation of the visual language.
• You can get into a lot of trouble if you try to mentally simulate the dynamics of a complex CLD, because they’re so underspecified (but you might be better off than talking, or making lists).
• You’re more likely to know what you’re talking about if you go through the process of building a model.
• A big, messy picture of a whole problem space can be a nice complement to a focused, high quality model.

Here’s why:

There are well documented conceptual problems with CLD notation. More importantly, it’s easy to make very bad CLDs. Just use lots of crossing lines (spaghetti), variable names with no sense of direction, neglect to label loop and link polarity, and mix in some clip art for good measure. (There’s some good advice on CLD notation here, but replace the S and O arrow polarity notation with + and -). As a practical matter, it’s been my experience that most causal loop diagrams leave a lot to the imagination, which you can easily discover by attempting to formalize one as a model. You’ll discover unstated parameters, aggregation questions, and other leaps of logic.

The Afghanistan diagram share’s many of those problems. It has the dreaded spaghetti topology. It doesn’t indicate loop polarities. Some variables are really concept areas of interest, rather than quantities that can vary. There’s no way to translate it directly to equations (however, the rumor mill has it that there is an underlying model).

Still, the Afghanistan diagram and other messy mind maps like it aren’t useless, as many NYT commenters asserted. First, it might be a good way to summarize the output of a brainstorming session. In that case, the goal is to surface as many relationships as possible up front. Detailed critique of each link or loop along the way tends to bog down such generative processes. If you don’t later drill into the details of the spaghetti to sort out the dynamics, you might remain as muddled as you were when you started, but that doesn’t make the spaghetti intrinsically useless.

Similarly, a spaghetti diagram can be a useful overview of the complicated territory covered by a model. With most audiences, you’d be crazy to start with the full diagram – you’ll just turn people off. Instead, the presentation should build up the big picture from smaller pieces, reflecting on the contribution of each link or loop to the overall dynamics. (Apparently this is how the Afghanistan diagram was actually presented). Of course, that only works if you have an underlying model; otherwise the incomplete formalization of a CLD makes it really easy to draw spurious conclusions. Without a model, all you really have is a dynamic hypothesis – which still might be a lot more than you had before you drew the diagram.

In my own work, I don’t use CLDs very much. I prefer stock-flow diagrams, and I can hardly get out of bed without a real model. Still, thinking back, I can think of two CLDs that have been very successful.

The first (below, click to enlarge) is a work product from the first day of a collaborative workshop on emissions offsets, which Ron Suiter and I ran in California. With support of WSPA, we assembled industry, regulators, NGOs, and offset providers to talk about the pros and cons of including offsets in AB32 regulations (particularly the cap & trade system). Immediately two worldviews emerged: offsets are essential, and offsets are a scam. This diagram explains both worldviews as competing perceptions about the relative strength of various feedback loops in the diagram.

Like most CLDs, this one’s not completely explicit about the “physics” of the system. Still, it communicated very well. I walked through it at the start of the second day of the workshop, and their were lots of positive comments and subsequent references to the framework. It’s important to note that I didn’t present this as a monolith – I built it up piece by piece (as you can see in the report), with color coding and references to the elements of the first day conversation that backed up each link or loop. I probably could translate this to a stock-flow diagram, but there’s no way I could have created and described it within the time available.

The second is a map of the transport fuels policy space, developed to support conversations with the Energy Commission and others in California:

The colored regions represent three models that were in use at the CEC and CalTrans at the time (around 2005, following AB2076 study). The key insight is not so much the about the specifics of the structure, but that the existing models don’t span the space. The supply and demand side (yellow & red) are covered by separate models, and the only integration is provided by a general equilibrium model (green) with incompatible aggregation and units of measure. I do present this diagram all at once, but only to subject matter experts who can quickly recognize the content.

This model does have a working model counterpart that maps more or less one to one to the CLD concepts:

I find that the stock-flow version (even with a few hidden parameters, as above) does freak people out on first contact, at least if they aren’t familiar with stock-flow diagrams. However, when presented in digestible chunks, it does make sense to them.

It’s interesting to contrast my diagrams with a hybrid stock-flow representation of the transport space, from Jeroen Struben and John Sterman’s work on the alt fuel/vehicle transition:

There’s more than one way to skin a cat.

The method makes it possible to see changes in behavior in time series with waaay to many oscillations to explore on a normal 2D time-value plot:

Read more on arXiv.

Somewhere in the blogosphere I ran across this nice infographic contrasting European aviation and Icelandic volcano emissions:

I think there’s a lot of potential for elegant presentation with Processing. There are several physics libraries and many simulations with a physical, chemical, or mathematical basis at OpenProcessing.org:

If you like code, it’s definitely worth a look.

A year or two back, I created a tool, based on VisAD, that uses the Vensim .dll to do multidimensional visualization of model output. It’s much cruder, but cooler in one way: it does interactive 3D. Anyway, I hoped that Tableau, used with Vensim, would be a good replacement for my unfinished tool.

After some experimentation, I think there’s a lot of potential, but it’s not going to be the match made in heaven that I hoped for. Cycle time is one obstacle: data can be exported from Vensim in .tab, .xls, or a relational table format (known as “data list” in the export dialog). If you go the text route (.tab), you have to pass through Excel to convert it to .csv, which Tableau reads. If you go the .xls route, you don’t need to pass through Excel, but may need to close/open the Tableau workspace to avoid file lock collisions. The relational format works, but yields a fundamentally different description of the data, which may be harder to work with.

I think where the pairing might really shine is with model output exported to a database server via Vensim’s ODBC features. I’m lukewarm on doing that with relational databases, because they just don’t get time series. A multidimensional database would be much better, but unfortunately I don’t have time to try at the moment.

Whether it works with models or not, Tableau is a nice tool, and I’d recommend a test drive.

The orange bits above are uncovered emissions – mostly the usual suspects: methane from cow burps, landfills, and coal mines; N2O from agriculture; and other small process or fugitive emissions. This broad scope is one of W-M’s strong points.

	No Policy	Policy
New
Old

The changes are rather dramatic, as you can see. The no-policy wheel looks like the old joke about playing Russian Roulette with an automatic. A tiny part of the difference is a baseline change, but most is not, as the report on the underlying modeling explains:

The new projections are considerably warmer than the 2003 projections, e.g., the median surface warming in 2091 to 2100 is 5.1°C compared to 2.4°C in the earlier study. Many changes contribute to the stronger warming; among the more important ones are taking into account the cooling in the second half of the 20th century due to volcanic eruptions for input parameter estimation and a more sophisticated method for projecting GDP growth which eliminated many low emission scenarios. However, if recently published data, suggesting stronger 20th century ocean warming, are used to determine the input climate parameters, the median projected warning at the end of the 21st century is only 4.1°C. Nevertheless all our simulations have a very small probability of warming less than 2.4°C, the lower bound of the IPCC AR4 projected likely range for the A1FI scenario, which has forcing very similar to our median projection.

I think the wheels are a cool idea, but I’d be curious to know how users respond to it. Do they cheat, and spin to get the outcome they hope for? Perhaps MIT should spice things up a bit, by programming an online version that gives users’ computers the BSOD if they roll a >7C world.

Hat tip to Travis Franck for pointing this out.