Black Swans

In this NYT editorial, Prof. Nassim Taleb gives his account of Black Swan theory as it relates to the events of September 11th. For a more in depth discussion, I highly recommend Dr Taleb's book The Black Swan: The Impact of the Highly Improbable. 

In essence, "black swans" are statistical outliers that, as a result of human predictive and computational limitations, are viewed as random or unthinkable events. To that effect incidents like 9/11 can, when treated epistemologically, rightly be considered as random events due to the critical lack of information of any prognostic value available prior to their occurrence. Certainly, with the correct information, such events could be predicted with relative ease, just as a storm might be predicted if one has knowledge of changes in barometric pressure over time. However, given the near infinitude of variables with which we must cope on a daily basis, predicting every possible event, even those resulting from potentially calculable human actions, is simply impractical. Indeed one could argue that, given the highly entropic nature of the human brain, such deliberative acts are even more difficult to predict on a global scale with any degree of certainty.

Nontheless, we as cognitive, rational actors attempt to do just this in an effort to make sense of the world around us. As such, from our limited purview we wind up classifying the great majority of possibile events that have yet to occur as improbable, thereby invoking a sense of randomness. This is particularly true for abstract events for which we lack the necessary information to predict. But, in reality, the occurrence of such events are no more random than any other event. They are simply perceived as such. As Taleb asserts, "almost all consequential events in history come from the unexpected." It is in hindsight that these events are generally "explained," through the lenses of history, science, philosophy, and the like. Randomness is, in effect, a matter of perspective.

Given that "black swans" are, by their very nature, unpredictable events, Taleb argues that U.S. national security efforts should focus on general protective measures rather than specific measures aimed at preventing events that have already occurred (such as planes crashing into a building). What general protective measures might be broadly applicable accross the spectrum of both the possible and predictable and the as of yet unconceived threats? Does the shift from protection towards resiliency add to or detract from these generalized defenses?

In light of the overwhelming amount of data with which we are constantly bombarded, what can we as a nation do to make better sense of it and to make better use of the intellectual resources availed to us in our efforts to more accurately predict future events, and so-called "black swans" in particular?

Much has been made, in recent years, of prediction markets like the Iowa Electronics Market and DARPAs Policy Analysis Market as means to more efficiently and more accurately predict future events. These speculative markets incentivize accurate predictions and form what might be considered the human analyst's version of distributed computing. Might not tools such as this be useful in predicting "black swans?"