Principia Discordia

Principia Discordia => Think for Yourself, Schmuck! => Topic started by: Cain on May 17, 2008, 03:57:59 PM

Title: Black Swan 101
Post by: Cain on May 17, 2008, 03:57:59 PM
I've been going through the book again with a pen and notepad, trying to get some of the ideas presented within to their most basic level, hopefully to create a short, scepticism guide for the tl/p/o;dr crowd.  Here is Part One.

=============================


What you don't know is far more relevant than what you do.

Being unexpected is what gives the Black Swan its impact.

'Experts' are no better at making predictions than anyone else – only at using technical jargon to maintain a plausible narrative.

Planning does not work, however allowing people to tinker and experiment and get rewarded for fortunate accidents does.

We learn specifically from a Black Swan event.  They do not help us generate metarules.

Recursive events (events that cause other events, feedback loops) are key to creating Black Swans and maintaining complexity.

There are few rewards or recognition for acts of prevention.

Platonicity:  what happens when we mistake the map for the territory, the abstract for the real, the neat concepts for the messy reality.  Potentially useful, but limited and have random and unforeseen side effects when they do fail (Anerstic Delusion).

Sterile scepticism: talk is cheap, talk about language problems, philosophy and pseudo-scepticism that plays around with word definitions to eliminate the problems of randomness are even cheaper.

Stories replace stories.  You need a better narrative to replace a bad or ineffective one.

The triplet of opacity:  everyone thinks they know more than they do, they assess history retroactively (hindsight is 20/20) and the over evaluation of factual information and "authoritative" persons.

History jumps, it does not crawl or steadily progress.

People will cluster around the same framework of analyses, no matter how non-factual or temporary these frameworks are.  This is a result of pathological categorization – wanting to put people in boxes to explain their behaviour.

Events fall into one of two domains: Extremistan or Mediocristan.  Events in Mediocristan are scalable and basically predictable.  Randomness is mild, it usually falls within the ranges of a Bell Curve.  In Extremistan, events are non-scalable and much more extreme.  Usually, Mediocristan events are those related to biology (for example, height) whereas those related to Extremistan are socially based (income, or book sales).

Inductive logic is for suckers: working from particulars to general rules is a recipe for disaster.  Past indicators are not a guarantee of future performance.

The Black Swan is a problem relative to expectation and lack of knowledge.  Those most certain of the future are the most likely to get tripped up and fall flat on their face.

Most people, given the choice, will choose to believe they live in Mediocristan, even when the evidence shows they do not.

Knowledge is domain specific: we have problems translating abstract classroom knowledge into real world applications.  Equally, we can easily navigate a social situation which, when presented as a logical problem, totally confounds us.

Absence of evidence is not evidence of absence.  No evidence of disease is not the same as there being no disease, and just because we do not know the function of something does not mean it has no function.

Corroborative facts cannot tell us what is true, only what is false.  Negative empiricism is superior to naïve empiricism, where one can 'prove' anything with facts.  A negative hypothesis is needed, falsification is needed.

Confirmation bias – we only look for evidence that supports our beliefs.

The narrative fallacy – we like to invent causes for events, even where none exist.  We put explanations and an element of causality into a sequence of facts.  Post hoc rationalization of events without meaning are common.

Narratives allow us to ignore the unusual.

Dopamine will lower your critical thinking abilities: you'll become susceptible to all sorts of fads, such as New Age mysticism, tarot cards or economic forecasts from a lack of it.

Information is costly to obtain – the more random the information the harder it is to store.  Hence stories allow us to store more information, but with the downside of a loss of appreciation of the true randomness present.

Both art and science are symptomatic of our need to reduce complexity and inflict basic order on human existence.

People can and will hold incompatible and disparate beliefs based on the same factual evidence.

In the absence of other information, we will rely on stereotypes or even complete nonsense to explain events.

There are two types of Black Swan, the narrated and the truly unexpected.  The narrated Black Swan is an overrated and hyped rare event, the true Black Swan is not talked about.

We are biologically constructed to seek small, yet repeatable and sustainable rewards. 

The Black Swan is an asymmetry of consequences – either positive or negative.

You can either be a sucker, and run blindly into the danger you didn't know was there – or you can slowly bleed in the face of dangers you knew existed while hoping that it will lead to a pay-off that is worth it.

Silent evidence: often we are mislead by the sample in any experiment, because we do not take into account those who failed to be counted (the dead supplicants fallacy).

The survivor bias: we work from the fact someone survived to find the secret of their success, but in reality the fact that they survived is the secret of their success.  People do not write about or promote knowledge of their failures.

The Ludic fallacy: life is not like a game.  The sources of uncertainty are not well defined, the "rules of the game" are not totally apparent.  If you treat life like a game, where you know all the rules, the unknown and unexpected will hit you like a truck.
Title: Re: Black Swan 101
Post by: Jasper on May 17, 2008, 06:03:31 PM
Not only is that a pile of scrittens, many of them seem worth three average one sentence memebombs.

Going on the eBook, this one. :)
Title: Re: Black Swan 101
Post by: e on May 17, 2008, 06:16:37 PM
Quote from: Cain on May 17, 2008, 03:57:59 PM
The Ludic fallacy: life is not like a game.  The sources of uncertainty are not well defined, the "rules of the game" are not totally apparent.  If you treat life like a game, where you know all the rules, the unknown and unexpected will hit you like a truck.

But...  But...    Lots of games have "elements of chance" in them.  That's part of what makes them fun!
Title: Re: Black Swan 101
Post by: Triple Zero on May 17, 2008, 06:29:42 PM
Cain: awesome. now pata-something (forgot his name) can read up on this :)

Felix: not to detract from Cain's list, but personally i found that "Fooled by Randomness"  by the same author treats pretty much the same subject, but in a littlebit more technical/objective-ish manner. the Black Swan really annoyed me at times, claiming untrue things. Fooled by Randomness doesnt make sweeping generalizations, but uses example stories, which somehow give the feeling of a more balanced view. You can also read both, of course.

TheStripedOne: yes, but the mistake in thinking that life is like a game, is that even the elements of chance in a game are controlled. you have the roll of the dice, you have the pieces according to their rules, even if it's random. the difference with life is that uncertainty can break any of the rules, anytime.

it'd be like playing chess, and suddenly this huge robot tank thing warhammer figure enters the board, obliterating your pieces in one turn.

except that in life, you wouldn't complain to the other for cheating, but thanks to the narrative fallacy, you will claim you couldn't have foreseen your defeat, as you are a chess player, not warhammer.

then, the next time you play a game you aren't going to take any chances this time, and have a similar warhammer figure at the ready to defend yourself--only to get squirted in the face with a supersoaker.
Title: Re: Black Swan 101
Post by: Cainad (dec.) on May 17, 2008, 08:51:21 PM
 :lol: @ supersoaker

The only game I can think of with a real Black Swan randomness element would be 1000 Blank White Cards... and even then it's limited by the imagination of the people involved.

I actually wrote a thing that used a billiards table as a metaphor for "the world." You can't expect to get the exact results you want by hitting the balls in the exact right way, because in real life the balls are all different weights, the table is lumpy and curved, and the other players are allowed to jab you with their sticks while you're lining up a shot.
Title: Re: Black Swan 101
Post by: Triple Zero on May 17, 2008, 08:57:19 PM
i read an amazing thing about numerically simulating billiards games (it could even have been from the Black Swan, can't remember). after about 18 shots have been made, you need such accurate information about the initial conditions of the system that any breath of the players or lunar position (or whatever) becomes relevant. (i might have confused this quote a littlebit, but you get the point, i hope)
Title: Re: Black Swan 101
Post by: Cainad (dec.) on May 17, 2008, 09:01:01 PM
I think I get it; you mean that as a system progresses, the amount of information needed to accurately predict future outcomes increases rapidly?
Title: Re: Black Swan 101
Post by: Golden Applesauce on May 17, 2008, 09:59:29 PM
Uncertainty grows exponentially - but that's getting into chaos theory.  To predict what happens to the billiards game with a given accuracy you need to know the masses of the balls, friction, force of the shot, angle of the shot, where on the ball it connected, where the balls are, where all the lumps on the table are, etc accurately.  There will always be some uncertainty, but if you measure things accurately enough you can deal with it.  But the amount of uncertainty in the system magnifies every time you do something to it.  So we might know the position of the 8 ball to the nearest .1 mm, but then when you hit it you'll only be able to predict its position to the nearest 1 mm or something (because depending on where it is it gets hit at a slightly different angle and goes over bumps in the table differently.)  No matter how accurately you know the initial conditions, after enough shots there will come a time when you can pretty confidently say that you think most of the balls are either on the table or in the pockets and not much else.
Title: Re: Black Swan 101
Post by: Dr. Pataphoros, SpD on May 19, 2008, 05:41:51 AM
If this is meant to be a "101" for whatever a Black Swan is, you may want to restructure it into a more coherent format.  An introduction and quick definition of repeatedly used terms followed by categorical breakdown of what you're getting at, maybe.  Right now it reads as a series of one-liners with a few commonalities, jumping from idea to idea. 

Coming into this post with no clue what I was getting "101'd" on I feel like I'm still missing some crucial unifying element to tie this pile of information together.  I get a sort-of general sense where all of this stuff is going, but there's so much disjointed stuff in there that I'd rather not make any assumptions.

Or maybe I'm just totally not getting it.

Title: Re: Black Swan 101
Post by: Golden Applesauce on May 19, 2008, 05:57:30 AM
I like the one liners style of information; it feels like a collage.  Much better than wall-of-text.

But what is a Black Swan event anyway?
Title: Re: Black Swan 101
Post by: Cainad (dec.) on May 19, 2008, 06:18:17 AM
Quote from: Golden Applesauce on May 19, 2008, 05:57:30 AM
I like the one liners style of information; it feels like a collage.  Much better than wall-of-text.

But what is a Black Swan event anyway?

Ooh, lemme try:

A Black Swan event is an unpredicted--and arguably unpredictable--occurrence that causes a significant change in the system in which it occurs. For our purposes, Black Swans occur mostly in social, economic, and political systems, rather than natural phenomena. One characteristic of a Black Swan is that it appears to be so improbable that no one suspects it could happen, and what gives it its profound effect is the fact that no one (or at least none of the important people) sees it coming and thus no one is prepared.

A classic example is the Great Depression of the early twentieth century. While we, in retrospect, can look back and see the mistakes that caused it, no one at the time predicted such a sudden, massive drop in stock prices. In fact, based on the recorded information they had at the time, it would have been considered so unlikely that it might as well have been impossible. This is because the information they had was entirely based on trends, and the trend had been, for several years, extremely positive market growth. The critical error they made, and which many analysts and forecasters continue to make, was the assumption that the stock market would follow the same pattern as natural phenomena; that is, they figured that any drop in the stock market would simply be a fluke, with little real consequence, just like the occasional birth of an albino or giant has little significant impact on the gene pool of a species.

But when dealing with things like the stock market, sudden drops are not flukes; they become the real forces of change in the system. One slight drop may have been a harmless fluke, but when the social aspect (namely, panic) is factored in, the result is a massive, catastrophic change that leaves all the analysts and forecasters flabbergasted and screaming, "The chances were on in a billion! We couldn't have known!" as they are tossed out the office windows.

This is only an illustrative example and I'm sure someone could go in to far greater depth about it, but I hope it provides some insight as to what a Black Swan event is supposed to be.
Title: Re: Black Swan 101
Post by: LMNO on May 19, 2008, 05:29:13 PM
I like the Turkey metaphor.

A turkey is born, and is well fed for 1000 days.

The turkey looks at it's life, and says, "I have been extremely well cared-for.  My owners must love me.  I am the luckiest turkey in the world.  Based upon past trends, I can expect to live a long and happy life."


Then, the fourth Thursday in November rolls around...

Title: Re: Black Swan 101
Post by: Cain on May 19, 2008, 05:30:42 PM
Yeah, thats inductive logic (which incidentally is the basis of most International Relations theory) for you.
Title: Re: Black Swan 101
Post by: Dr. Pataphoros, SpD on May 19, 2008, 06:42:51 PM
It seems to me that the primary basis of the Black Swan is rests largely on indeterminism.  We can't know everything, ergo we can't be sure of anything.  I'd be interested to know how free will (or lack thereof) fits into the model.
Title: Re: Black Swan 101
Post by: Jasper on May 19, 2008, 08:37:03 PM
People operating on the meme of free will tend to behave accordingly and vice-versa.

Nuff said.
Title: Re: Black Swan 101
Post by: Bebek Sincap Ratatosk on May 19, 2008, 09:19:12 PM
Quote from: Padre Pataphoros on May 19, 2008, 06:42:51 PM
It seems to me that the primary basis of the Black Swan is rests largely on indeterminism.  We can't know everything, ergo we can't be sure of anything.  I'd be interested to know how free will (or lack thereof) fits into the model.

Well, Free Will could be included in the model, but I think that immaterial. The premise rests, I think, on the inability of any human to grok all of the variables on any given topic. So, while free will may be a variable, so too is "Earthquake in Columbus Ohio"... Free Will might make it on to the model, but since the probability of an earthquake in Columbus appears very low based on historical trends, it may not make it on to the model. Thus when the fault line under my house goes for a walk, lots of people will get a surprise and their models will crumble.

Occam's Razor can't shave a Black Swan.

Title: Re: Black Swan 101
Post by: PeregrineBF on May 20, 2008, 07:21:17 AM
Indeterminism/free will:
The Free Will Theorem (http://"http://arxiv.org/abs/quant-ph/0604079")
Reply to Comments of Bassi, Ghirardi, and Tumulka on the Free Will Theorem (http://"http://arxiv.org/abs/quant-ph/0701016")
On Adler's "Conway-Kochen Twin Argument" (http://"http://arxiv.org/abs/quant-ph/0610147")
I suggest reading at least the first one, it's not hard to understand and is quite interesting.
Title: Re: Black Swan 101
Post by: Dr. Pataphoros, SpD on May 20, 2008, 03:22:41 PM
Quote from: Ratatosk on May 19, 2008, 09:19:12 PM
Quote from: Padre Pataphoros on May 19, 2008, 06:42:51 PM
It seems to me that the primary basis of the Black Swan is rests largely on indeterminism.  We can't know everything, ergo we can't be sure of anything.  I'd be interested to know how free will (or lack thereof) fits into the model.

Well, Free Will could be included in the model, but I think that immaterial.


Not immaterial at all.  If free will exists then so does the Black Swan, because that means that people can and will make choices independent of past events or influences and there is no way for us to predict them.  If free will doesn't exist then we are all a bunch of squishy robots making choices consistent with our (incredibly complex) programming, which is based solely on past events and influences.  If that is the case, then given enough computing power we eventually could know how someone will react to a given situation by analyzing every last bit of data available about a person and deciding what they will do in that circumstance.  Likewise we could analyze the data present in the physical environment and be able to accurately determine future events that account for the environment and the people acting within that environment.

The turkey metaphor, and other similar "proofs" don't work, because the outcome is knowable and predictable in advance to everyone except the turkey.  Just because the turkey is ignorant does not mean it is impossible to know what will happen next.  With some deduction and reasoning a human in the turkey's place will likely figure things out sooner or later.  To say that we cannot possibly know every variable is likewise misleading, since we don't need to know every last variable, just the ones most likely to have an impact.  Even then we don't usually need all of those to make a reasonably accurate prediction. 

After all is said and done, whether the prediction turned out to be innacurate is likewise unimportant.  We MUST look toward the future and attempt to discern what is ahead so that we can plan for eventualities.  If, during that process, we get excited about the possibilities we see on the horizon then that will only serve to stimulate and motivate us towards that outcome.  If a scientist predicts we'll be immortal by 20XX he's probably not making that supposition without at least a significant amount of the variables in question either known or narrowed down to a certain managable degree.

So, in the end, I find the concept of the Black Swan to be ultimately unhelpful, except in the case where a scientific singularity does occur.  If that happens all bets will be off anyhow.  At least, that's my prediction.
Title: Re: Black Swan 101
Post by: hooplala on May 20, 2008, 04:08:03 PM
Quote from: Cain on May 17, 2008, 03:57:59 PMThere are two types of Black Swan, the narrated and the truly unexpected.  The narrated Black Swan is an overrated and hyped rare event, the true Black Swan is not talked about.

Sorry to pop in here without having read the rest of the thread yet... but I find the above quote very interesting.

What would be an example of a narrated Black Swan event, and a true Black Swan event?
Title: Re: Black Swan 101
Post by: Dr. Pataphoros, SpD on May 20, 2008, 04:19:25 PM
Quote from: Hoopla on May 20, 2008, 04:08:03 PM
Quote from: Cain on May 17, 2008, 03:57:59 PMThere are two types of Black Swan, the narrated and the truly unexpected.  The narrated Black Swan is an overrated and hyped rare event, the true Black Swan is not talked about.

Sorry to pop in here without having read the rest of the thread yet... but I find the above quote very interesting.

What would be an example of a narrated Black Swan event, and a true Black Swan event?

As far as I can determine:
Narrated = Bird Flu/SARS/Mad Cow/West Nile (pick one) will kill us all! 
True = Aliens come from space and abduct George W. in the final days of his presidency and make him President of the Galaxy.
Title: Re: Black Swan 101
Post by: hooplala on May 20, 2008, 04:21:47 PM
Quote from: Padre Pataphoros on May 20, 2008, 04:19:25 PM
Quote from: Hoopla on May 20, 2008, 04:08:03 PM
Quote from: Cain on May 17, 2008, 03:57:59 PMThere are two types of Black Swan, the narrated and the truly unexpected.  The narrated Black Swan is an overrated and hyped rare event, the true Black Swan is not talked about.

Sorry to pop in here without having read the rest of the thread yet... but I find the above quote very interesting.

What would be an example of a narrated Black Swan event, and a true Black Swan event?

As far as I can determine:
Narrated = Bird Flu/SARS/Mad Cow/West Nile (pick one) will kill us all! 
True = Aliens come from space and abduct George W. in the final days of his presidency and make him President of the Galaxy.

I've been predicting that last one for close to nine years, so it can hardly count...
Title: Re: Black Swan 101
Post by: Bebek Sincap Ratatosk on May 20, 2008, 05:02:30 PM
Quote from: Padre Pataphoros on May 20, 2008, 03:22:41 PM
Quote from: Ratatosk on May 19, 2008, 09:19:12 PM
Quote from: Padre Pataphoros on May 19, 2008, 06:42:51 PM
It seems to me that the primary basis of the Black Swan is rests largely on indeterminism.  We can't know everything, ergo we can't be sure of anything.  I'd be interested to know how free will (or lack thereof) fits into the model.

Well, Free Will could be included in the model, but I think that immaterial.


Not immaterial at all.  If free will exists then so does the Black Swan, because that means that people can and will make choices independent of past events or influences and there is no way for us to predict them. 

Well it would certainly indicate a higher likelihood that humans might be directly responsible for sombunal Black Swans.

Quote
If free will doesn't exist then we are all a bunch of squishy robots making choices consistent with our (incredibly complex) programming, which is based solely on past events and influences.  If that is the case, then given enough computing power we eventually could know how someone will react to a given situation by analyzing every last bit of data available about a person and deciding what they will do in that circumstance.  Likewise we could analyze the data present in the physical environment and be able to accurately determine future events that account for the environment and the people acting within that environment.

Well, this is where I think most Black Swans come from anyway... people get confused between the map: "We know how someone will react" "we can analyze the data and determine future events" and the territory: "What actually happens". It's the bits of the territory that aren't on the map which give rise to Black Swans, I think maybe.
Quote
The turkey metaphor, and other similar "proofs" don't work, because the outcome is knowable and predictable in advance to everyone except the turkey.  Just because the turkey is ignorant does not mean it is impossible to know what will happen next.  With some deduction and reasoning a human in the turkey's place will likely figure things out sooner or later.  To say that we cannot possibly know every variable is likewise misleading, since we don't need to know every last variable, just the ones most likely to have an impact.  Even then we don't usually need all of those to make a reasonably accurate prediction. 

I disagree... a Human, in the turkey's position seems unlikely to figure out that he's lunch... unless he can speak to his captors and they tell him. If we take a baby, keep it for its entire life in a 30 x 30 barn, feed it well and care for it... I think it highly unlikely that it will say to itself "I think I'ma gonna be dinner!!! OSHI!" I think it far more likely that it will be very surprised when someone comes in with an axe.

Title: Re: Black Swan 101
Post by: Cain on May 20, 2008, 05:17:43 PM
Quote from: Padre Pataphoros on May 20, 2008, 04:19:25 PM
Quote from: Hoopla on May 20, 2008, 04:08:03 PM
Quote from: Cain on May 17, 2008, 03:57:59 PMThere are two types of Black Swan, the narrated and the truly unexpected.  The narrated Black Swan is an overrated and hyped rare event, the true Black Swan is not talked about.

Sorry to pop in here without having read the rest of the thread yet... but I find the above quote very interesting.

What would be an example of a narrated Black Swan event, and a true Black Swan event?

As far as I can determine:
Narrated = Bird Flu/SARS/Mad Cow/West Nile (pick one) will kill us all! 
True = Aliens come from space and abduct George W. in the final days of his presidency and make him President of the Galaxy.

Basically yes.  Its when really, really unlikely threats are hyped.  Like your examples, pretty much!
Title: Re: Black Swan 101
Post by: hooplala on May 20, 2008, 05:18:35 PM
2012?


Srsly though, this is very interesting, and I may have to go pick up this book (and the one 000 mentioned) very soon.
Title: Re: Black Swan 101
Post by: Cain on May 20, 2008, 05:19:15 PM
Ratatosk, Taleb calls the difference between the map and the territory the Platonic gap, and outright states "this is where Black Swans happen".  Its the difference between nice, conceptual, clearly dinlineated ideas about reality and messy reality itself.

So yes, pretty much.  He just has a nicer term for it.
Title: Re: Black Swan 101
Post by: Cain on May 20, 2008, 05:19:53 PM
Quote from: Hoopla on May 20, 2008, 05:18:35 PM
2012?


Srsly though, this is very interesting, and I may have to go pick up this book (and the one 000 mentioned) very soon.

Possibly, although its only hyped in fringe circles.  Terrorism, serial killers or shark attacks, would be better ones.
Title: Re: Black Swan 101
Post by: hooplala on May 20, 2008, 05:23:45 PM
What about things like violence in video games and television?  Or is that too broad?
Title: Re: Black Swan 101
Post by: Cain on May 20, 2008, 05:25:30 PM
Quote from: Hoopla on May 20, 2008, 05:23:45 PM
What about things like violence in video games and television?  Or is that too broad?

Far too broad.  They're not even supposedly unexpected events.
Title: Re: Black Swan 101
Post by: Bebek Sincap Ratatosk on May 20, 2008, 05:32:47 PM
Quote from: Cain on May 20, 2008, 05:19:15 PM
Ratatosk, Taleb calls the difference between the map and the territory the Platonic gap, and outright states "this is where Black Swans happen".  Its the difference between nice, conceptual, clearly dinlineated ideas about reality and messy reality itself.

So yes, pretty much.  He just has a nicer term for it.

Ah, very good... as the gap between the Platonic forms and the reality... nice!
Title: Re: Black Swan 101
Post by: hooplala on May 20, 2008, 05:39:33 PM
Quote from: Cain on May 20, 2008, 05:25:30 PM
Quote from: Hoopla on May 20, 2008, 05:23:45 PM
What about things like violence in video games and television?  Or is that too broad?

Far too broad.  They're not even supposedly unexpected events.

Gotcha.
Title: Re: Black Swan 101
Post by: Cain on May 20, 2008, 05:41:25 PM
Quote from: Ratatosk on May 20, 2008, 05:32:47 PM
Quote from: Cain on May 20, 2008, 05:19:15 PM
Ratatosk, Taleb calls the difference between the map and the territory the Platonic gap, and outright states "this is where Black Swans happen".  Its the difference between nice, conceptual, clearly dinlineated ideas about reality and messy reality itself.

So yes, pretty much.  He just has a nicer term for it.

Ah, very good... as the gap between the Platonic forms and the reality... nice!

Carl von Clausewitz called it 'friction' when it occured during war.  And it leads directly into the Aneristic Delusion, of course.
Title: Re: Black Swan 101
Post by: Bebek Sincap Ratatosk on May 20, 2008, 05:44:06 PM
Quote from: Cain on May 20, 2008, 05:25:30 PM
Quote from: Hoopla on May 20, 2008, 05:23:45 PM
What about things like violence in video games and television?  Or is that too broad?

Far too broad.  They're not even supposedly unexpected events.

Instead maybe:
The Earthquake in China appears to have lead to a rapid explosion of Information as individuals with cell phone etc outpaced the government in passing Information about the incident. This could be a major Black Swan for the government, since once Information is flowing freely, it may be nearly impossible to stop. Further, such an event might highlight information that the government would normally have redacted (like why many of the collapsed buildings were schools...)
Title: Re: Black Swan 101
Post by: Dr. Pataphoros, SpD on May 20, 2008, 05:45:20 PM
Quote from: Ratatosk on May 20, 2008, 05:02:30 PM

Well it would certainly indicate a higher likelihood that humans might be directly responsible for sombunal Black Swans.


Since identifying a Black Swan requires a human observer then I would say that we're responsible for all of them, not just some.  An event that is unexpected implies someone is expecting something else.

Quote

Well, this is where I think most Black Swans come from anyway... people get confused between the map: "We know how someone will react" "we can analyze the data and determine future events" and the territory: "What actually happens". It's the bits of the territory that aren't on the map which give rise to Black Swans, I think maybe.

To take your metaphor and run with it: a map is a graphical depiction of the Earth's surface as viewed from above.  It's an approximation based on giving us information we need (elevation, terrain features, distance between points, etc..) and eschewing the details we don't need (like the correct colors and textures of the terrain, precise down-to-the-inch accuracy, locations of transient objects, etc...).  The map cannot tell us what type of events will occur along our trip, but we don't need it to--that's not its purpose, nor would anyone expect to use it to such an end.  

In the same way, it is irrelevant (and unhelpful) to the planning of our future to consider the unforseeable.  If we worry about whether or not our projections are likely to become true they'll have less chance to eventually be true, "on time" or otherwise.  I guess what I'm saying is the map is more useful to our purposes than the territory.  You can navigate territory without a map, but it's hard.  You have no way to guage your progress if you don't have a reference to the big picture.

Quote

I disagree... a Human, in the turkey's position seems unlikely to figure out that he's lunch... unless he can speak to his captors and they tell him. If we take a baby, keep it for its entire life in a 30 x 30 barn, feed it well and care for it... I think it highly unlikely that it will say to itself "I think I'ma gonna be dinner!!! OSHI!" I think it far more likely that it will be very surprised when someone comes in with an axe.


You've got me there, but I still find the analogy irrelevant.  Just because the subject of the execution doesn't know its coming doesn't mean outside observers can't conclude what will happen--it's all a matter of perspective and available information.  Babies and Animals are not truly capable of higher reasoning, so it's silly to use them.  They can hardly follow day to day events beyond the assurance of their own survival and will of course be duped by the evil baby-and-turkey-eating farmer.

There are, however, a million ways for a normal person to figure out what is going on beyond the walls of the barn, and any intelligent being will be inquisitive.  Unless he's being held under duress he'll find answers, and if he is being held under duress he'll already have one of them.

EDIT: Mai spellign iz nao gudr
Title: Re: Black Swan 101
Post by: hooplala on May 20, 2008, 05:46:19 PM
Quote from: Ratatosk on May 20, 2008, 05:44:06 PM
Quote from: Cain on May 20, 2008, 05:25:30 PM
Quote from: Hoopla on May 20, 2008, 05:23:45 PM
What about things like violence in video games and television?  Or is that too broad?

Far too broad.  They're not even supposedly unexpected events.

Instead maybe:
The Earthquake in China appears to have lead to a rapid explosion of Information as individuals with cell phone etc outpaced the government in passing Information about the incident. This could be a major Black Swan for the government, since once Information is flowing freely, it may be nearly impossible to stop. Further, such an event might highlight information that the government would normally have redacted (like why many of the collapsed buildings were schools...)

This is fascinating.

Cain, is this theory taken seriously by many scientists?
Title: Re: Black Swan 101
Post by: hooplala on May 20, 2008, 05:48:39 PM
Quote from: Padre Pataphoros on May 20, 2008, 05:45:20 PM
Just because the subject of the execution doesn't know its coming doesn't mean outside observers can't conclude what will happen--it's all a matter of perspective and available information.  Babies and Animals are not truly capable of higher reasoning, so it's silly to use them.  They can hardly follow day to day events beyond the assurance of their own survival and will of course be duped by the evil baby-and-turkey-eating farmer.
There are, however, a million ways for a normal person to figure out what is going on beyond the walls of the barn, and any intelligent being will be inquisitive.  Unless he's being held under duress he'll find answers, and if he is being held under duress he'll already have one of them.

You seem to be forgetting about self-imposed blinders.
Title: Re: Black Swan 101
Post by: Dr. Pataphoros, SpD on May 20, 2008, 05:56:55 PM
Quote from: Hoopla on May 20, 2008, 05:48:39 PM
Quote from: Padre Pataphoros on May 20, 2008, 05:45:20 PM
Just because the subject of the execution doesn't know its coming doesn't mean outside observers can't conclude what will happen--it's all a matter of perspective and available information.  Babies and Animals are not truly capable of higher reasoning, so it's silly to use them.  They can hardly follow day to day events beyond the assurance of their own survival and will of course be duped by the evil baby-and-turkey-eating farmer.
There are, however, a million ways for a normal person to figure out what is going on beyond the walls of the barn, and any intelligent being will be inquisitive.  Unless he's being held under duress he'll find answers, and if he is being held under duress he'll already have one of them.

You seem to be forgetting about self-imposed blinders.

The process of minimizing future uncertainty is synonymous with removing those blinders.  Anyone who keeps them on is only preparing themselves for failure.
Title: Re: Black Swan 101
Post by: Bebek Sincap Ratatosk on May 20, 2008, 05:57:59 PM
Quote from: Padre Pataphoros on May 20, 2008, 05:45:20 PM
Quote from: Ratatosk on May 20, 2008, 05:02:30 PM

Well it would certainly indicate a higher likelihood that humans might be directly responsible for sombunal Black Swans.


Since identifying a Black Swan requires a human observer then I would say that we're responsible for all of them, not just some.  An event that is unexpected implies someone is expecting something else.


Eh? I thought you were arguing that Free Will was necessary for Black Swans... not that human perception was necessary to categorize Black Swans... which are you arguing here?

Quote
Quote

Well, this is where I think most Black Swans come from anyway... people get confused between the map: "We know how someone will react" "we can analyze the data and determine future events" and the territory: "What actually happens". It's the bits of the territory that aren't on the map which give rise to Black Swans, I think maybe.

To take your metaphor and run with it: a map is a graphical depiction of the Earth's surface as viewed from above.  It's an approximation based on giving us information we need (elevation, terrain features, distance between points, etc..) and eschewing the details we don't need (like the correct colors and textures of the terrain, precise down-to-the-inch accuracy, locations of transient objects, etc...).  The map cannot tell us what type of events will occur along our trip, but we don't need it to--that's not its purpose, nor would anyone expect to use it to such an end.  

In the same way, it is irrelevant (and unhelpful) to the planning of our future to consider the unforseeable.  If we worry about whether or not our prjections are likely to become true they'll have less chance to eventually be true, "on time" or otherwise.  I guess what I'm saying is the map is more useful to our purposes than the territory.  You can navigate territory without a map, but it's hard.  You have no way to guage your progress if you don't have a reference to the big picture.

See, though, its in the "details we don't need" where the Black Swan is gonna make a showing. For example, you may examine all the available maps and determine the best location for your new Nuclear Lab. However, unknown to you, an as of yet, undiscovered fault line lies beneath your facility. It's not on the Map, because the Map is not the Territory... the map is just a representation of our limited knowledge and our preferred categorization of data which we consider important.

However, i think you're trying to apply this incorrectly, I don't think Taleb is making the argument that we shouldn't use models to make predictions... rather I think he's saying that we cannot KNOW that our prediction is right, due to the limited nature of both our perception and the model. We're discussing the relativity of the value of such predictions. Also, it wouldn't hurt scientists to be a little less certain on occasion ;-)

Quote
Quote

I disagree... a Human, in the turkey's position seems unlikely to figure out that he's lunch... unless he can speak to his captors and they tell him. If we take a baby, keep it for its entire life in a 30 x 30 barn, feed it well and care for it... I think it highly unlikely that it will say to itself "I think I'ma gonna be dinner!!! OSHI!" I think it far more likely that it will be very surprised when someone comes in with an axe.


You've got me there, but I still find the analogy irrelevant.  Just because the subject of the execution doesn't know its coming doesn't mean outside observers can't conclude what will happen--it's all a matter of perspective and available information.  Babies and Animals are not truly capable of higher reasoning, so it's silly to use them.  They can hardly follow day to day events beyond the assurance of their own survival and will of course be duped by the evil baby-and-turkey-eating farmer.
There are, however, a million ways for a normal person to figure out what is going on beyond the walls of the barn, and any intelligent being will be inquisitive.  Unless he's being held under duress he'll find answers, and if he is being held under duress he'll already have one of them.


<i>it's all a matter of perspective and available information</i>

Yes, it is. We as humans have limited perspective and limited available information about reality. It's useful, but let's not assume that we're observers outside the barn, or that we ever could be observers from outside the barn.

Besides, the other turkeys outside the barn aren't gonna have a better clue than the one inside, are they?
Title: Re: Black Swan 101
Post by: Bebek Sincap Ratatosk on May 20, 2008, 05:58:55 PM
Quote from: Padre Pataphoros on May 20, 2008, 05:56:55 PM
Quote from: Hoopla on May 20, 2008, 05:48:39 PM
Quote from: Padre Pataphoros on May 20, 2008, 05:45:20 PM
Just because the subject of the execution doesn't know its coming doesn't mean outside observers can't conclude what will happen--it's all a matter of perspective and available information.  Babies and Animals are not truly capable of higher reasoning, so it's silly to use them.  They can hardly follow day to day events beyond the assurance of their own survival and will of course be duped by the evil baby-and-turkey-eating farmer.
There are, however, a million ways for a normal person to figure out what is going on beyond the walls of the barn, and any intelligent being will be inquisitive.  Unless he's being held under duress he'll find answers, and if he is being held under duress he'll already have one of them.

You seem to be forgetting about self-imposed blinders.

The process of minimizing future uncertainty is synonymous with removing those blinders.  Anyone who keeps them on is only preparing themselves for failure.

I think you may misunderstand the metaphor...
Title: Re: Black Swan 101
Post by: Dr. Pataphoros, SpD on May 20, 2008, 06:45:29 PM
Quote from: Ratatosk on May 20, 2008, 05:58:55 PM
Quote from: Padre Pataphoros on May 20, 2008, 05:56:55 PM

The process of minimizing future uncertainty is synonymous with removing those blinders.  Anyone who keeps them on is only preparing themselves for failure.


I think you may misunderstand the metaphor...

I think I misunderstand most of this concept entirely.  I'm still trying to discover why it is useful for us to catagorize something as unknowable when we don't even know what it is that we don't know.  This whole Black Swan business seems to be an exercise in needless circular thinking.

Quote from: Ratatosk on May 20, 2008, 05:57:59 PM
Quote from: Padre Pataphoros on May 20, 2008, 05:45:20 PM

Since identifying a Black Swan requires a human observer then I would say that we're responsible for all of them, not just some.  An event that is unexpected implies someone is expecting something else.



Eh? I thought you were arguing that Free Will was necessary for Black Swans... not that human perception was necessary to categorize Black Swans... which are you arguing here?


This started out with me wondering what impact free will had on a Black Swan situation.  I concluded that if non-deterministic free will is true then All of Everything Yet To Occur is a Black Swan.  We can't know everything, so we can never be sure of anything. 

Regardless of what constitutes a Black Swan, one cannot exist without an intelligent observer with a presupposed idea of what will occur.  If there is no one to have an expectation, then there cannot be an unexpected event.  I could wax tautological further, but I think you take my meaning.

Quote

See, though, its in the "details we don't need" where the Black Swan is gonna make a showing. For example, you may examine all the available maps and determine the best location for your new Nuclear Lab. However, unknown to you, an as of yet, undiscovered fault line lies beneath your facility. It's not on the Map, because the Map is not the Territory... the map is just a representation of our limited knowledge and our preferred categorization of data which we consider important.

However, i think you're trying to apply this incorrectly, I don't think Taleb is making the argument that we shouldn't use models to make predictions... rather I think he's saying that we cannot KNOW that our prediction is right, due to the limited nature of both our perception and the model. We're discussing the relativity of the value of such predictions. Also, it wouldn't hurt scientists to be a little less certain on occasion ;-)


Okay, I see now what you're getting at.  The problem is, as I keep saying, it seems useless to have a complicated metaphorical label just to tell us we can't tell the future for really real.  Unless there is a level of depth I am continually missing.

Quote

it's all a matter of perspective and available information

Yes, it is. We as humans have limited perspective and limited available information about reality. It's useful, but let's not assume that we're observers outside the barn, or that we ever could be observers from outside the barn.

Besides, the other turkeys outside the barn aren't gonna have a better clue than the one inside, are they?

Depends on how many Thanksgivings they've lived through.

Title: Re: Black Swan 101
Post by: Cainad (dec.) on May 20, 2008, 06:53:20 PM
Quote from: Padre Pataphoros on May 20, 2008, 06:45:29 PM

Quote from: Ratatosk on May 20, 2008, 05:57:59 PM
Quote from: Padre Pataphoros on May 20, 2008, 05:45:20 PM

Since identifying a Black Swan requires a human observer then I would say that we're responsible for all of them, not just some.  An event that is unexpected implies someone is expecting something else.



Eh? I thought you were arguing that Free Will was necessary for Black Swans... not that human perception was necessary to categorize Black Swans... which are you arguing here?


This started out with me wondering what impact free will had on a Black Swan situation.  I concluded that if non-deterministic free will is true then All of Everything Yet To Occur is a Black Swan.  We can't know everything, so we can never be sure of anything. 


Good god, no. Most of All of Everything Yet To Occur falls within a standard deviation bell curve of probability. A Black Swan is an event in which the rare, outrageous exception turns out to be more significant than the more probable events, which tend to be moderate.

Imagine that, while playing a Monopoly game, someone gets a Chance card that says "You win 5 million dollars." It would be completely unexpected (because the real game of Monopoly has no such card), there would only be one small chance for it to occur, but it would nevertheless be more significant than every other event in the game put together.
Title: Re: Black Swan 101
Post by: Cain on May 20, 2008, 06:58:35 PM
Also requires a large impact to be a Black Swan.

Winning $5 million in imaginary money is not a Black Swan.

Losing $20 trillion in real money because a very very unlikely event just happened, on the other hand, is.
Title: Re: Black Swan 101
Post by: Bebek Sincap Ratatosk on May 20, 2008, 07:04:28 PM
Quote from: Cain on May 20, 2008, 06:58:35 PM
Also requires a large impact to be a Black Swan.

Winning $5 million in imaginary money is not a Black Swan.

Losing $20 trillion in real money because a very very unlikely event just happened, on the other hand, is.

LOL

True enough in some sense. I think, Padre Pataphoros, that you may be confusing the value of the concept by misapplication. I don't think anyone has argued that Black Swans mean that all predictive data is useless, or that it has any effect on the nature of scientific prediction. Rather, its a philosophical point which focuses on the unknowable, which, throughout history seems to have been where the largest changes have wandered into reality from.
Title: Re: Black Swan 101
Post by: Bu🤠ns on May 20, 2008, 07:05:21 PM
Quote from: Cain on May 20, 2008, 06:58:35 PM
Also requires a large impact to be a Black Swan.

Winning $5 million in imaginary money is not a Black Swan.

Losing $20 trillion in real money because a very very unlikely event just happened, on the other hand, is.

that would be winning or losing 20 trillion, right?  because a Black Swan isn't necessarily a good or bad happening, right?
Title: Re: Black Swan 101
Post by: Cain on May 20, 2008, 07:07:53 PM
Quote from: burnstoupee on May 20, 2008, 07:05:21 PM
Quote from: Cain on May 20, 2008, 06:58:35 PM
Also requires a large impact to be a Black Swan.

Winning $5 million in imaginary money is not a Black Swan.

Losing $20 trillion in real money because a very very unlikely event just happened, on the other hand, is.

that would be winning or losing 20 trillion, right?  because a Black Swan isn't necessarily a good or bad happening, right?

Yes.  Certain scientific discoveries or inventions could be considered black swans too.
Title: Re: Black Swan 101
Post by: Bebek Sincap Ratatosk on May 20, 2008, 07:08:29 PM
Quote from: Cain on May 20, 2008, 07:07:53 PM
Quote from: burnstoupee on May 20, 2008, 07:05:21 PM
Quote from: Cain on May 20, 2008, 06:58:35 PM
Also requires a large impact to be a Black Swan.

Winning $5 million in imaginary money is not a Black Swan.

Losing $20 trillion in real money because a very very unlikely event just happened, on the other hand, is.

that would be winning or losing 20 trillion, right?  because a Black Swan isn't necessarily a good or bad happening, right?

Yes.  Certain scientific discoveries or inventions could be considered black swans too.

Vulcanization, perhaps? LSD?
Title: Re: Black Swan 101
Post by: Dr. Pataphoros, SpD on May 20, 2008, 07:20:03 PM
Quote from: Ratatosk on May 20, 2008, 07:04:28 PM
Quote from: Cain on May 20, 2008, 06:58:35 PM
Also requires a large impact to be a Black Swan.

Winning $5 million in imaginary money is not a Black Swan.

Losing $20 trillion in real money because a very very unlikely event just happened, on the other hand, is.

LOL

True enough in some sense. I think, Padre Pataphoros, that you may be confusing the value of the concept by misapplication. I don't think anyone has argued that Black Swans mean that all predictive data is useless, or that it has any effect on the nature of scientific prediction. Rather, its a philosophical point which focuses on the unknowable, which, throughout history seems to have been where the largest changes have wandered into reality from.

  When I asked for a clear definition of Black Swan THAT is the element that I was missing to make all of this make sense.

...


...  ...


Hm.  I no longer have a point to argue.  Carry on.
Title: Re: Black Swan 101
Post by: Bebek Sincap Ratatosk on May 20, 2008, 07:23:49 PM
Quote from: Padre Pataphoros on May 20, 2008, 07:20:03 PM
Quote from: Ratatosk on May 20, 2008, 07:04:28 PM
Quote from: Cain on May 20, 2008, 06:58:35 PM
Also requires a large impact to be a Black Swan.

Winning $5 million in imaginary money is not a Black Swan.

Losing $20 trillion in real money because a very very unlikely event just happened, on the other hand, is.

LOL

True enough in some sense. I think, Padre Pataphoros, that you may be confusing the value of the concept by misapplication. I don't think anyone has argued that Black Swans mean that all predictive data is useless, or that it has any effect on the nature of scientific prediction. Rather, its a philosophical point which focuses on the unknowable, which, throughout history seems to have been where the largest changes have wandered into reality from.

  When I asked for a clear definition of Black Swan THAT is the element that I was missing to make all of this make sense.

...


...  ...


Hm.  I no longer have a point to argue.  Carry on.

Then...

LET'S GET DANZING!

http://www.youtube.com/watch?v=dlf6n7IzZYk (http://www.youtube.com/watch?v=dlf6n7IzZYk)
Title: Re: Black Swan 101
Post by: Bu🤠ns on May 23, 2008, 08:04:14 AM
cain, or anyone else who has read the book:  Would you kindly elaborate more on the differences between Extremistan and Mediocristan including how scaleable knowledge vs. non-scaleable knowledge applies to both?  I get the idea i'm just having a hard time putting it in a simple "two dudes at a coffee break" explaination (if that's even possible).
Title: Re: Black Swan 101
Post by: Cain on May 23, 2008, 10:15:19 AM
Quote from: burnstoupee on May 23, 2008, 08:04:14 AM
cain, or anyone else who has read the book:  Would you kindly elaborate more on the differences between Extremistan and Mediocristan including how scaleable knowledge vs. non-scaleable knowledge applies to both?  I get the idea i'm just having a hard time putting it in a simple "two dudes at a coffee break" explaination (if that's even possible).

A scalable property is one that belongs in Extremistan - one that is subject to extremes.  A non-scalable property is one that belongs in Mediocristan.

For example, consider weight.  No matter how large a sample you have of people, when you study their weight on average, it will fall into a set range that the rest of the population resides in.  You don't get people who weigh 5000 tons, or 2 lb.  There are biological limits, and these can be tracked along the Bell curve.  Weight is a feature belonging to Mediocristan.

Now instead, let us consider income.  Income can be anything from nothing to the earnings of Bill Gates.  It is subject to high extremes and variations, because there are no physical limits on its growth.  It cannot be plotted by use of the Bell curve, and relying on the mean average is foolish, because the large variations can swing the "average" in one way or another (consider how GDP is used to calculate a countries wealth, especially in Saudi Arabia or China, where you have a small but fabulously wealthy upper class ruling over a destitute and numerous peasant class).  That is belonging to Extremistan.
Title: Re: Black Swan 101
Post by: LMNO on May 23, 2008, 03:49:44 PM
A simple test is this:  Will calculating the average mean of the group give a reasonable frame for all the people in that group?

Example:

1) weight.  210lb, 150lb, 190lb, 140lb, 280lb.  Average: 194lb.  All the given weights are close to this number.

2) income: $19k, $50k, $85k, $12k, $200M.  Average: $40,033,200  This number is completely unrelated to the individual incomes.
Title: Re: Black Swan 101
Post by: Bu🤠ns on May 23, 2008, 07:19:08 PM
so then do black swans happen in extremistan more often than they would in mediocristan due to the unpredictablitly of the limits? would it be a matter of basing knowledge on the end results rather than the processes?
Title: Re: Black Swan 101
Post by: Dr. Pataphoros, SpD on May 23, 2008, 07:53:50 PM
Quote from: burnstoupee on May 23, 2008, 07:19:08 PM
so then do black swans happen in extremistan more often than they would in mediocristan due to the unpredictablitly of the limits? would it be a matter of basing knowledge on the end results rather than the processes?

I think a Black Swan coming out of Mediocristan would be more surprising than from Extremistan.
Title: Re: Black Swan 101
Post by: Cain on May 23, 2008, 07:57:25 PM
According to the theory, it can't happen.  All "surprises" in Mediocristan fall within given and predictable range and thus are not very surprising at all.  Because the variables and possible results are known ahead of time, it is by definition impossible for a Black Swan to happen, because it needs to be unpredictable.
Title: Re: Black Swan 101
Post by: Triple Zero on May 23, 2008, 08:01:00 PM
what cain said, black swans only happen in extremistan. per definition, almost.

another point Taleb makes in his books is that there are a lot more phenomena that count as extremistan than we tend to think.

in fact, he argues that humans are biologically wired to discount extremistan phenomena and make predictions about them as if they were from mediocristan (not always, but a lot)
Title: Re: Black Swan 101
Post by: Dr. Pataphoros, SpD on May 23, 2008, 08:03:47 PM
Quote from: Cain on May 23, 2008, 07:57:25 PM
According to the theory, it can't happen.  All "surprises" in Mediocristan fall within given and predictable range and thus are not very surprising at all.  Because the variables and possible results are known ahead of time, it is by definition impossible for a Black Swan to happen, because it needs to be unpredictable.

Didn't we just get done arguing that everything is essentially unpredictable?  Using the weight analogy, what happens when someone falls drastically outside the norm?  Like the (in)famous John Brower Minnoch, who weighed nearly 1,500 lbs. at one point.
Title: Re: Black Swan 101
Post by: Triple Zero on May 23, 2008, 08:18:17 PM
that example demonstrates it as well, assume this is the most extreme example you can find, it is less than 10x the amount of your average human.

compare to the annual income statistic, which can span multiple orders of magnitude.

mathematically, it's also a different distribution. weight is Gaussian, while annual income has an exponential-like (or worse) distribution, causing these effects.
Title: Re: Black Swan 101
Post by: Dr. Pataphoros, SpD on May 23, 2008, 08:26:56 PM
I see.  So, given the size of the sample there's a "sweet spot" where the variation within that sample is prime for unexpected results.  The overweight guy is way outside the mean, but he's still on the graph because the variation of possible wieghts is significantly smaller than the actual number of people in the human race.  Conversely, there are nearly as many salaries as there are people, so the spread is greater.

That sound about right?
Title: Re: Black Swan 101
Post by: Cainad (dec.) on May 24, 2008, 10:53:02 AM
You're getting there. Also consider the impact of each individual occurrence. Is the general rule (as determined by probability) or the exception more significant?

I thought a visual aid might be useful for presenting this topic, again using the "weight versus income" comparison. These examples require an understanding of what standard deviation is.

This is something from Mediocristan:
(http://img149.imageshack.us/img149/8663/mediocristanpv6.jpg)

Notice that the guy who weighs 1,500 lbs, while very exceptional and quite rare (as indicated by his position on the standard deviation curve), ultimately is not very significant. On his own, he would not raise the average weight by more than a tiny bit, unless this graph were taken from a very small sample of people. Nor does he particularly skew the gene pool. The bulk of humanity still falls within a predictable range.

In this typical Mediocristan event, things that fall to the far end of the bell curve may be impressive or surprising, but they have little to no long-term influence. They are flukes, quirks, and generally ignorable.

Now let's look at something from Extremistan:
(http://img149.imageshack.us/img149/8245/extremistancy0.jpg)

Not only does someone like Bill Gates practically fall off the chart because of his wealth, those few people who have annual incomes in the billions cause a graph of worldwide average income to become distorted. The number of people who actually fall into the "average" range is much less than standard probability dictates.

So we see that in Extremistan, the people and things which are at the tail ends of the probability curve are actually far more significant than the people and things which fall into the more predictable range. The extreme concentration of wealth in the hands of a small elite, and the subsequent enormous population of poor people, has far more impact than the so-called average annual income of the world's population.

(Note: even though it is from Extremistan, this is not an example of a true Black Swan because everybody already knows about it)

(Graphs not to scale, you pedantic spags)
Title: Re: Black Swan 101
Post by: PeregrineBF on May 24, 2008, 11:54:23 AM
Black swans can occur even in Mediocristan. Meteors, volcanoes, Outside Context Problems (http://"http://en.wikipedia.org/wiki/Outside_Context_Problem), etc. No matter how well you define your context there will always be outliers, and in at least some of the cases these outliers will be very, very big.
Either that, or someone comes around with a bottle of spray paint and starts decorating the swans.
Title: Re: Black Swan 101
Post by: Verbal Mike on May 24, 2008, 12:14:13 PM
No PBF, I think the point is that when a Black Swan shows up in Mediocristan, that means it was actually Extremistan. If you believe a certain volcano to be perfectly safe and dormant, its eruption is one hell of an Extremistani Black Swan.
Title: Re: Black Swan 101
Post by: Triple Zero on May 24, 2008, 03:27:00 PM
exactly, what Verb said.

the point is, that if it was really mediocristan, and probabilities were distributed like a gaussian, the chance of a black swan type event occurring in a gaussian distribution would be to happen every once in a few thousand lifetimes of the universe.

clearly this is not the case, as black swans happen a lot more often then that.

the conclusion is that the actual probability distribution must be different from a gaussian. btw that's why i disagree with the second image in Verb's post, because these kind of probability distributions do not look anything like a "bell" curve, but a lot more like a one-sided exponential downward slope.
(with a few boundary effects near the low end of the slope btw, income distribution on the low end trails off a bit faster, because the difference between "no food" and "not enough food to survive" are both fatal, causing a slight distortion from the "perfect" exponential curve)

and indeed PBF, "out of context" problems are exactly the kind of things that "turn" perceived mediocristan situations into extremistan ones. but that's a fallacy, saying "yeah this event behaves gaussian, as long as we ignore the really big outliers" basically means the event is not gaussian at all, and you're going to do a real shitty job at predicting the events sooner or later. sometimes with catastrophal results.
one of the things Taleb explains in his books is why this distinction really matters.
Title: Re: Black Swan 101
Post by: Cainad (dec.) on May 24, 2008, 06:42:24 PM
Quote from: triple zero on May 24, 2008, 03:27:00 PM

the conclusion is that the actual probability distribution must be different from a gaussian. btw that's why i disagree with the second image in Verb's post, because these kind of probability distributions do not look anything like a "bell" curve, but a lot more like a one-sided exponential downward slope.
(with a few boundary effects near the low end of the slope btw, income distribution on the low end trails off a bit faster, because the difference between "no food" and "not enough food to survive" are both fatal, causing a slight distortion from the "perfect" exponential curve)


My image, my post, thank yuo very much :spag:

But seriously, I agree that the second image is not accurate. I suppose that's kind of the point; Extremistan causes bell curve distribution of probability to become a pile of worthless fail. If someone could design a more accurate image, I think it would be beneficial.


As a side note, I think the use of the terms Mediocristan and Extremistan is pretty much begging for menu-eating. It creates the impression that there are places where everything really is mediocre or extreme.
Title: Re: Black Swan 101
Post by: Triple Zero on May 24, 2008, 06:56:13 PM
(oh that was you, heh sorry)

and yeah, an important thing is to realize that mediocristan and extremistan are very much intermingled.

also, pics of exponential distributions, pick one:
http://images.search.yahoo.com/search/images?p=exponential+distribution
(they're not all alike, like the gaussians, because they come in a wide range of forms--although some of the results of that search are not quite exponential distributions)
Title: Re: Black Swan 101
Post by: P3nT4gR4m on June 05, 2008, 09:35:51 AM
So the general rule is - nobody sees a black swan coming.

I guess the cool kids are the ones who see it first and realise what it is?
Title: Re: Black Swan 101
Post by: Cain on June 05, 2008, 10:35:29 AM
Well, you can't really see it coming, for the aforementioned reasons, but you can structure what you are doing (whatever that is) if you know one is possible, so that it doesn't wipe you out, and maybe even benefits you.  Being aware it can happen is the best defence, really.
Title: Re: Black Swan 101
Post by: P3nT4gR4m on June 05, 2008, 11:22:06 AM
I was thinking more along the lines of looking at an emerging phenomena, realising it's a black swan, as early as possible and second guessing the consequences for maximum gain.

eg. I wake up tomorrow and there's a bunch of undead shambling round the streets - my first thought would be grab an island and secure a supply line, escape route and perimeter thereof. I guess most people would be freaking out and doing the "how can this be happening" - spiel
Title: Re: Black Swan 101
Post by: Verbal Mike on June 05, 2008, 11:44:24 AM
And most governments would be denying it until the president has actually been eaten alive.
Title: Re: Black Swan 101
Post by: Bebek Sincap Ratatosk on June 05, 2008, 03:56:06 PM
Quote from: SillyCybin on June 05, 2008, 11:22:06 AM
I was thinking more along the lines of looking at an emerging phenomena, realising it's a black swan, as early as possible and second guessing the consequences for maximum gain.

eg. I wake up tomorrow and there's a bunch of undead shambling round the streets - my first thought would be grab an island and secure a supply line, escape route and perimeter thereof. I guess most people would be freaking out and doing the "how can this be happening" - spiel

But, Silly, in that case, the Black Swan would be that the Zombie Virus was a virulent airborne strain...
Title: Re: Black Swan 101
Post by: P3nT4gR4m on June 05, 2008, 04:16:07 PM
Hmmmm .... Tricky fuckers these swans. Never trusted them. There's a rumour going around they can break a mans arm with their wings. This has, of course, never happened in recorded history. I think the swans may have started the rumour  :tinfoilhat:
Title: Re: Black Swan 101
Post by: Cain on June 05, 2008, 04:24:59 PM
They also frequently get eaten by the foreign enemy de jour as they THREATEN TO TAKE OVER OUR COUNTRY (also no citation on that).
Title: Re: Black Swan 101
Post by: LMNO on June 05, 2008, 04:29:22 PM
Perhaps this is a good example of a Black Swan:

Moodys is a renowned investment firm that uses complex formula to rate investments.  A few years ago, they gave top rating to a some products.  On their advice, many, many people bought in, for billions.

This product was tied to the sub-prime market, and crashed.  Of course.

Upon review, the AAA rating given was because of a computer glitch. http://www.forbes.com/afxnewslimited/feeds/afx/2008/05/21/afx5032513.html


The glitch would be considered a black swan because:

1.  It occured outside the "rules" of investment strategy.
2.  No one expected it until it happened.
3.  It had a major impact.
Title: Re: Black Swan 101
Post by: P3nT4gR4m on June 05, 2008, 11:25:47 PM
No offence but I think everyone will agree - zombies are a much more inspiring example :roll:
Title: Re: Black Swan 101
Post by: Bebek Sincap Ratatosk on June 06, 2008, 03:22:05 AM
Quote from: SillyCybin on June 05, 2008, 11:25:47 PM
No offence but I think everyone will agree - zombies are a much more inspiring example :roll:

Woah, I so totally agree,,,
  \
:hippie:
Title: Re: Black Swan 101
Post by: hooplala on June 06, 2008, 04:38:43 AM
ME TOO!! ..... OoooOOOooh - what's this??  Scarlett Johansen's new album!  THERE'S a Black Swan for you!
                                                               \
(http://www.horrordvds.com/reviews/a-m/dayotd-se/dayotd-se_shot4l.jpg)
Title: Re: Black Swan 101
Post by: P3nT4gR4m on June 06, 2008, 09:10:04 AM
Bub = Best zombie evar!
Title: Re: Black Swan 101
Post by: Cain on June 06, 2008, 12:02:57 PM
Quote from: LMNO on June 05, 2008, 04:29:22 PM
Perhaps this is a good example of a Black Swan:

Moodys is a renowned investment firm that uses complex formula to rate investments.  A few years ago, they gave top rating to a some products.  On their advice, many, many people bought in, for billions.

This product was tied to the sub-prime market, and crashed.  Of course.

Upon review, the AAA rating given was because of a computer glitch. http://www.forbes.com/afxnewslimited/feeds/afx/2008/05/21/afx5032513.html


The glitch would be considered a black swan because:

1.  It occured outside the "rules" of investment strategy.
2.  No one expected it until it happened.
3.  It had a major impact.

The entire sub-prime business was a black swan too.  Virtually no-one in the financial industry cared to think about what happened if too many people defaulted on payments at the same time.
Title: Re: Black Swan 101
Post by: Bebek Sincap Ratatosk on June 06, 2008, 03:33:00 PM
Quote from: Cain on June 06, 2008, 12:02:57 PM
Quote from: LMNO on June 05, 2008, 04:29:22 PM
Perhaps this is a good example of a Black Swan:

Moodys is a renowned investment firm that uses complex formula to rate investments.  A few years ago, they gave top rating to a some products.  On their advice, many, many people bought in, for billions.

This product was tied to the sub-prime market, and crashed.  Of course.

Upon review, the AAA rating given was because of a computer glitch. http://www.forbes.com/afxnewslimited/feeds/afx/2008/05/21/afx5032513.html


The glitch would be considered a black swan because:

1.  It occured outside the "rules" of investment strategy.
2.  No one expected it until it happened.
3.  It had a major impact.

The entire sub-prime business was a black swan too.  Virtually no-one in the financial industry cared to think about what happened if too many people defaulted on payments at the same time.

But was that because it was unpredictable, or predictable, if it were not for the sand encasing their head?  :lulz:
Title: Re: Black Swan 101
Post by: Cainad (dec.) on June 06, 2008, 10:12:14 PM
OSHIT

Now we must clarify whether a Black Swan is unpredictable, or merely unpredicted. Or does it matter?
Title: Re: Black Swan 101
Post by: Verbal Mike on June 06, 2008, 11:34:27 PM
Doesn't matter.
Title: Re: Black Swan 101
Post by: Bebek Sincap Ratatosk on June 06, 2008, 11:59:34 PM
Well, maybe they're variations... one could be avoided with proper controls and planning... the other, not so much?
Title: Re: Black Swan 101
Post by: Triple Zero on June 07, 2008, 12:02:06 AM
no rat, that's not how black swans work.
Title: Re: Black Swan 101
Post by: Verbal Mike on June 07, 2008, 12:06:33 AM
I don't think it matters. People will always have a certain idea of what they can predict, and this will almost invariably be true of less than they think. It's natural to assume Extreminstani events and eventualities are in Mediocristan, and it's surprising when this turns out to be wrong.
Title: Re: Black Swan 101
Post by: Bebek Sincap Ratatosk on June 07, 2008, 12:10:48 AM
Quote from: triple zero on June 07, 2008, 12:02:06 AM
no rat, that's not how black swans work.

Huh? Well, I was asking not saying ;-)

I thought they were the things one couldn't plan for... but rather they're the things you don't plan for? Or...?
Title: Re: Black Swan 101
Post by: Verbal Mike on June 07, 2008, 12:17:24 AM
They're the things you didn't think about which turn out to change the way you think about things.
Title: Re: Black Swan 101
Post by: Bu🤠ns on June 07, 2008, 07:30:55 AM
i've been dilligently listening to the audiobook on my rides home from work and from what i can gather is Taleb presents the idea of a black swan and it's 3 characteristics.  then he presents a story of how certain fallacies and fundamental assumptions (most of what cain posted in the OP) about the way things work don't account for randomness.  some of the biggies are the narrative fallacy and the silent evidence fallacy.  see, the book tends to read in a kind of story-like kind of way with anecdotes and such drawn mostly from Taleb's personal experience. So while i have found cain's notes incredibly helpful i can see how his short notes might seem confusing without also referencing the book.  It's a really great read (ahem, listen), as it is immediately applicable to daily life.
Title: Re: Black Swan 101
Post by: Cain on June 11, 2008, 02:28:17 PM
http://business.timesonline.co.uk/tol/business/economics/article4022091.ece?print=yes&randnum=1212475411171

May be of interest
Title: Re: Black Swan 101
Post by: Bebek Sincap Ratatosk on June 11, 2008, 02:49:49 PM
http://www.cnn.com/2008/WORLD/asiapcf/06/11/pakistan.troops.killed/index.html (http://www.cnn.com/2008/WORLD/asiapcf/06/11/pakistan.troops.killed/index.html)

QuoteA U.S. official with knowledge of the reports acknowledged Wednesday that U.S. planes launched an airstrike in the area on Tuesday but did so in "hot pursuit" of suspected militants who had conducted an ambush on the Afghan side of the border.

QuoteA Pakistani military statement condemned the "completely unprovoked and cowardly act" that killed 11 members of a Pakistani paramilitary force on Pakistani soil. It said the attack "had hit at the very basis of cooperation and sacrifice with which Pakistani soldiers are supporting the coalition in war against terror."

Ohh, Ohh, Teacher!! Is dropping 500lb bombs on some of your close friends a sort of Black Swan?
Title: Re: Black Swan 101
Post by: Bebek Sincap Ratatosk on June 11, 2008, 02:55:24 PM
QuoteTaleb had been vindicated. "It was my greatest vindication. But to me that wasn't a black swan; it was a white swan. I knew it would happen and I said so. It was a black swan to Ben Bernanke [the chairman of the Federal Reserve].

Of Course! Black Swans are specific to Black Iron Prisons, because they live "where our eyes don't go"TMBG. So the BiP of an economist doesn't see the swan, until it poops on his head... and guys like Taleb saw the swan ahead of time, and thus it was just a variable, rather than a surprise.

Brilliant.
Title: Re: Black Swan 101
Post by: Verbal Mike on June 11, 2008, 07:39:22 PM
That article was made of win and kittens. Black kittens.
Title: Re: Black Swan 101
Post by: Triple Zero on June 18, 2008, 02:10:00 AM
Quote from: Ratatosk on June 11, 2008, 02:55:24 PM
Of Course! Black Swans are specific to Black Iron Prisons, because they live "where our eyes don't go"TMBG. So the BiP of an economist doesn't see the swan, until it poops on his head... and guys like Taleb saw the swan ahead of time, and thus it was just a variable, rather than a surprise.

hey!

BINGO!

black swans are indeed caused by the BIP

Taleb's most important point is that they are caused by a bar shared by (nearly) everybody.

thanks for that insight, Rat!
Title: Re: Black Swan 101
Post by: P3nT4gR4m on June 18, 2008, 09:00:16 AM
Comes back to what I was saying about recognising it as early as possible or even before it appears. We set up some kind of early warning think-tank and the opportunities for massive profit/lulz are enormous.