News:

All you can say in this site's defence is that it, rather than reality, occupies the warped minds of some of the planet's most twisted people; gods know what they would get up to if it wasn't here.  In these arguably insane times, any lessening or attenuation of madness is maybe something to be thankful for.

LessWrongWiki: What the hell are they talking about?

Started by LMNO, August 27, 2013, 04:23:24 PM

LMNO

This thread is for discussing some of the entries in the LessWrong Wiki (http://lesswrong.com/). Some of the entries are fairly wordy and confusing, so why not talk them through?

Nephew Twiddleton

Ah, thanks, LMNO.

So I've started reading it, and I read the Intuitive Explanation of Bayes Theorem.

Maybe not that intuitive since I still don't really understand what Bayesian reasoning entails.
Strange and Terrible Organ Laminator of Yesterday's Heavy Scene
Sentence or sentence fragment pending

Soy El Vaquero Peludo de Oro

TIM AM I, PRIMARY OF THE EXTRA-ATMOSPHERIC SIMIANS

LMNO

Here's the really, really easy version.

Moe:  I believe, based upon the knowledge I currently have, that there's a 30% chance it will rain today.

Larry: Have you looked out the window?

Moe: No.

Larry: There are thunderclouds in the sky.

Moe: Updating my knowledge base, I now believe there is a 90% chance of rain.

There you go.  Based upon incoming knowledge, adjust your predictions.  The rest is math.

Nephew Twiddleton

Quote from: LMNO, PhD (life continues) on August 27, 2013, 04:48:18 PM
Here's the really, really easy version.

Moe:  I believe, based upon the knowledge I currently have, that there's a 30% chance it will rain today.

Larry: Have you looked out the window?

Moe: No.

Larry: There are thunderclouds in the sky.

Moe: Updating my knowledge base, I now believe there is a 90% chance of rain.

There you go.  Based upon incoming knowledge, adjust your predictions.  The rest is math.

Ok. I was getting lost in the breast cancer example, and trying to follow along with the math there, and not seeing exactly what they were trying to say. I'm making an attempt to read one article a day, or at least every other day, so I'll probably have quite a bit to ask about.
Strange and Terrible Organ Laminator of Yesterday's Heavy Scene
Sentence or sentence fragment pending

Soy El Vaquero Peludo de Oro

TIM AM I, PRIMARY OF THE EXTRA-ATMOSPHERIC SIMIANS

LMNO

Oho!  The breast cancer bit is kind of awesome, but it really deals with a deeper issue, which is that most people have no idea how to deal with probabilities.

If something has a 0.01% chance of killing you, and I tell you that plucking your pubic hair out one by one will cut that probability in half, would you reach for the tweezers?

Nephew Twiddleton

Quote from: LMNO, PhD (life continues) on August 27, 2013, 04:57:09 PM
Oho!  The breast cancer bit is kind of awesome, but it really deals with a deeper issue, which is that most people have no idea how to deal with probabilities.

If something has a 0.01% chance of killing you, and I tell you that plucking your pubic hair out one by one will cut that probability in half, would you reach for the tweezers?

No. The chances of me dying are already pretty slim, and reducing that by half by that method doesn't sound very pleasant for such a marginal benefit.
Strange and Terrible Organ Laminator of Yesterday's Heavy Scene
Sentence or sentence fragment pending

Soy El Vaquero Peludo de Oro

TIM AM I, PRIMARY OF THE EXTRA-ATMOSPHERIC SIMIANS

LMNO

And all the breast cancer example is showing is, it's often really, really hard to make that calculation when dealing with multiple probabilities.

Nephew Twiddleton

Quote from: LMNO, PhD (life continues) on August 27, 2013, 05:12:03 PM
And all the breast cancer example is showing is, it's often really, really hard to make that calculation when dealing with multiple probabilities.

Ah, gotcha. I think the multiple probabilities is where I got lost.
Strange and Terrible Organ Laminator of Yesterday's Heavy Scene
Sentence or sentence fragment pending

Soy El Vaquero Peludo de Oro

TIM AM I, PRIMARY OF THE EXTRA-ATMOSPHERIC SIMIANS

Nephew Twiddleton

So why do people get really excited over it?
Strange and Terrible Organ Laminator of Yesterday's Heavy Scene
Sentence or sentence fragment pending

Soy El Vaquero Peludo de Oro

TIM AM I, PRIMARY OF THE EXTRA-ATMOSPHERIC SIMIANS

McGrupp

Quote from: LMNO, PhD (life continues) on August 27, 2013, 04:48:18 PM
Here's the really, really easy version.

Moe:  I believe, based upon the knowledge I currently have, that there's a 30% chance it will rain today.

Larry: Have you looked out the window?

Moe: No.

Larry: There are thunderclouds in the sky.

Moe: Updating my knowledge base, I now believe there is a 90% chance of rain.

There you go.  Based upon incoming knowledge, adjust your predictions.  The rest is math.

That is a really good explanation.

I love the lesswrong wiki. Conjunction fallacy made my head hurt until I realized I was reading their explanation backwards. Then it made a lot of sense.

LMNO

For the rest of the board:  The problem given is this:

1% of women at age forty who participate in routine screening have breast cancer.  80% of women with breast cancer will get positive mammographies.  9.6% of women without breast cancer will also get positive mammographies.  A woman in this age group had a positive mammography in a routine screening.  What is the probability that she actually has breast cancer?

only 100 out of 10,000 women have cancer.
-> 80 of them will have a positive test
-> 20 will have a negative test

9,900 do not have cancer
-> 8,950 will have a negative test
-> 950 will have a positive test

So, how many people have a positive result?  80+950 = 1030. And, 80 out of 1030 positive results is 80/1030 = 7.8%

So if you give a test to 10,000 women, and you get 1030 positive tests, only 80 of those women have cancer.

A woman with a positive test has a 7.8% chance of having cancer.

The point being, a lot of people look at the "80% chance of detecting breast cancer" and stop there.  You need to be aware of all the other probabilities involved in order to get the correct answer.

Please note these numbers are made up; Elizer used breast cancer as an example so people would pay attention.

LMNO

So, the main, MAIN point is:  Your "prior probability" was that 1% of women who get screened have breast cancer.  That's the "reality".  The box is opened, they either do or don't.

The probabilities of the test are "conditional probabilities".  You can have a test that is correct 90% of the time, or a test that's correct 10% of the time.  That doesn't change the prior probability that 1% of the women have cancer; it just changes what you can guess about a person if they have a positive test.  That final guess is known as the "revised" or "posterior" probability.

The problem being is that most people think: "I got a positive test, so there's an 80% chance I have cancer!"

That's why people get so excited when they're shown Bayes.  It allows for correct thinking.

Nephew Twiddleton

Quote from: LMNO, PhD (life continues) on August 27, 2013, 06:05:55 PM
So, the main, MAIN point is:  Your "prior probability" was that 1% of women who get screened have breast cancer.  That's the "reality".  The box is opened, they either do or don't.

The probabilities of the test are "conditional probabilities".  You can have a test that is correct 90% of the time, or a test that's correct 10% of the time.  That doesn't change the prior probability that 1% of the women have cancer; it just changes what you can guess about a person if they have a positive test.  That final guess is known as the "revised" or "posterior" probability.

The problem being is that most people think: "I got a positive test, so there's an 80% chance I have cancer!"

That's why people get so excited when they're shown Bayes.  It allows for correct thinking.

Gotcha.

Also, I was thinking the opposite. The woman still has a 1% chance of breast cancer, because I figured I was being tricked somehow.
Strange and Terrible Organ Laminator of Yesterday's Heavy Scene
Sentence or sentence fragment pending

Soy El Vaquero Peludo de Oro

TIM AM I, PRIMARY OF THE EXTRA-ATMOSPHERIC SIMIANS

LMNO

She had a 1% chance.  Now that she's got a positive test result, she now has a 7.8% chance.

Nephew Twiddleton

Quote from: LMNO, PhD (life continues) on August 27, 2013, 06:32:52 PM
She had a 1% chance.  Now that she's got a positive test result, she now has a 7.8% chance.

I got that part after it was explained.

That does seem counterintuitive though.
Strange and Terrible Organ Laminator of Yesterday's Heavy Scene
Sentence or sentence fragment pending

Soy El Vaquero Peludo de Oro

TIM AM I, PRIMARY OF THE EXTRA-ATMOSPHERIC SIMIANS