One statistician groans every 3 minutes from misleading uses of statistics

1 person dies every X minutes from cause Y

We hear that all the time.  Pick a cause of death and google “one person dies from [your cause here] every minutes”.   Let’s try melanoma.  In the United States, one person dies from melanoma every hour.   Ooh, that sounds scary.

But really, what does it mean?

Suppose 1 person per ten-thousand people dies from a particular disease per year.  Well, if there are 1 million people in the population, then, on average, one person dies every 53 minutes.   But, if there were 1 trillion people in the population, three people would die from this disease, on average, every second. In both cases, the mortality rate is the same, but the # dead per unit time statistic is much scarier sounding in the latter population.

Why use a statistic that isn’t population-size invariant?

Read Full Post »

Status quo bias

People are uncomfortable with change.  They tend to have emotional attachment to the status quo. Hence, there is a bias towards ideas that were introduced first.

Thankfully there is a pretty simple way to guard against this type of cognitive bias: a temporal reversal test (another rationality test).  If X is the status quo and you are uncomfortable with the proposed alternative Y, ask yourself how you would feel if Y was status quo and X was the new idea (i.e., reverse the order).

Here are some examples:

New technology

It is not uncommon for people to fear new technology.  For example, I’ve heard people express concerns about e-books and things like Kindle replacing paper books.  They have an attachment to paper books.  However, what if e-books had come first.  With e-books, you can have access to 100s of books via a lightweight electronic device; you can take notes on the pages; you can highlight passages; you can undo your highlights and notes; the condition of the book doesn’t decline.  Now, suppose e-books were all you ever knew.  And then someone came up with a new idea:  paper books!  Every book takes up physical space, weighs as much as your e-book reader, can get physically damaged, cannot undo any marks you make on it, etc.  Would you really think the paper book is a superior idea?

School year

If students had always went to school year-round (minus some vacation time), but now someone proposed giving students the summer off, would that become the new policy?

Death

I’ve heard people say that they are not really interested in immortality, because all living things must die.  But what if immortality came first.  Would anyone really argue in favor of a finite lifespan?  Remember, we will never run out of fun.

If the multiverse were intelligently designed, I could see, perhaps, trusting nature.  But that’s not the case.  Nature gets a lot of things wrong.  So the “it’s natural” argument isn’t sufficient.

Cyronics

Eliezer Yudkowsky used cryonics as an example (link):

If you found yourself in a world where everyone was signed up for cryonics as a matter of routine – including everyone who works at your office – you wouldn’t be the first lonely dissenter to earn the incredulous stares of your coworkers by unchecking the box that kept you signed up for cryonics, in exchange for an extra $300 per year.

…

The point being that if cryonics were taken for granted, it would go on being taken for granted; it is only the state of non-cryonics that is unstable, subject to being disrupted by rational argument.

Suffering

This is similar to the death example, but I’ve heard people say that pain and suffering can be good (makes you stronger, appreciate when you’re not suffering more, etc).  But if you reverse it, and the status quo didn’t include that suffering, would anyone prefer the suffering.  I don’t hear people wanting to get rid of anesthesia or machinery that reduces the need for physical labor (although I suspect people did argue against these when they were first introduced).

How aware are we of this bias?

I notice that religious folks tend to introduce their offspring to god at a young age.  Are they aware, at least subconsciously, that we are biased towards the ideas that are introduced first?  What if people learned about science, cognitive biases and rationality first, and then were introduced to god as an adult?  Would rates of religiosity decrease? (likely, given the heritability of religion)

We also make sure our kids know at a very young age that the United States is the greatest country in the history of greatest countries.

Parents tend to push their own political party on their kids at a young age.

It seems as if we exploit status quo bias when it comes to things that are important to us that are not easily supported with evidence.

Read Full Post »

Teacher and school markets

I am not in favor of giving teachers pay raises strictly based on years of experience and training/education, which I believe is the current standard (for public schools).  Ideally, salary would be proportional to quality for teachers and principals.  However, evaluating teachers and schools can be challenging.  For example, there are several major problems/challenges with using student test scores as the marker of success.

Problems with test score based evaluations:

1. teachers teach to the test; the focus would likely be on skills and memorization; this could lead to a more homogeneous and less creative group of students than is ideal for society

2. selection bias:  substantial variation in student quality across schools (by student quality here I mean independent from the effect of the school)

3.  incentive to cheat:  both teachers and administrators have incentive to cheat.  if the metric is a change score, then there is incentive to do poorly on the pre-test.

4.  at the state level there is incentive to make the test easier to show ‘improvement’ (e.g., link)

5.  kids are self-centered;  they have little personal incentive to try hard on these tests (when I was in school, I recall other students admitting that they were just going to randomly fill in dots, since they weren’t going to be graded on it)

Alternative proposal:

I’m going to ignore the logistics of this (how to pay for it, how to implement it, etc) for now.  Think of it as something more like a thought experiment.

Suppose we have a lot of fairly small schools, so that parents could choose between about 3 local schools (without having to travel long distances).  Parents would have a choice of which school to send their kids to.  A lottery would be used for schools that got more applicants than they could admit.

Similarly, within a school parents could choose between approximately 3 teachers.  A lottery would be used if a teach got too many applicants.  (note that this is a very different model than we see in public schools now, where parents are discouraged from asking for a specific teacher)

Ideally, these schools would all serve a single community, and therefore eliminates problem #2 above.

Schools would be  judged primarily based on how many people wanted to go to that school.  That is, schools would be judged based on demand for that school.  I think it’s likely that demand would be correlated with quality (a school with a good local reputation would most desirable).  It would be difficult to game this system (eliminating concerns #2 and 3).  If a school was getting very few applicants, that would be a reason to consider hiring a new principal.

Teachers would be judged primarily based on demand as well.  A teacher with a good reputation would get the most applicants.  This would likely be correlated with quality.

In addition, the schools would likely reflect local preferences.  If a community preferred skills and memorization to creativity, they’d send their kids to teachers and schools with those values.  I could easily see different types of schools emerging in a single community, reflecting diverse preferences of parents.

One possible concern is that teachers and schools that were more generous with their grades (e.g., pass everybody) would be more popular.  I’m not sure if that’s a problem, however.  It would reflect the preferences of the community.   In addition, the diploma would eventually be devalued (if these students were not having success post-high school).

Another possible problem is the small sample size.  Suppose all of the local schools were excellent.  Even if one had much lower demand than the others, that might not be reason to make major changes.  So, having both a relative metric (like demand) and an absolute metric might be necessary.  The same issue exists for teachers (i.e., a high demand teacher still might be bad, if all teachers in that grade are bad; a low demand teacher still might be good, if all teachers in that grade are good).

Read Full Post »

Chapter titles

Every chapter in Sister Carrie has a title.  All of them are interesting. They give a hint at what’s to come, but in most cases the meaning only becomes clear after you read the chapter.  I actually looked forward to reading the name of the next chapter.

Here are the first 10 chapter titles:

1. THE MAGNET ATTRACTING: A WAIF AMID FORCES
2. WHAT POVERTY THREATENED:  OF GRANITE AND BRASS
3. WE QUESTION OF FORTUNE: FOUR-FIFTY A WEEK
4. THE SPENDING OF FANCY: FACTS ANSWER WITH SNEERS
5. A GLITTERING NIGHT FLOWER: THE USE OF A NAME
6. THE MACHINE AND THE MAIDEN: A KNIGHT OF TO-DAY
7. THE LURE OF THE MATERIAL: BEAUTY SPEAKS FOR ITSELF
8. IMITATIONS BY WINTER: AN AMBASSADOR SUMMONED
9. CONVENTION’S OWN TINDER-BOX: THE EYE THAT IS GREEN
10. THE COUNSEL OF WINTER: FORTUNE’S AMBASSADOR CALLS

Read Full Post »

Compatibilism

I don’t see why free will and determinism wouldn’t be compatible.  At the moment I make a choice, I’m picking the option that I prefer.  Does it matter if what led me to that preference was entirely determined by prior occurrences?

Let’s think about what it would mean to not have free will.  Suppose I preferred option A, but just before I make the decision some external force affects my neurons and causes me to prefer option B. Well, at the moment I chose B, that was my preference.  That scenario is not inconsistent with free will or determinism (the external force is just part of the prior chain of events).

I think what people mean by free will is that they could have made a different decision.  Sure, they could have, if things had been different.  That is what they mean.  And that is free will…and determinism.

I pretty much agree with Katja Grace:

…you feel like your actions are neither determined nor random. You choose them.

And that is precisely why they are determined. They are determined by you. And you already exist to the finest detail at the time you are making the decision. If you made choices (or some element of them) not controlled by your personality, experience, thoughts and anything else that comes under the heading of ‘the state of your brain as a result of genetics and your prior environments’, they would be random, which still isn’t free will…

The narrator of Dostoyevsky‘s Notes from Underground was disturbed by determinism:

If, for instance, some day they calculate and prove to me that I made a long nose at someone because I could not help making a long nose at him and that I had to do it in that particular way, what FREEDOM is left me..? Then I should be able to calculate my whole life for thirty years beforehand. In short, if this could be arranged there would be nothing left for us to do;

Just because someone with perfect knowledge could accurately predict what you would do, that doesn’t mean you don’t have freedom. If what you did wasn’t predictable (i.e. included some random elements), how would that give you any more freedom (you have no control over the randomness)?

—

I liked this paragraph on free will from the book Sister Carrie by Theodore Dreiser:

Our civilization is still in a middle stage, scarcely beast, in that it is no longer wholly guided by instinct; scarcely human, in that it is not yet wholly guided by reason.  On the tiger no responsibility rests.  We see him aligned by nature with the forces of life — he is born into their keeping and without thought he is protected.  We see man far removed from the lairs of the jungles, his innate instincts dulled by too near an approach to free-will, his free-will not sufficiently developed to replace his instincts and afford him perfect guidance.  He is becoming too wise to hearken always to instincts and desires; he is still too weak to always prevail against them.  As a beast, the forces of life aligned him with them; as a man, he has not yet wholly learned to align himself with the forces.  In this intermediate stage he wavers — neither drawn in harmony with nature by his instincts nor yet wisely putting himself into harmony by his own free-will.  He is even as a wisp in the wind, moved by every breath of passion acting now by his will and now by his instincts, erring with one, only to retrieve by the other, falling by one, only to rise by the other — a creature of incalculable variability.

Read Full Post »

Placebos

Placebos don’t work if you know they’re placebos.  Thankfully, there is a placebo for everyone!

Prescription drugs

If you trust Western medicine, you can take prescription drugs.  For example, if you have mild or moderate depression symptoms, you can take anti-depressants.  These drugs can’t seem to beat sugar pills in clinical trials (i.e., both the drugs and sugar pills ‘work’ equally well; link and link).

Over-the-counter medicine

If your child has a cold, you can give them over-the-counter cold medication.   The American Academy of Pediatrics recommends

The American Academy of Pediatrics strongly recommends that over-the-counter cough and cold medications not be given to infants and children younger than 2 years because of the risk of life-threatening side effects. Also, several studies show that cold and cough products don’t work in children younger than 6 years and can have potentially serious side effects.

In a recent survey over 60% of parents said that these cold medicines are somewhat or very effective at relieving cold symptoms (link).

Homeopathy

If you don’t trust drug companies, you might seek out alternative treatments.  Homeopathy involves putting stuff in water, then diluting it repeatedly until all that is left is water.  Not surprisingly, homeopathic remedies (also known as expensive water) don’t outperform placebos (link).

Acupuncture

If you’re into Chinese medicine, you might try acupuncture.  It’s harder to have a blinded study with acupuncture placebos, but some studies have used sham acupuncture (needles not inserted all the way and in non-traditional spots) as the control.  For example, a study of lower back pain found acupuncture and sham acupuncture worked equally well, and both beat conventional therapy (link).  It’s a little unclear if acupuncture is just a placebo (link), so let’s put it in the ‘maybe’ category.

God?

I’ve heard plenty of people say that God saved them.  If God doesn’t exist, then this is another example of a placebo.  I’ll leave this one for you to decide.

Read Full Post »

Toyota fatal crashes graphs

First, let’s look at the when the Toyota unintended sudden acceleration fatal accidents* happened:

The distribution does seem to be fairly uniform from 2004-2008, with an increase in 2009.  Interestingly, 5 of the crashes in 2009 occurred in the first half of the year and the other 12 occurred in the 2nd half.

Next is a bar chart of the number of fatal accident by the year make of the car.  Given that we saw more crashes in 2009, we’d expect the distribution of year make of car to be skewed towards 2009.  But instead, it looks like this:

Why would we see more fatal crashes in 2009 if 2005 cars were one of the big culprits?

For more information, see my previous post

*using data from the LA Times

Read Full Post »

Sudden unintended acceleration and gullible media

The Toyota sudden unintended acceleration (SUA) problem has been setting off my skeptic alarm for some time.  Too much of it just doesn’t make sense.  For example, it rarely happens and no one seems to know why.

In general the media haven’t been very skeptical of these reports.  Instead, they’re all just bashing Toyota and even the National Highway Traffic Safety Administration.  The worst of this came the other day, when there was that story from California about the runaway Prius that had to be stopped by a California Highway patrol officer (link).  When I saw that story on the news (and it was all over the news) I immediately thought “that’s an obvious hoax by someone who heard about the Toyota SUA problems and wanted attention or to get in on a lawsuit.”  Yet, the story was presented uncritically by the media.  Of course, it turns out that it was a hoax (link).

Anyway, I’m glad to see that people are starting to question whether the SUA problems are real.  For example, here and here.  Megan McArdle notes that the majority of the drivers in these incidents were over 55, were probably starting up their vehicle from a complete stop when the crash happened and about a third of them were immigrants.  She concludes:

At any rate, when you look at these incidents all together, it’s pretty clear why Toyota didn’t investigate this “overwhelming evidence” of a problem:  they look a lot like typical cases of driver error.  I don’t know that all of them are.  But I do know that however advanced Toyota’s electronics are, they’re not yet clever enough to be able to pick on senior citizens.

Also mentioned is the fact that there have been SUA scares in the US in the past.  For example, the media fueled the Audi SUA scare in the 80s (link):

60 Minutes aired a report titled Out of Control on November 23, 1986, featuring interviews with six people who had sued Audi after reporting unintended acceleration, including footage of an Audi 5000 ostensibly displaying a surge of acceleration while the brake pedal was depressed. Subsequent investigation revealed that 60 Minutes had not disclosed they had engineered the vehicle’s behavior — fitting a canister of compressed air on the passenger-side floor, linked via a hose to a hole drilled into the transmission — the arrangement executed by one of the experts who had testified on behalf of a a plaintiff in a then pending lawsuit against Audi’s parent company.

Audi contended, prior to findings by outside investigators, that the problems were caused by driver error, specifically pedal misapplication. Subsequently, the National Highway Traffic Safety Administration (NHTSA) concluded that the majority of unintended acceleration cases, including all the ones that prompted the 60 Minutes report, were caused by driver error such as confusion of pedals

Another interesting aspect is when these fatal crashes occurred and the years of the cars involved.  I’ll cover that in my next post.

Read Full Post »

The mistake of seeking balance over truth

From Bukowski‘s essay Looking Back at the Big One in the book Portions From A Wine-Stained Notebook:

..in many creative minds, there is the natural urge to see the other side.  And a desire to sometimes stand with the other side just for the hell of it.  Because the first side has been there so long, so steady, and seems so worn….In an attempt to get beyond Good and Evil (if such do exist), the balance sometimes wavers and one goes to Evil (saying it might be there) because it seems more interesting — especially when your own countrymen just blithely accept to follow what they are told is Good (and never doubting it).  Generally, there is a tendency in intelligent men not to believe what most of the masses believe, and most of the time this puts them right near target; other times it gets their asses burned, especially in the political arena where the winners dictate which side is right….a Loser has never won a War Crimes Tribunal yet.

When I was young if I noticed everyone was on one side of an issue…  no, that’s not it… when I noticed that people didn’t even realize there was another side to an issue, I would tend go as far to the other side as possible to achieve some kind of balance, like the jumping guy in this figure:

It is a natural thing.  When you see everyone just accepting what they are told without giving it critical thought, it can be very frustrating.  You want to yell and scream “what’s wrong with you people?!?!”  And to try and get their attention, you say the most radical thing possible.  But..as I have gotten older (hopefully more mature), I realized that this is not the right approach.  I have learned to resist the urge to rebel just for the sake of shaking things up.  Now, I try to stand on the platform where truth is, regardless of whether it is close to where the masses are standing or not.  Not only does being provocative for the sake of shaking things up tend to not persuade those on the other side, it can even make them feel more confident in their beliefs as their single opponent does not have the force of truth behind him.

Read Full Post »