Lack of new information
if I observe an event that had a probability of 1 of occurring, I have no new knowledge
That is: If P(B)=1, then P(A|B)=P(A)
if I observe an event that had the same probability of occurring for all hypotheses under consideration, then I have no new knowledge about those hypotheses
Suppose A can take values a1,…,aK. If P(B|A=a1)=…=P(B|A=aK), then P(A|B)=P(A).
The Sleeping Beauty Problem
Consider the Sleeping Beauty problem (quoting wikipedia):
Sleeping Beauty volunteers to undergo the following experiment. On Sunday she is given a drug that sends her to sleep. A fair coin is then tossed just once in the course of the experiment to determine which experimental procedure is undertaken. If the coin comes up heads, Beauty is awakened and interviewed on Monday, and then the experiment ends. If the coin comes up tails, she is awakened and interviewed on Monday, given a second dose of the sleeping drug, and awakened and interviewed again on Tuesday. The experiment then ends on Tuesday, without flipping the coin again. The sleeping drug induces a mild amnesia, so that she cannot remember any previous awakenings during the course of the experiment (if any). During the experiment, she has no access to anything that would give a clue as to the day of the week. However, she knows all the details of the experiment.
Each interview consists of one question, “What is your credence now for the proposition that our coin landed heads?”
When she is asked the interview question, she knows the details of the experiment, and that she has been woken up at least one time.
Let A represent the result of the coin flip (1=heads, 0=tails).
Let B=1 if sleeping beauty has been woken up at least one time, and B=0 otherwise.
P(B=1|A=1)=P(B=1|A=0)=1. (regardless of whether heads or tails was selected, there was a probability of 1 that Beauty would be woken up, and would have no memory of whether she had been woken up in the past)
So, the fact that she was woken up does not make it more likely that the flip came up tails. The fact that she was woken up and asked a question provided her with no new information about heads or tails.
Beauty doesn’t like being wrong. If it landed heads she wants her guess to be p=1 (where p is her guess for the probability of heads). The farther away from that optimal value she is the bigger the loss.
Let’s make that idea more concrete by adding the following twist to the problem: suppose every time she is interviewed, she has to pay $|p-A|.
Beauty, being a rationalist, will want to minimize her expected loss. If heads comes up, she will lose $|p-1| on Monday. If tails comes up, she will lose $|p-0| on Monday and $|p-0| on Tuesday. Thus, she will choose p to minimize (1/2)*|p-1|+(1/2)*(|p-0|+|p-0|).
The value of p that minimizes her expected loss is p=0. That’s my surprise solution to the Sleeping Beauty problem – she should say she’s sure it’s tails.
If instead you use squared error loss, i.e., you minimize (1/2)(p-1)2+(1/2)*2*(p-0)2, then you get the popular p=1/3 solution. But why squared error loss?
If Beauty sticks to probability laws, and updates based on evidence, she should guess p=0.5.
But, if Beauty counts being wrong on Monday and Tuesday as twice as disturbing as being wrong just on Monday, then she should guess p=0. (this isn’t really her credence for heads, just the value that she views as optimal (minimizing loss))