Gambler's Fallacy

Suppose we are throwing coins and bet if it is going to be head ($H$) or tails ($T$). After throwing the coin three times, it resulted in heads all of the throws. Do you think that the fourth throw will be heads, or will it be tails? 

Most people would assume that it is not likely to throw heads three times in a row, and it is even more unlikely that it would be four times. So, intuitively, we assume that the probability to throw tails is higher than the probability that we throw heads for the fourth time. However, this assumption is false, and is known as the Gambler's Fallacy.

To show you why it is false, we will take a mathematical approach using probability theory. Let $A_i$ be the $i$-th outcome of the coin game, which will be either $H$ or $T$, where the probability of getting either result is $\text{Pr}(A_i) = \frac{1}{2}$.

Now we want to know what the probability is of getting three times head in a row, which is

$$\text{Pr}(HHH) = \text{Pr}(H)\cdot\text{Pr}(H)\cdot\text{Pr}(H)=\frac{1}{2}\cdot\frac{1}{2}\cdot\frac{1}{2} = \frac{1}{8}.$$

Note that we can calculate each throw separately, and multiply the probabilities, because the throwing of a coin is an independent event. I don't have to convince you that the previous throw will not affect the next throw. Thus, we can write the equation above in a more general form

$$
\text{Pr}\left( \bigcap_{i=1}^n A_i \right) = \prod\limits_{i=1}^n \text{Pr}(A_i) = \frac{1}{2^n}.
$$

Now there are two key observations to make, the first one is that no matter how many times we had head or tails, the probability of the next throw to be head or tails is still $\frac{1}{2}$, so

$$
\text{Pr}(A_5\ |\ A_1 \cap A_2 \cap A_3 \cap A_4) = \text{Pr}(A_5) = \frac{1}{2}.
$$

The second observation is that you are equally likely to get $HTHTT$ as you are to get $HHHHH$. This basically tells us that you are equally likely to have any sequence of outcomes! Our brain likes patterns and thinks it is special that we have five heads in a row, while statistically the odds are the same as getting any other sequence after five throws.

Still not convinced? Let's take a look at all the possible outcomes after three throws:

Using the illustration, it should be easy to see that after $n$ throws, the probability that you get that sequence, is as likely as getting any other sequence, and is equal to $\frac{1}{2^n}$. Therefore we can conclude that there is absolutely nothing special about getting only heads or only tails. Every path you get after $n$ throws, has the same probability as any other path.

Note that the order of the sequence is important here. Otherwise HTT, THT, and TTH would be the same outcome! We are specifically asking, what is the probability to first throw heads, then tails, and then heads again.

If you have lost 100 throws, the probability of getting it right the next time, is the same as getting it wrong.