An experiment which can only have two outcomes is a Bernouilli trial.
Example 1.
X is a random variable which takes the value 1 with probability \( p \) and 0 with probability \( q = 1-p \).
Calculate mean and variance.
\begin{align}
\mathbb{E}[X] = 1. p + 0. (1-p) = p \\
\mathbb{E}[X^2] = 1^2. p + 0. (1-p) = p \\
\end{align}
so \( \text{Var}(X) = p -p^2 = p(1-p) = pq \)
If we have a sequence of Bernouilli trials (with the same probability of success \( p \), the number of successes is a Binomial random variable.
Binomial random variable.
\( ( X_1, X_2,…,X_n ) \) are a sequence of independent identically distributed Bernouilli trials then
\( Y = X_1+ X_2+…+X_n \) is a binomial random variable.
We write \( Y = \text{bin}(n,p). Y \) is the number of successs in \( n \) trials. We know the number of ways to choose k successes from n trials is \( \binom{n}{k} \) and the probability of k successes in a particular order is \( p^k (1-p)^{n-k} \) so
$$ P(Y = k) = \binom{n}{p} p^k (1-p)^{n-k} $$
Example 3.
Calculate the mean and variance of Y.
\begin{align}
\mathbb{E}[Y] &= \mathbb{E}[X_1 + X_2 +…+ X_n]] \\
&= n\mathbb{E}[X] = np
\end{align}
\begin{align}
\mathbb{E}[Y^2] &= \mathbb{E}[(X_1+X_2+…+X_n)^2] \\
&=\mathbb{E}[\sum_{i=1}^{i=n} X_i^2 + 2\sum_{i \leq j} X_i X_j] \\ &=\sum_{i=1}^{i=n}\mathbb{E}[X_i^2] + 2 \sum_{i \leq j} \mathbb{E}[X_i X_j] \; \text{using linearity} \\
&= np + 2 \sum_{ i \leq j} p^2 \; \text{since } X_i \text{ and } X_j \; i\neq j \text{ are independent} \\
&= np + n (n-1) p^2 \\
\end{align}
so $$\text{Var}(Y) = np +n(n-1)p^2 -n^2p^2 = np(1-p)=npq $$
Notice this is \(n\text{Var}(X)\) which is what we expect!
Example 4.
Alternative Derivation.
\begin{align}
\mathbb{E}[Y] &= \sum_{k=0}^{k=n} \binom{n}{k} p^k q^{n-k} k \\
&= \sum_{k=0}^{k=n} \binom{n}{k} (\frac{p}{q})^k q^{n} k
\end{align}
Recall binomial theorem
\begin{align}
(1+x)^n = \sum_{k=0}^{k=n} \binom{n}{k} x^k \\
\frac{d }{dx} (1+x)^n = n (1+x)^{n-1} = \sum_{k=0}^{k=n} \binom{n}{k}k x^{k-1}
\end{align}
Let \( x = \frac{p}{q} \) so
\begin{align}
\mathbb{E}[X] &= n q^n\frac{p}{q}(1+\frac{p}{q})^{n-1} \\
&= np (q+p)^{n-1} = np
\end{align}
Similarly
\begin{align}
\frac{d}{dx^2} (1+x)^n = n (n-1) (1+x)^{n-2} = \sum_{k=0}^{k=n} \binom{n}{k}k(k-1)x^{k-2}
\end{align}
so that
\begin{align}
n (n-1) (1+x)^{n-2} x^2 q^n =\sum_{k=0}^{k=n} \binom{n}{k} k^2 x^k q^n -\sum_{k=0}^{k=n} \binom{n}{k} k x^k q^n
\end{align}
and again with \( x = \frac{p}{q} \)
\begin{align}
&\sum_{k=0}^{k=n} \binom{n}{k} k^2 x^k q^n -\sum_{k=0}^{k=n} \binom{n}{k} k x^k q^n \\
=&\sum_{k=0}^{k=n} \binom{n}{k} k^2 (\frac{p}{q})^k q^n -\sum_{k=0}^{k=n} \binom{n}{k} k (\frac{p}{q})^k q^n \\
= &\mathbb{E}[X^2] – \mathbb{E}[X]
\end{align}
so
\begin{align}
n (n-1) (1+\frac{p}{q})^{n-2} (\frac{p}{q})^2 q^n = \mathbb{E}[X^2] – \mathbb{E}[X]
\end{align}
Simplifying
\begin{align}
n(n-1) p^2 = \mathbb{E}[X^2] – \mathbb{E}[X]
\end{align}
\begin{align}
\text{Var}(X) =& \mathbb{E}[X^2] – \mathbb{E}^2[X] \\
&= n(n-1) p^2 + np – n^2p^2 \\
&= np(1-p) = npq
\end{align}