Discrete Probability Distributions: A Note

In practice, we often construct new random variables by performing arithmetic operations on other random variables. Let $X$ be a random variable, and let $c$ and $d$ be constants. If Y is a new random variable, and $Y = cX + d$, then $$E(Y) = cE(X) + d$$ $$V(Y) = c^2V(X)$$

B. Discrete Uniform Distribution

• Notation: $X \sim DU(a, b)$ (X follows a Discrete Uniform distribution with parameters a and b)

• Probability Mass Function (PMF): $$f(x) = P(X = x) = \frac{1}{b - a + 1}, \quad x = a, a+1, …, b$$ This means each value between a and b (inclusive) has an equal probability of occurring.

• Mean (Expected Value): $$E(X) = \mu = \frac{a + b}{2}$$

• Variance: $$Var(X) = \sigma^2 = \frac{(b - a + 1)^2 - 1}{12}$$

• Example: Rolling a fair six-sided die. Here, $a = 1$ and $b = 6$.

  • $P(X = 3) = \frac{1}{6 - 1 + 1} = \frac{1}{6}$ (The probability of rolling a 3 is 1/6)
  • $E(X) = \frac{1 + 6}{2} = 3.5$
  • $Var(X) = \frac{(6 - 1 + 1)^2 - 1}{12} = \frac{36 - 1}{12} = \frac{35}{12} \approx 2.92$

• Notation: $X \sim Bin(n, p)$ (X follows a Binomial distribution with parameters n and p)

• Probability Mass Function (PMF): $$f(x) = P(X = x) = \binom{n}{x} p^x (1 - p)^{n - x}, \quad x = 0, 1, …, n$$ where $\binom{n}{x} = \frac{n!}{x!(n-x)!}$ is the binomial coefficient, representing the number of ways to choose x successes from n trials.

• Mean (Expected Value): $$E(X) = \mu = np$$

• Variance: $$Var(X) = \sigma^2 = np(1 - p)$$

• Example: Flipping a fair coin 5 times (n=5, p=0.5). What’s the probability of getting exactly 2 heads (x=2)?

• $P(X = 2) = \binom{5}{2} (0.5)^2 (1 - 0.5)^{5 - 2} = \frac{5!}{2!3!} (0.25)(0.125) = 10 \times 0.25 \times 0.125 = 0.3125$

• The binomial coefficient $\binom{5}{2} = \frac{5!}{2!3!} = \frac{5\times4\times3\times2\times1}{(2\times1)(3\times2\times1)} = \frac{120}{12} = 10$

• $E(X) = 5 \times 0.5 = 2.5$

• $Var(X) = 5 \times 0.5 \times (1 - 0.5) = 1.25$

• Notation: $X \sim NB(r, p)$
• Probability Mass Function (pmf): $$f(x) = P(X=x) = \binom{x-1}{r-1}p^{r}(1-p)^{x-r} \quad x=r, r+1, r+2,…$$
• Mean: $$\mu = E(X) = \frac{r}{p}$$
• Variance: $$\sigma^{2} = V(X) = \frac{r(1-p)}{p^{2}}$$

• Notation: $X \sim Geom(p)$ (X follows a Geometric distribution with parameter p)

• Probability Mass Function (PMF): $$f(x) = P(X = x) = p(1 - p)^{x - 1}, \quad x = 1, 2, …$$

• Mean (Expected Value): $$E(X) = \mu = \frac{1}{p}$$

• Variance: $$Var(X) = \sigma^2 = \frac{1 - p}{p^2}$$

• Example: Rolling a die until you get a 6 (p = 1/6). What’s the probability it takes exactly 3 rolls?

• $P(X = 3) = (\frac{1}{6})(1 - \frac{1}{6})^{3 - 1} = (\frac{1}{6})(\frac{5}{6})^2 = \frac{25}{216} \approx 0.116$

• $E(X) = \frac{1}{1/6} = 6$

• $Var(X) = \frac{1 - 1/6}{(1/6)^2} = \frac{5/6}{1/36} = 30$

• Notation: $X \sim Poisson(\lambda)$ (X follows a Poisson distribution with parameter λ) Note: Often, the parameter is given as $\lambda T$, where $\lambda$ is the rate and $T$ is the length of the interval. We use $\lambda T$ in formulas for consistency.

• Probability Mass Function (PMF): $$f(x) = P(X = x) = \frac{e^{-\lambda T} (\lambda T)^x}{x!}, \quad x = 0, 1, 2, …$$

• Mean (Expected Value): $$E(X) = \mu = \lambda T$$

• Variance: $$Var(X) = \sigma^2 = \lambda T$$

• Example: Customers arrive at a store at an average rate of 5 per hour ($\lambda = 5$). What’s the probability that exactly 2 customers arrive in a given hour (T = 1)?

• $P(X = 2) = \frac{e^{-5 \times 1} (5 \times 1)^2}{2!} = \frac{e^{-5} \times 25}{2} \approx 0.084$

• $E(X) = 5 \times 1 = 5$

• $Var(X) = 5 \times 1 = 5$