Discrete Probability Distribution

Probability Mass Function

Denoted by $P$ . Gives the probability that a discrete random variable $X$ is exactly equal to some value $x$ .

\sum_{x} P(x) = 1

Parameters

Mean

E(X) = \mu = \sum_{i} x_i \cdot P(X = x_i)

Here:

$x_i$: each possible value of $X$
$P(X = x_i)$: the probability of observing that value

Variance

\text{Var}(X) = \sigma^2 = \sum_{i} (x_i - \mu)^2 \cdot P(X = x_i)

An equivalent computational formula is:

\text{Var}(X) = \sum_{i} x_i^2 \cdot P(X = x_i) - \mu^2

Cumulative Distribution Function

F(x) = \sum_{t \leq x} P(X = t)

F(x) = F(\lfloor x \rfloor)\;\;\text{for } x \in \mathbb{Z}^{c}

Example

Consider a discrete random variable $X$ with the following probability distribution:

X	P(X)
1	0.2
2	0.3
3	0.4
4	0.1

Solution Reveal Hide

The mean would be: $E(X) = 1 \cdot 0.2 + 2 \cdot 0.3 + 3 \cdot 0.4 + 4 \cdot 0.1 = 2.4$ .

The variance would be: $\text{Var}(X) = (1-2.4)^2 \cdot 0.2 + (2-2.4)^2 \cdot 0.3 + (3-2.4)^2 \cdot 0.4 + (4-2.4)^2 \cdot 0.1 = 0.84$ .

Types

Uniform Distribution

A type of discrete probability distribution where all outcomes are equally likely. If a random variable $X$ can take $n$ distinct values, each value has a probability of $P(X = x) = \frac{1}{n}$ .

Bernoulli Distribution

A type of discrete probability distribution where there are only two possible outcomes, often referred to as “success” and “failure”. If a random variable $X$ can take values 0 or 1, with $P(X = 1) = p$ and $P(X = 0) = 1 - p$ , then $X$ follows a Bernoulli distribution.

PMF of Bernoulli distribution:

P(x) = p^x (1-p)^{1-x} \;\;;\; x = 0\;\text{or}\;1

Here:

$p$: the probability of success

Binomial Distribution

When an experiment consists of $n$ independent repeated Bernoulli trials, the experiment is said to be a binomial experiment. The number of successes ( $X$ ) follows a binomial distribution. And has a PMF:

P(X) = \binom{n}{x} p^x (1-p)^{n-x}\;\;;\; x = 0, 1, \ldots, n

Here:

$n$: the number of trials
$\theta$: the probability of success

Assumptions made in binomial distribution:

Each trial is independent.
Each trial has only two possible outcomes.
The probability of success is constant across trials.
The number of trials is fixed.

For a large sample of a binomial distribution:

\mu = np \;\; \land \;\; \sigma = \sqrt{np(1-p)}

Poisson Distribution

If $X$ is the number of independent successes occur within a fixed time interval, then $X$ is said to follow a Poisson distribution. And has a PMF:

P(x) = \frac{e^{-\lambda} \lambda^x}{x!}\;\;;\; x=0,1,2,\ldots

Here:

$\lambda$: average number of successes occurred within the fixed time interval

\text{E}(X) = \lambda = \text{Var}(X)

If $X_i \sim \text{Poisson}(\lambda_i)$ where $i = 1, 2, \ldots, n$ , and $X_i$ is independent then:

\sum_{i=1}^n X_i \sim \text{Poisson}\bigg( \sum_{i=1}^n \lambda_i \bigg)