Suppose two independent random samples of size n 1 n_1 n 1 n 2 n_2 n 2 μ 1 \mu_1 μ 1 μ 2 \mu_2 μ 2 σ 1 \sigma_1 σ 1 σ 2 \sigma_2 σ 2
H 0 : μ 1 − μ 2 = d 0 H_0:\ \mu_1 - \mu_2 = d_0 H 0 : μ 1 − μ 2 = d 0 H 1 : μ 1 − μ 2 < d 0 or μ 1 − μ 2 ≠ d 0 or μ 1 − μ 2 > d 0 H_1:\ \mu_1 - \mu_2 < d_0\ \text{ or }\ \mu_1 - \mu_2 \neq d_0\ \text{ or }\ \mu_1 - \mu_2 > d_0 H 1 : μ 1 − μ 2 < d 0 or μ 1 − μ 2 = d 0 or μ 1 − μ 2 > d 0 The appropriate test statistics depends on σ 1 \sigma_1 σ 1 σ 2 \sigma_2 σ 2
Assumptions
Independence
Distribution
Known vs unknown variances
Equal-variance assumption only for pooled t; otherwise use Welch t.
Test Statistic
Case 1
Both σ 1 \sigma_1 σ 1 σ 2 \sigma_2 σ 2
Z = ( X ˉ 1 − X ˉ 2 ) − d 0 σ 1 2 n 1 + σ 2 2 n 2 ∼ N ( 0 , 1 ) under H 0 Z \;=\; \frac{(\bar X_1 - \bar X_2) - d_0}{\sqrt{\frac{\sigma_1^2}{n_1} + \frac{\sigma_2^2}{n_2}}}
\;\sim\; N(0,1)\ \text{under } H_0 Z = n 1 σ 1 2 + n 2 σ 2 2 ( X ˉ 1 − X ˉ 2 ) − d 0 ∼ N ( 0 , 1 ) under H 0 Decision
Two-tailedH 0 H_0 H 0 if ∣ Z ∣ ≥ z 1 − α / 2 |Z|\ge z_{1-\alpha/2} ∣ Z ∣ ≥ z 1 − α /2
Right-tailed ( > ) (>) ( > ) if Z ≥ z 1 − α Z\ge z_{1-\alpha} Z ≥ z 1 − α
Left-tailed ( < ) (<) ( < ) if Z ≤ − z 1 − α Z\le -z_{1-\alpha} Z ≤ − z 1 − α
Confidence interval for d 0 d_0 d 0
( X ˉ 1 − X ˉ 2 ) ± z 1 − α / 2 σ 1 2 n 1 + σ 2 2 n 2 (\bar X_1 - \bar X_2) \pm z_{1-\alpha/2}\sqrt{\frac{\sigma_1^2}{n_1} + \frac{\sigma_2^2}{n_2}} ( X ˉ 1 − X ˉ 2 ) ± z 1 − α /2 n 1 σ 1 2 + n 2 σ 2 2 Case 2
Both σ 1 \sigma_1 σ 1 σ 2 \sigma_2 σ 2
Type I
Variances are equal and unknown. Pooled t statistic is used.
Pooled variance s p s_p s p
s p 2 = ( n 1 − 1 ) s 1 2 + ( n 2 − 1 ) s 2 2 ν s_p^2 \;=\; \frac{(n_1-1)s_1^2 + (n_2-1)s_2^2}{\nu} s p 2 = ν ( n 1 − 1 ) s 1 2 + ( n 2 − 1 ) s 2 2 Here: ν = n 1 + n 2 − 2 \nu = n_1 + n_2 - 2 ν = n 1 + n 2 − 2
The test statistic for this use case is:
t = ( X ˉ 1 − X ˉ 2 ) − d 0 s p 1 n 1 + 1 n 2 ∼ t ν under H 0 t \;=\; \frac{(\bar X_1 - \bar X_2) - d_0}{s_p \sqrt{\frac{1}{n_1} + \frac{1}{n_2}}}
\;\sim\; t_{\nu} \text{under } H_0 t = s p n 1 1 + n 2 1 ( X ˉ 1 − X ˉ 2 ) − d 0 ∼ t ν under H 0 Decision
Two-tailedif ∣ t ∣ ≥ t v , , 1 − α / 2 |t|\ge t_{v,,1-\alpha/2} ∣ t ∣ ≥ t v ,, 1 − α /2
Right-tailedif t ≥ t v , , 1 − α t\ge t_{v,,1-\alpha} t ≥ t v ,, 1 − α
Left-tailedif t ≤ − t v , , 1 − α t\le -t_{v,,1-\alpha} t ≤ − t v ,, 1 − α
Confidence interval for d 0 d_0 d 0
( X ˉ 1 − X ˉ 2 ) ± t v , 1 − α / 2 s p 1 n 1 + 1 n 2 (\bar X_1 - \bar X_2) \pm t_{v,\,1-\alpha/2}\, s_p \sqrt{\tfrac{1}{n_1} + \tfrac{1}{n_2}} ( X ˉ 1 − X ˉ 2 ) ± t v , 1 − α /2 s p n 1 1 + n 2 1 Type II
Unknown and unequal variances. Welch t statistic is used.
Approximate degrees of freedom is:
ν = ( s 1 2 n 1 + s 2 2 n 2 ) 2 ( s 1 2 / n 1 ) 2 n 1 − 1 + ( s 2 2 / n 2 ) 2 n 2 − 1 \nu \;=\; \frac{\big(\frac{s_1^2}{n_1} + \frac{s_2^2}{n_2}\big)^2}{\frac{(s_1^2/n_1)^2}{n_1-1} + \frac{(s_2^2/n_2)^2}{n_2-1}} ν = n 1 − 1 ( s 1 2 / n 1 ) 2 + n 2 − 1 ( s 2 2 / n 2 ) 2 ( n 1 s 1 2 + n 2 s 2 2 ) 2 If ν > 30 \nu\gt 30 ν > 30
t = ( X ˉ 1 − X ˉ 2 ) − d 0 s 1 2 n 1 + s 2 2 n 2 ∼ t v under H 0 t \;=\; \frac{(\bar X_1 - \bar X_2) - d_0}{\sqrt{\frac{s_1^2}{n_1} + \frac{s_2^2}{n_2}}} \sim t_v\ \text{under } H_0 t = n 1 s 1 2 + n 2 s 2 2 ( X ˉ 1 − X ˉ 2 ) − d 0 ∼ t v under H 0 Confidence interval for d 0 d_0 d 0
( X ˉ 1 − X ˉ 2 ) ± t v , 1 − α / 2 s 1 2 n 1 + s 2 2 n 2 (\bar X_1 - \bar X_2) \pm t_{v,\,1-\alpha/2}\, \sqrt{\tfrac{s_1^2}{n_1} + \tfrac{s_2^2}{n_2}} ( X ˉ 1 − X ˉ 2 ) ± t v , 1 − α /2 n 1 s 1 2 + n 2 s 2 2 P-Value
For any computed statistic T T T
• Two-tailed H 1 : μ 1 − μ 2 ≠ d 0 H_1:\mu_1-\mu_2\neq d_0 H 1 : μ 1 − μ 2 = d 0
p -value = 2 min { Pr ( T ≤ t obs ) , Pr ( T ≥ t obs ) } p\text{-value} = 2\,\min\{\Pr(T \le t_{\text{obs}}),\ \Pr(T \ge t_{\text{obs}})\} p -value = 2 min { Pr ( T ≤ t obs ) , Pr ( T ≥ t obs )} • Right-tailed H 1 : μ 1 − μ 2 > d 0 H_1:\mu_1-\mu_2> d_0 H 1 : μ 1 − μ 2 > d 0
p -value = Pr ( T ≥ t obs ) p\text{-value} = \Pr(T \ge t_{\text{obs}}) p -value = Pr ( T ≥ t obs ) • Left-tailed H 1 : μ 1 − μ 2 < d 0 H_1:\mu_1-\mu_2< d_0 H 1 : μ 1 − μ 2 < d 0
p -value = Pr ( T ≤ t obs ) p\text{-value} = \Pr(T \le t_{\text{obs}}) p -value = Pr ( T ≤ t obs ) Decision: reject H 0 H_0 H 0 p -value ≤ α p\text{-value}\le \alpha p -value ≤ α
