Student's t-test is a statistical test used to test whether the difference between the response of two groups is statistically significant or not. It is any statistical hypothesis test in which the test statistic follows a Student's t-distribution under the null hypothesis. It is most commonly applied when the test statistic would follow a normal distribution if the value of a scaling term in the test statistic were known (typically, the scaling term is unknown and is therefore a nuisance parameter). When the scaling term is estimated based on the data, the test statistic—under certain conditions—follows a Student's t distribution. The t-test's most common application is to test whether the means of two populations are significantly different. In many cases, a Z-test will yield very similar results to a t-test because the latter converges to the former as the size of the dataset increases.
Contents
History
The term "t-statistic" is abbreviated from "hypothesis test statistic". In statistics, the t-distribution was first derived as a posterior distribution in 1876 by Helmert and Lüroth. The t-distribution also appeared in a more general form as Pearson type IV distribution in Karl Pearson's 1895 paper. However, the t-distribution, also known as Student's t-distribution, gets its name from William Sealy Gosset, who first published it in English in 1908 in the scientific journal Biometrika using the pseudonym "Student" because his employer preferred staff to use pen names when publishing scientific papers. Gosset worked at the Guinness Brewery in Dublin, Ireland, and was interested in the problems of small samples – for example, the chemical properties of barley with small sample sizes. Hence a second version of the etymology of the term Student is that Guinness did not want their competitors to know that they were using the t-test to determine the quality of raw material. Although it was William Gosset after whom the term "Student" was coined, it was actually through the work of Ronald Fisher that the distribution became well known as "Student's distribution" and "Student's t-test".
Gosset devised the t-test as an economical way to monitor the quality of stout. The t-test work was submitted to and accepted in the journal Biometrika and published in 1908.
Guinness had a policy of allowing technical staff leave for study (so-called "study leave"), which Gosset used during the first two terms of the 1906–1907 academic year in Professor Karl Pearson's Biometric Laboratory at University College London. Gosset's identity was then known to fellow statisticians and to editor-in-chief Karl Pearson.
Uses
One-sample t-test
A one-sample Student's t-test is a location test of whether the mean of a population has a value specified in a null hypothesis. In testing the null hypothesis that the population mean is equal to a specified value μ0, one uses the statistic
t
=
x
¯
−
μ
0
s
/
n
,
{\displaystyle t={\frac {{\bar {x}}-\mu _{0}}{s/{\sqrt {n}}}},}
where
x
¯
{\displaystyle {\bar {x}}}
is the sample mean, s is the sample standard deviation and n is the sample size. The degrees of freedom used in this test are n − 1. Although the parent population does not need to be normally distributed, the distribution of the population of sample means
x
¯
{\displaystyle {\bar {x}}}
is assumed to be normal.
By the central limit theorem, if the observations are independent and the second moment exists, then
t
{\displaystyle t}
Two-sample t-tests
A two-sample location test of the null hypothesis that the means of two populations are equal. All such tests are usually called Student's t-tests, though strictly speaking that name should only be used if the variances of the two populations are also assumed to be equal; the form of the test used when this assumption is dropped is sometimes called Welch's t-test. These tests are often referred to as unpaired or independent samples t-tests, as they are typically applied when the statistical units underlying the two samples being compared are non-overlapping.
Two-sample t-tests for a difference in means involve independent samples (unpaired samples) or paired samples. Paired t-tests are a form of blocking, and have greater power (probability of avoiding a type II error, also known as a false negative) than unpaired tests when the paired units are similar with respect to "noise factors" (see confounder) that are independent of membership in the two groups being compared. In a different context, paired t-tests can be used to reduce the effects of confounding factors in an observational study.
The independent samples t-test is used when two separate sets of independent and identically distributed samples are obtained, and one variable from each of the two populations is compared. For example, suppose we are evaluating the effect of a medical treatment, and we enroll 100 subjects into our study, then randomly assign 50 subjects to the treatment group and 50 subjects to the control group. In this case, we have two independent samples and would use the unpaired form of the t-test.
Paired samples t-tests typically consist of a sample of matched pairs of similar units, or one group of units that has been tested twice (a "repeated measures" t-test).
A typical example of the repeated measures t-test would be where subjects are examined prior to a treatment, say for high blood pressure, and the same subjects are examined again after treatment with a blood-pressure-lowering medication. By comparing the same patient's numbers before and after treatment, we are effectively using each patient as their own control. That way the correct rejection of the null hypothesis (here: of no difference made by the treatment) can become much more likely, with statistical power increasing simply because the random inter-patient variation has now been eliminated. However, an increase of statistical power comes at a price: more examinations are required, each subject having to be examined twice.
Assumptions
Most test statistics have the form t = Z/s, where Z and s are functions of the data.
Z may be sensitive to the alternative hypothesis (i.e., its magnitude tends to be larger when the alternative hypothesis is true), whereas s is a scaling parameter that allows the distribution of t to be determined.
As an example, in the one-sample t-test
t
=
Z
s
=
X
¯
−
μ
σ
^
/
n
,
{\displaystyle t={\frac {Z}{s}}={\frac {{\bar {X}}-\mu }{{\hat {\sigma }}/{\sqrt {n}}}},}
where
X
¯
{\displaystyle {\bar {X}}}
is the sample mean from a sample X1, X2, …, Xn, of size n, s is the standard error of the mean,
σ
^
=
1
Calculations
Explicit expressions that can be used to carry out various t-tests are given below. In each case, the formula for a test statistic that either exactly follows or closely approximates a t-distribution under the null hypothesis is given. Also, the appropriate degrees of freedom are given in each case. Each of these statistics can be used to carry out either a one-tailed or two-tailed test.
Once the t value and degrees of freedom are determined, a p-value can be found using a table of values from Student's t-distribution. If the calculated p-value is below the threshold chosen for statistical significance (usually the 0.10, the 0.05, or 0.01 level), then the null hypothesis is rejected in favor of the alternative hypothesis.
Slope of a regression line
Suppose one is fitting the model
Y
=
α
+
β
x
+
ε
,
{\displaystyle Y=\alpha +\beta x+\varepsilon ,}
where x is known, α and β are unknown, ε is a normally distributed random variable with mean 0 and unknown variance σ2, and Y is the outcome of interest. We want to test the null hypothesis that the slope β is equal to some specified value β0 (often taken to be 0, in which case the null hypothesis is that x and y are uncorrelated).
Let
α
^
,
β
^
=
least-squares estimators
,
S
E
α
^
,
S
E
β
^
=
Independent two-sample t-test
Given two groups (1, 2), this test is only applicable when:
the two sample sizes are equal,
it can be assumed that the two distributions have the same variance.
Violations of these assumptions are discussed below.
The t statistic to test whether the means are different can be calculated as follows:
t
=
X
¯
1
−
X
¯
2
s
p
2
n
,
{\displaystyle t={\frac {{\bar {X}}_{1}-{\bar {X}}_{2}}{s_{p}{\sqrt {\frac {2}{n}}}}},}
where
s
p
=
s
X
1
2
+
Exact method for unequal variances and sample sizes
The test deals with the famous Behrens–Fisher problem, i.e., comparing the difference between the means of two normally distributed populations when the variances of the two populations are not assumed to be equal, based on two independent samples.
The test is developed as an exact test that allows for unequal sample sizes and unequal variances of two populations. The exact property still holds even with extremely small and unbalanced sample sizes (e.g.
m
≡
n
X
=
50
{\displaystyle \ m\equiv n_{\mathsf {X}}=50\ }
vs.
n
≡
n
Y
=
5
{\displaystyle \ n\equiv n_{\mathsf {Y}}=5\ }
).
The statistic to test whether the means are different can be calculated as follows:
Let
X
=
[
X
1
,
Dependent t-test for paired samples
This test is used when the samples are dependent; that is, when there is only one sample that has been tested twice (repeated measures) or when there are two samples that have been matched or "paired". This is an example of a paired difference test. The t statistic is calculated as
t
=
X
¯
D
−
μ
0
s
D
/
n
,
{\displaystyle t={\frac {{\bar {X}}_{D}-\mu _{0}}{s_{D}/{\sqrt {n}}}},}
where
X
¯
D
{\displaystyle {\bar {X}}_{D}}
and
s
D
{\displaystyle s_{D}}
are the average and standard deviation of the differences between all pairs. The pairs are e.g. either one person's pre-test and post-test scores or between-pairs of persons matched into meaningful groups (for instance, drawn from the same family or age group: see table). The constant μ0 is zero if we want to test whether the average of the difference is significantly different. The degree of freedom used is n − 1, where n represents the number of pairs.
Worked examples
Let A1 denote a set obtained by drawing a random sample of six measurements:
A
1
=
{
30.02
,
29.99
,
30.11
,
29.97
,
30.01
,
29.99
}
{\displaystyle A_{1}=\{30.02,\ 29.99,\ 30.11,\ 29.97,\ 30.01,\ 29.99\}}
and let A2 denote a second set obtained similarly:
A
2
=
{
29.89
,
29.93
,
29.72
,
29.98
,
30.02
,
Unequal variances
If the approach for unequal variances (discussed above) is followed, the results are
s
1
2
n
1
+
s
2
2
n
2
≈
0.04849
{\displaystyle {\sqrt {{\frac {s_{1}^{2}}{n_{1}}}+{\frac {s_{2}^{2}}{n_{2}}}}}\approx 0.04849}
and the degrees of freedom
d.f.
≈
7.031.
{\displaystyle {\text{d.f.}}\approx 7.031.}
The test statistic is approximately 1.959, which gives a two-tailed test p-value of 0.09077.
Equal variances
If the approach for equal variances (discussed above) is followed, the results are
s
p
≈
0.08399
{\displaystyle s_{p}\approx 0.08399}
and the degrees of freedom
d.f.
=
10.
{\displaystyle {\text{d.f.}}=10.}
The test statistic is approximately equal to 1.959, which gives a two-tailed p-value of 0.07857.
Related statistical tests
Alternatives to the t-test for location problems
The t-test provides an exact test for the equality of the means of two i.i.d. normal populations with unknown, but equal, variances. (Welch's t-test is a nearly exact test for the case where the data are normal but the variances may differ.) For moderately large samples and a one tailed test, the t-test is relatively robust to moderate violations of the normality assumption. In large enough samples, the t-test asymptotically approaches the z-test, and becomes robust even to large deviations from normality.
If the data are substantially non-normal and the sample size is small, the t-test can give misleading results. See Location test for Gaussian scale mixture distributions for some theory related to one particular family of non-normal distributions.
When the normality assumption does not hold, a non-parametric alternative to the t-test may have better statistical power. However, when data are non-normal with differing variances between groups, a t-test may have better type-1 error control than some non-parametric alternatives. Furthermore, non-parametric methods, such as the Mann-Whitney U test discussed below, typically do not test for a difference of means, so should be used carefully if a difference of means is of primary scientific interest. For example, Mann-Whitney U test will keep the type 1 error at the desired level alpha if both groups have the same distribution. It will also have power in detecting an alternative by which group B has the same distribution as A but after some shift by a constant (in which case there would indeed be a difference in the means of the two groups). However, there could be cases where group A and B will have different distributions but with the same means (such as two distributions, one with positive skewness and the other with a negative one, but shifted so to have the same means). In such cases, MW could have more than alpha level power in rejecting the Null hypothesis but attributing the interpretation of difference in means to such a result would be incorrect.
In the presence of an outlier, the t-test is not robust. For example, for two independent samples when the data distributions are asymmetric (that is, the distributions are skewed) or the distributions have large tails, then the Wilcoxon rank-sum test (also known as the Mann–Whitney U test) can have three to four times higher power than the t-test. The nonparametric counterpart to the paired samples t-test is the Wilcoxon signed-rank test for paired samples. For a discussion on choosing between the t-test and nonparametric alternatives, see Lumley, et al. (2002).
A design which includes both paired observations and independent observations
When both paired observations and independent observations are present in the two sample design, assuming data are missing completely at random (MCAR), the paired observations or independent observations may be discarded in order to proceed with the standard tests above. Alternatively making use of all of the available data, assuming normality and MCAR, the generalized partially overlapping samples t-test could be used.
Multivariate testing
A generalization of Student's t statistic, called Hotelling's t-squared statistic, allows for the testing of hypotheses on multiple (often correlated) measures within the same sample. For instance, a researcher might submit a number of subjects to a personality test consisting of multiple personality scales (e.g. the Minnesota Multiphasic Personality Inventory). Because measures of this type are usually positively correlated, it is not advisable to conduct separate univariate t-tests to test hypotheses, as these would neglect the covariance among measures and inflate the chance of falsely rejecting at least one hypothesis (Type I error). In this case a single multivariate test is preferable for hypothesis testing. Fisher's Method for combining multiple tests with alpha reduced for positive correlation among tests is one. Another is Hotelling's T2 statistic follows a T2 distribution. However, in practice the distribution is rarely used, since tabulated values for T2 are hard to find. Usually, T2 is converted instead to an F statistic.
For a one-sample multivariate test, the hypothesis is that the mean vector (μ) is equal to a given vector (μ0). The test statistic is Hotelling's t2:
t
2
=
n
(
x
¯
−
μ
0
)
′
S
−
1
(
x
¯
−
μ
The two-sample t-test is a special case of simple linear regression
The two-sample t-test is a special case of simple linear regression. This relationship is illustrated by the following example.
A clinical trial examines 6 patients given a drug or a placebo. Three (3) patients get 0 units of drug (the placebo group). Three (3) patients get 1 unit of drug (the active treatment group). At the end of treatment, the researchers measure the change from baseline in the number of words that each patient can recall in a memory test.
A table of the patients' word recall and drug dose values is shown below.
Data and code are given for the analysis using the R programming language with the t.test and lmfunctions for the t-test and linear regression. Here are the same (fictitious) data above generated in R.
Perform the t-test. Notice that the assumption of equal variance, var.equal=T, is required to make the analysis exactly equivalent to simple linear regression.
Running the R code gives the following results.
The mean word.recall in the 0 drug.dose group is 2.
The mean word.recall in the 1 drug.dose group is 6.
The difference between treatment groups in the mean word.recall is 6 – 2 = 4.
The difference in word.recall between drug doses is significant (p=0.00805).
Perform a linear regression of the same data. Calculations may be performed using the R function lm() for a linear model.
Power and sample size for t-tests
The power of a test is the probability that the test rejects the null hypothesis when the alternative hypothesis is true.
Power calculation for a two-sample t-test requires the following information
.
The difference between the means of the two groups
The within-group standard deviation (if the two groups have the same standard deviation)
The sample size (number of subjects) in each group
The required p-value (alpha) for significance
To calculate power, it can be useful to compute the standardized effect size, which is the difference between the two means divided by the within-group standard deviation. For example, suppose that the mean of group A is 14 and the mean of group B is 10, the within-group standard deviation is 8 units (assuming the two groups have the same standard deviation). Then the difference between group means is 14-10 = 4 units, and the standardized effect size is (14-10)/8 = 4/8 = 0.5.
The graph below shows power for standardized effect sizes from 0.1 to 1, for sample sizes per group from 10 to 50, assuming an equal number of subjects per group. N per group is the number of observations in each group. For example, for a standardized effect size of 0.5, a sample size of N = 10 per group gives power slightly less than 0.2, while a sample size of N = 50 per group gives power of approximately 0.7.
Calculators for power and sample size are available at many websites, such as the following.
Sample size calculator
Sample Size Calculator for Comparing Two Independent Means
Software implementations
Many spreadsheet programs and statistics packages, such as QtiPlot, LibreOffice Calc, Microsoft Excel, SAS, SPSS, Stata, DAP, gretl, R, Python, PSPP, Wolfram Mathematica, MATLAB and Minitab, include implementations of Student's t-test.