Tutorial: Repeated Measures

DABEST version 2023.02.14 expands the repertoire of plots for experiments with repeated-measures designs. DABEST now allows for the calculation and plotting of effect sizes for:

  • Comparing each group to a shared control (control vs. group i; baseline)

  • Comparing each measurement to the one directly preceding it (group i vs group i+1; sequential)

This is an improved version of paired data plotting in previous versions, which only supported computations involving one test group and one control group.

To use these features, you can simply declare the argument paired = "sequential" or paired = "baseline" correspondingly while running dabest.load(). As in the previous version, you must also pass a column in the dataset that indicates the identity of each observation, using the id_col keyword.

(Please note that paired = True and paired = False are no longer valid in v2023.02.14)

Load Libraries

1  import numpy as np
2  import pandas as pd
3  import dabest
4
5  print("We're using DABEST v{}".format(dabest.__version__))
We're using DABEST v2023.02.14

Create dataset for demo

 1  from scipy.stats import norm # Used in generation of populations.
 2
 3  np.random.seed(9999) # Fix the seed so the results are replicable.
 4  # pop_size = 10000 # Size of each population.
 5  Ns = 20 # The number of samples taken from each population
 6
 7  # Create samples
 8  c1 = norm.rvs(loc=3, scale=0.4, size=Ns)
 9  c2 = norm.rvs(loc=3.5, scale=0.75, size=Ns)
10  c3 = norm.rvs(loc=3.25, scale=0.4, size=Ns)
11
12  t1 = norm.rvs(loc=3.5, scale=0.5, size=Ns)
13  t2 = norm.rvs(loc=2.5, scale=0.6, size=Ns)
14  t3 = norm.rvs(loc=3, scale=0.75, size=Ns)
15  t4 = norm.rvs(loc=3.5, scale=0.75, size=Ns)
16  t5 = norm.rvs(loc=3.25, scale=0.4, size=Ns)
17  t6 = norm.rvs(loc=3.25, scale=0.4, size=Ns)
18
19
20  # Add a `gender` column for coloring the data.
21  females = np.repeat('Female', Ns/2).tolist()
22  males = np.repeat('Male', Ns/2).tolist()
23  gender = females + males
24
25  # Add an `id` column for paired data plotting.
26  id_col = pd.Series(range(1, Ns+1))
27
28  # Combine samples and gender into a DataFrame.
29  df = pd.DataFrame({'Control 1' : c1,     'Test 1' : t1,
30                     'Control 2' : c2,     'Test 2' : t2,
31                     'Control 3' : c3,     'Test 3' : t3,
32                     'Test 4'    : t4,     'Test 5' : t5, 'Test 6' : t6,
33                     'Gender'    : gender, 'ID'  : id_col
34                    })
1  two_groups_paired_sequential = dabest.load(df, idx=("Control 1", "Test 1"),
2                                             paired="sequential", id_col="ID")
1  two_groups_paired_sequential
DABEST v2023.02.14
==================

Good evening!
The current time is Mon Aug 30 00:09:54 2021.

Paired effect size(s) for the sequential design of repeated-measures experiment
with 95% confidence intervals will be computed for:
1. Test 1 minus Control 1

5000 resamples will be used to generate the effect size bootstraps.
1  two_groups_paired_baseline = dabest.load(df, idx=("Control 1", "Test 1"),
2                                paired="baseline", id_col="ID")
1  two_groups_paired_baseline
DABEST v2023.02.14
==================

Good evening!
The current time is Mon Aug 30 00:13:17 2021.

Paired effect size(s) for repeated measures against baseline
with 95% confidence intervals will be computed for:
1. Test 1 minus Control 1

5000 resamples will be used to generate the effect size bootstraps.

When only 2 paired data groups are involved, assigning either baseline or sequential to paired will give you the same numerical results.

1  two_groups_paired_sequential.mean_diff
DABEST v2023.02.14
==================

Good evening!
The current time is Mon Aug 30 00:14:44 2021.

The paired mean difference for the sequential design of repeated-measures experiment
between Control 1 and Test 1 is 0.48 [95%CI 0.237, 0.73].
The p-value of the two-sided permutation t-test is 0.001, calculated for legacy purposes only.

5000 bootstrap samples were taken; the confidence interval is bias-corrected and accelerated.
Any p-value reported is the probability of observing theeffect size (or greater),
assuming the null hypothesis ofzero difference is true.
For each p-value, 5000 reshuffles of the control and test labels were performed.

To get the results of all valid statistical tests, use .mean_diff.statistical_tests
1  two_groups_paired_baseline.mean_diff
DABEST v2023.02.14
==================

Good evening!
The current time is Mon Aug 30 00:18:09 2021.

The paired mean difference for repeated measures against baseline
between Control 1 and Test 1 is 0.48 [95%CI 0.237, 0.73].
The p-value of the two-sided permutation t-test is 0.001, calculated for legacy purposes only.

5000 bootstrap samples were taken; the confidence interval is bias-corrected and accelerated.
Any p-value reported is the probability of observing the effect size (or greater),
assuming the null hypothesis ofzero difference is true.
For each p-value, 5000 reshuffles of the control and test labels were performed.

To get the results of all valid statistical tests, use .mean_diff.statistical_tests

For paired data, we use slopegraphs (another innovation from Edward Tufte) to connect paired observations. Both Gardner-Altman and Cumming plots support this.

1  two_groups_paired_sequential.mean_diff.plot();
_images/tutorial_32_0.png
1  two_groups_paired_sequential.mean_diff.plot(float_contrast=False);
_images/tutorial_33_0.png
1  two_groups_paired_baseline.mean_diff.plot();
_images/tutorial_32_0.png
1  two_groups_paired_baseline.mean_diff.plot(float_contrast=False);
_images/tutorial_33_0.png

You can also create repeated-measures plots with multiple test groups. In this case, declaring paired to be sequential or baseline will generate the same slopegraph, reflecting the repeated-measures experimental design, but different contrast plots, to show the sequential or baseline comparison:

1  sequential_repeated_measures = dabest.load(df, idx=("Control 1", "Test 1", "Test 2", "Test 3"),
2                                             paired="sequential", id_col="ID")
1  sequential_repeated_measures.mean_diff
DABEST v2023.02.14
==================

Good evening!
The current time is Mon Aug 30 00:51:21 2021.

The paired mean difference for the sequential design of repeated-measures experiment
between Control 1 and Test 1 is 0.48 [95%CI 0.237, 0.73].
The p-value of the two-sided permutation t-test is 0.001, calculated for legacy purposes only.

The paired mean difference for the sequential design of repeated-measures experiment
between Test 1 and Test 2 is -1.02 [95%CI -1.36, -0.716].
The p-value of the two-sided permutation t-test is 0.0, calculated for legacy purposes only.

The paired mean difference for the sequential design of repeated-measures experiment
between Test 2 and Test 3 is 0.716 [95%CI 0.14, 1.22].
The p-value of the two-sided permutation t-test is 0.022, calculated for legacy purposes only.

5000 bootstrap samples were taken; the confidence interval is bias-corrected and accelerated.
Any p-value reported is the probability of observing theeffect size (or greater),
assuming the null hypothesis ofzero difference is true.
For each p-value, 5000 reshuffles of the control and test labels were performed.

To get the results of all valid statistical tests, use .mean_diff.statistical_tests
1  sequential_repeated_measures.mean_diff.plot();
_images/tutorial_103_0.png
1  baseline_repeated_measures = dabest.load(df, idx=("Control 1", "Test 1", "Test 2", "Test 3"),
2                                             paired="baseline", id_col="ID")
1  baseline_repeated_measures.mean_diff
DABEST v2023.02.14
==================

Good evening!
The current time is Mon Aug 30 00:56:37 2021.

The paired mean difference for repeated measures against baseline
between Control 1 and Test 1 is 0.48 [95%CI 0.237, 0.73].
The p-value of the two-sided permutation t-test is 0.001, calculated for legacy purposes only.

The paired mean difference for repeated measures against baseline
between Control 1 and Test 2 is -0.542 [95%CI -0.975, -0.198].
The p-value of the two-sided permutation t-test is 0.014, calculated for legacy purposes only.

The paired mean difference for repeated measures against baseline
between Control 1 and Test 3 is 0.174 [95%CI -0.297, 0.706].
The p-value of the two-sided permutation t-test is 0.505, calculated for legacy purposes only.

5000 bootstrap samples were taken; the confidence interval is bias-corrected and accelerated.
Any p-value reported is the probability of observing theeffect size (or greater),
assuming the null hypothesis ofzero difference is true.
For each p-value, 5000 reshuffles of the control and test labels were performed.

To get the results of all valid statistical tests, use .mean_diff.statistical_tests
1  baseline_repeated_measures.mean_diff.plot();
_images/tutorial_104_0.png

As with unpaired data, DABEST empowers you to perform complex visualizations and statistics for paired data as well.

1  multi_baseline_repeated_measures = dabest.load(df, idx=(("Control 1", "Test 1", "Test 2", "Test 3"),
2                                                    ("Control 2", "Test 4", "Test 5", "Test 6")),
3                                             paired="baseline", id_col="ID")
4  multi_baseline_repeated_measures.mean_diff.plot();
_images/tutorial_105_0.png