The Barplot

Let’s say we have performed an experiment with 20 control subjects, and 20 test subjects. We begin our data analysis by making a barplot of the data.

The barplot has several shortcomings, despite enjoying widespread usage in academic journals. We’re not the first ones (see this, this, or that) to point out the myriad flaws with the barplot. Importantly, the barplot does not show us the effect size.

Alternatively, we can use a boxplot to visualize the data.

The Boxplot

Unfortunately, the boxplot still doesn’t show all our data. We still lack information about the underlying distribution of your data. Is it normally distributed? Is there skew in the points? What is the sample size? More importantly, boxplots do not display the effect size.

To display several data points across one or more categories, we can use the jitter plot.

The Jitter Plot

Jitter plots avoid overlapping datapoints (i.e. datapoints with the same y-value) by adding a random factor to each point along the orthogonal x-axes. Thus, while a jitter plot displays all datapoints (implicitly indicating the sample size visually), it might not accurately depict the underlying distribution of the data.

Estimation Statistics

Estimation plots emerge from estimation statistics a simple framework that avoids the pitfalls of significance testing. It focuses on the effect sizes of one’s experiment/interventions, and uses familiar statistical concepts: means, mean differences, and error bars.

Significance testing calculates the probability (the P value) that the experimental data would be observed, if the intervention did not produce a change in the metric measured (i.e. the null hypothesis). This leads analysts to apply a false dichotomy on the experimental intervention.

Estimation statistics, on the other hand, focuses on the magnitude of the effect (the effect size) and its precision. This encourages analysts to gain a deeper understanding of the metrics used, and how they relate to the natural processes being studied.

An Estimation Plot For Every Experimental Design

For each of the most routine significance tests, there is an estimation replacement:

Unpaired Student’s t-test → Two-group estimation plot

Paired Student’s t-test → Paired estimation plot

The Gardner-Altman estimation plot can also display effect sizes for repeated measures (aka a paired experimental design) using a Tufte slopegraph instead of a swarmplot.

For comparisons between 3 or more groups that typically employ analysis of variance (ANOVA) methods, one can use the Cumming estimation plot33 The Cumming plot is named after Geoff Cumming, and draws its design heavily from his 2012 textbook Understanding the New Statistics., which can be considered a variant of the Gardner-Altman plot.

One-way ANOVA + multiple comparisons → Multi two-group estimation plot

The effect size and 95% CIs are still plotted a separate axes, but unlike the Gardner-Altman plot, this axes is positioned beneath the raw data. Such a design frees up visual space in the upper panel, allowing the display of summary measurements (mean ± standard deviation) for each group. These are shown as gapped lines to the right of each group. The mean of each group is indicated as a gap in the line, adhering to Edward Tufte’s dictum to keep the data-ink ratio low.

Repeated measures ANOVA → Multi paired estimation plot

Ordered groups ANOVA → Shared-control estimation plot