API

Loading Data

dabest.load(data, idx=None, x=None, y=None, paired=None, id_col=None, ci=95, resamples=5000, random_seed=12345, proportional=False, delta2=False, experiment=None, experiment_label=None, x1_level=None, mini_meta=False)

Loads data in preparation for estimation statistics.

This is designed to work with pandas DataFrames.

Parameters:

  • data (pandas DataFrame) –

  • idx (tuple) – List of column names (if ‘x’ is not supplied) or of category names (if ‘x’ is supplied). This can be expressed as a tuple of tuples, with each individual tuple producing its own contrast plot

  • x (string or list, default None) – Column name(s) of the independent variable. This can be expressed as a list of 2 elements if and only if ‘delta2’ is True; otherwise it can only be a string.

  • y (string, default None) – Column names for data to be plotted on the x-axis and y-axis.

  • paired (string, default None) – The type of the experiment under which the data are obtained. If ‘paired’ is None then the data will not be treated as paired data in the subsequent calculations. If ‘paired’ is ‘baseline’, then in each tuple of x, other groups will be paired up with the first group (as control). If ‘paired’ is ‘sequential’, then in each tuple of x, each group will be paired up with its previous group (as control).

  • id_col (default None.) – Required if paired is True.

  • ci (integer, default 95) – The confidence interval width. The default of 95 produces 95% confidence intervals.

  • resamples (integer, default 5000.) – The number of resamples taken to generate the bootstraps which are used to generate the confidence intervals.

  • random_seed (int, default 12345) – This integer is used to seed the random number generator during bootstrap resampling, ensuring that the confidence intervals reported are replicable.

  • proportional (boolean, default False.) – An indicator of whether the data is binary or not. When set to True, it specifies that the data consists of binary data, where the values are limited to 0 and 1. The code is not suitable for analyzing proportion data that contains non-numeric values, such as strings like ‘yes’ and ‘no’. When False or not provided, the algorithm assumes that the data is continuous and uses a non-proportional representation.

  • delta2 (boolean, default False) – Indicator of delta-delta experiment

  • experiment (String, default None) – The name of the column of the dataframe which contains the label of experiments

  • experiment_lab (list, default None) – A list of String to specify the order of subplots for delta-delta plots. This can be expressed as a list of 2 elements if and only if ‘delta2’ is True; otherwise it can only be a string.

  • x1_level (list, default None) – A list of String to specify the order of subplots for delta-delta plots. This can be expressed as a list of 2 elements if and only if ‘delta2’ is True; otherwise it can only be a string.

  • mini_meta (boolean, default False) – Indicator of weighted delta calculation.

Return type:

A Dabest object.

Example

Load libraries.

>>> import numpy as np
>>> import pandas as pd
>>> import dabest

Create dummy data for demonstration.

>>> np.random.seed(88888)
>>> N = 10
>>> c1 = sp.stats.norm.rvs(loc=100, scale=5, size=N)
>>> t1 = sp.stats.norm.rvs(loc=115, scale=5, size=N)
>>> df = pd.DataFrame({'Control 1' : c1, 'Test 1': t1})

Load the data.

>>> my_data = dabest.load(df, idx=("Control 1", "Test 1"))

For proportion plot.

>>> np.random.seed(88888)
>>> N = 10
>>> c1 = np.random.binomial(1, 0.2, size=N)
>>> t1 = np.random.binomial(1, 0.5, size=N)
>>> df = pd.DataFrame({'Control 1' : c1, 'Test 1': t1})
>>> my_data = dabest.load(df, idx=("Control 1", "Test 1"),proportional=True)

Computing Effect Sizes

class dabest._classes.Dabest(data, idx, x, y, paired, id_col, ci, resamples, random_seed, proportional, delta2, experiment, experiment_label, x1_level, mini_meta)

Parses and stores pandas DataFrames in preparation for estimation statistics. You should not be calling this class directly; instead, use dabest.load() to parse your DataFrame prior to analysis.

property mean_diff

Returns an EffectSizeDataFrame for the mean difference, its confidence interval, and relevant statistics, for all comparisons as indicated via the idx and paired argument in dabest.load().

Example

>>> from scipy.stats import norm
>>> import pandas as pd
>>> import dabest
>>> control = norm.rvs(loc=0, size=30, random_state=12345)
>>> test    = norm.rvs(loc=0.5, size=30, random_state=12345)
>>> my_df   = pd.DataFrame({"control": control,
                            "test": test})
>>> my_dabest_object = dabest.load(my_df, idx=("control", "test"))
>>> my_dabest_object.mean_diff

Notes

This is simply the mean of the control group subtracted from the mean of the test group.

\[\text{Mean difference} = \overline{x}_{Test} - \overline{x}_{Control}\]

where \(\overline{x}\) is the mean for the group \(x\).

property median_diff

Returns an EffectSizeDataFrame for the median difference, its confidence interval, and relevant statistics, for all comparisons as indicated via the idx and paired argument in dabest.load().

Example

>>> from scipy.stats import norm
>>> import pandas as pd
>>> import dabest
>>> control = norm.rvs(loc=0, size=30, random_state=12345)
>>> test    = norm.rvs(loc=0.5, size=30, random_state=12345)
>>> my_df   = pd.DataFrame({"control": control,
                            "test": test})
>>> my_dabest_object = dabest.load(my_df, idx=("control", "test"))
>>> my_dabest_object.median_diff

Notes

This is the median difference between the control group and the test group.

If the comparison(s) are unpaired, median_diff is computed with the following equation:

\[\text{Median difference} = \widetilde{x}_{Test} - \widetilde{x}_{Control}\]

where \(\widetilde{x}\) is the median for the group \(x\).

If the comparison(s) are paired, median_diff is computed with the following equation:

\[\text{Median difference} = \widetilde{x}_{Test - Control}\]

Things to note

Using median difference as the statistic in bootstrapping may result in a biased estimate and cause problems with BCa confidence intervals. Consider using mean difference instead.

When plotting, consider using percentile confidence intervals instead of BCa confidence intervals by specifying ci_type = ‘percentile’ in .plot().

For detailed information, please refer to Issue 129.

property cohens_d

Returns an EffectSizeDataFrame for the standardized mean difference Cohen’s d, its confidence interval, and relevant statistics, for all comparisons as indicated via the idx and paired argument in dabest.load().

Example

>>> from scipy.stats import norm
>>> import pandas as pd
>>> import dabest
>>> control = norm.rvs(loc=0, size=30, random_state=12345)
>>> test    = norm.rvs(loc=0.5, size=30, random_state=12345)
>>> my_df   = pd.DataFrame({"control": control,
                            "test": test})
>>> my_dabest_object = dabest.load(my_df, idx=("control", "test"))
>>> my_dabest_object.cohens_d

Notes

Cohen’s d is simply the mean of the control group subtracted from the mean of the test group.

If paired is None, then the comparison(s) are unpaired; otherwise the comparison(s) are paired.

If the comparison(s) are unpaired, Cohen’s d is computed with the following equation:

\[d = \frac{\overline{x}_{Test} - \overline{x}_{Control}} {\text{pooled standard deviation}}\]

For paired comparisons, Cohen’s d is given by

\[d = \frac{\overline{x}_{Test} - \overline{x}_{Control}} {\text{average standard deviation}}\]

where \(\overline{x}\) is the mean of the respective group of observations, \({Var}_{x}\) denotes the variance of that group,

\[\text{pooled standard deviation} = \sqrt{ \frac{(n_{control} - 1) * {Var}_{control} + (n_{test} - 1) * {Var}_{test} } {n_{control} + n_{test} - 2} }\]

and

\[\text{average standard deviation} = \sqrt{ \frac{{Var}_{control} + {Var}_{test}} {2}}\]

The sample variance (and standard deviation) uses N-1 degrees of freedoms. This is an application of Bessel’s correction, and yields the unbiased sample variance.

References

https://en.wikipedia.org/wiki/Effect_size#Cohen’s_d https://en.wikipedia.org/wiki/Bessel%27s_correction https://en.wikipedia.org/wiki/Standard_deviation#Corrected_sample_standard_deviation

property cohens_h

Returns an EffectSizeDataFrame for the standardized mean difference Cohen’s h, its confidence interval, and relevant statistics, for all comparisons as indicated via the idx and directional argument in dabest.load().

Example

>>> from scipy.stats import randint
>>> import pandas as pd
>>> import dabest
>>> control = randint.rvs(0, 2, size=30, random_state=12345)
>>> test    = randint.rvs(0, 2, size=30, random_state=12345)
>>> my_df   = pd.DataFrame({"control": control,
                            "test": test})
>>> my_dabest_object = dabest.load(my_df, idx=("control", "test")
>>> my_dabest_object.cohens_h

Notes

Cohen’s h uses the information of proportion in the control and test groups to calculate the distance between two proportions. It can be used to describe the difference between two proportions as “small”, “medium”, or “large”. It can be used to determine if the difference between two proportions is “meaningful”.

A directional Cohen’s h is computed with the following equation:

\[h = 2 * \arcsin{\sqrt{proportion_{Test}}} - 2 * \arcsin{\sqrt{proportion_{Control}}}\]

For a non-directional Cohen’s h, the equation is:

\[h = |2 * \arcsin{\sqrt{proportion_{Test}}} - 2 * \arcsin{\sqrt{proportion_{Control}}}|\]

References

https://en.wikipedia.org/wiki/Cohen%27s_h

property hedges_g

Returns an EffectSizeDataFrame for the standardized mean difference Hedges’ g, its confidence interval, and relevant statistics, for all comparisons as indicated via the idx and paired argument in dabest.load().

Example

>>> from scipy.stats import norm
>>> import pandas as pd
>>> import dabest
>>> control = norm.rvs(loc=0, size=30, random_state=12345)
>>> test    = norm.rvs(loc=0.5, size=30, random_state=12345)
>>> my_df   = pd.DataFrame({"control": control,
                            "test": test})
>>> my_dabest_object = dabest.load(my_df, idx=("control", "test"))
>>> my_dabest_object.hedges_g

Notes

Hedges’ g is cohens_d corrected for bias via multiplication with the following correction factor:

\[\frac{ \Gamma( \frac{a} {2} )} {\sqrt{ \frac{a} {2} } \times \Gamma( \frac{a - 1} {2} )}\]

where

\[a = {n}_{control} + {n}_{test} - 2\]

and \(\Gamma(x)\) is the Gamma function.

References

https://en.wikipedia.org/wiki/Effect_size#Hedges’_g https://journals.sagepub.com/doi/10.3102/10769986006002107

property cliffs_delta

Returns an EffectSizeDataFrame for Cliff’s delta, its confidence interval, and relevant statistics, for all comparisons as indicated via the idx and paired argument in dabest.load().

Example

>>> from scipy.stats import norm
>>> import pandas as pd
>>> import dabest
>>> control = norm.rvs(loc=0, size=30, random_state=12345)
>>> test    = norm.rvs(loc=0.5, size=30, random_state=12345)
>>> my_df   = pd.DataFrame({"control": control,
                            "test": test})
>>> my_dabest_object = dabest.load(my_df, idx=("control", "test"))
>>> my_dabest_object.cliffs_delta

Notes

Cliff’s delta is a measure of ordinal dominance, ie. how often the values from the test sample are larger than values from the control sample.

\[\text{Cliff's delta} = \frac{\#({x}_{test} > {x}_{control}) - \#({x}_{test} < {x}_{control})} {{n}_{Test} \times {n}_{Control}}\]

where \(\#\) denotes the number of times a value from the test sample exceeds (or is lesser than) values in the control sample.

Cliff’s delta ranges from -1 to 1; it can also be thought of as a measure of the degree of overlap between the two samples. An attractive aspect of this effect size is that it does not make an assumptions about the underlying distributions that the samples were drawn from.

References

https://en.wikipedia.org/wiki/Effect_size#Effect_size_for_ordinal_data https://psycnet.apa.org/record/1994-08169-001

class dabest._classes.MiniMetaDelta(effectsizedataframe, permutation_count, ci=95)

A class to compute and store the weighted delta. A weighted delta is calculated if the argument mini_meta=True is passed during dabest.load().

The weighted delta is calcuated as follows:

\[\theta_{\text{weighted}} = \frac{\Sigma\hat{\theta_{i}}w_{i}}{{\Sigma}w_{i}}\]

where:

\[\hat{\theta_{i}} = \text{Mean difference for replicate }i\]
\[w_{i} = \text{Weight for replicate }i = \frac{1}{s_{i}^2}\]
\[s_{i}^2 = \text{Pooled variance for replicate }i = \frac{(n_{test}-1)s_{test}^2+(n_{control}-1)s_{control}^2}{n_{test}+n_{control}-2}\]
\[n = \text{sample size and }s^2 = \text{variance for control/test.}\]

Example

>>> from scipy.stats import norm
>>> import pandas as pd
>>> import dabest
>>> Ns = 20
>>> c1 = norm.rvs(loc=3, scale=0.4, size=Ns)
>>> c2 = norm.rvs(loc=3.5, scale=0.75, size=Ns)
>>> c3 = norm.rvs(loc=3.25, scale=0.4, size=Ns)
>>> t1 = norm.rvs(loc=3.5, scale=0.5, size=Ns)
>>> t2 = norm.rvs(loc=2.5, scale=0.6, size=Ns)
>>> t3 = norm.rvs(loc=3, scale=0.75, size=Ns)
>>> my_df   = pd.DataFrame({'Control 1' : c1,     'Test 1' : t1,
                   'Control 2' : c2,     'Test 2' : t2,
                   'Control 3' : c3,     'Test 3' : t3})
>>> my_dabest_object = dabest.load(my_df, idx=(("Control 1", "Test 1"), ("Control 2", "Test 2"), ("Control 3", "Test 3")), mini_meta=True)
>>> my_dabest_object.mean_diff.mini_meta_delta

Notes

As of version 2023.02.14, weighted delta can only be calculated for mean difference, and not for standardized measures such as Cohen’s d.

Details about the calculated weighted delta are accessed as attributes of the mini_meta_delta class. See the Tutorial: Mini-Meta Delta for details on usage.

Refer to Chapter 10 of the Cochrane handbook for further information on meta-analysis: https://training.cochrane.org/handbook/current/chapter-10

property bca_high

The bias-corrected and accelerated confidence interval upper limit.

property bca_low

The bias-corrected and accelerated confidence interval lower limit.

property bootstraps

Return the bootstrapped differences from all the experiment groups.

property difference

Return the weighted-average delta calculated from the raw data.

to_dict()

Returns all attributes of the dabest.MiniMetaDelta object as a dictionary.

class dabest._classes.DeltaDelta(effectsizedataframe, permutation_count, ci=95)

A class to compute and store the delta-delta statistics for experiments with a 2-by-2 arrangement where two independent variables, A and B, each have two categorical values, 1 and 2. The data is divided into two pairs of two groups, and a primary delta is first calculated as the mean difference between each of the pairs:

\[ \begin{align}\begin{aligned}\Delta_{1} = \overline{X}_{A_{2}, B_{1}} - \overline{X}_{A_{1}, B_{1}}\\\Delta_{2} = \overline{X}_{A_{2}, B_{2}} - \overline{X}_{A_{1}, B_{2}}\end{aligned}\end{align} \]

where \(\overline{X}_{A_{i}, B_{j}}\) is the mean of the sample with A = i and B = j, \(\Delta\) is the mean difference between two samples.

A delta-delta value is then calculated as the mean difference between the two primary deltas:

\[\Delta_{\Delta} = \Delta_{2} - \Delta_{1}\]

and:

and the standard deviation of the delta-delta value is calculated from a pooled variance of the 4 samples:

\[s_{\Delta_{\Delta}} = \sqrt{\frac{(n_{A_{2}, B_{1}}-1)s_{A_{2}, B_{1}}^2+(n_{A_{1}, B_{1}}-1)s_{A_{1}, B_{1}}^2+(n_{A_{2}, B_{2}}-1)s_{A_{2}, B_{2}}^2+(n_{A_{1}, B_{2}}-1)s_{A_{1}, B_{2}}^2}{(n_{A_{2}, B_{1}} - 1) + (n_{A_{1}, B_{1}} - 1) + (n_{A_{2}, B_{2}} - 1) + (n_{A_{1}, B_{2}} - 1)}}\]

where \(s\) is the standard deviation and \(n\) is the sample size.

Example

>>> import numpy as np
>>> import pandas as pd
>>> import dabest
>>> from scipy.stats import norm # Used in generation of populations.
>>> np.random.seed(9999) # Fix the seed so the results are replicable.
>>> from scipy.stats import norm # Used in generation of populations.
>>> N = 20
>>> # Create samples
>>> y = norm.rvs(loc=3, scale=0.4, size=N*4)
>>> y[N:2*N] = y[N:2*N]+1
>>> y[2*N:3*N] = y[2*N:3*N]-0.5
>>> # Add a `Treatment` column
>>> t1 = np.repeat('Placebo', N*2).tolist()
>>> t2 = np.repeat('Drug', N*2).tolist()
>>> treatment = t1 + t2
>>> # Add a `Rep` column as the first variable for the 2 replicates of experiments done
>>> rep = []
>>> for i in range(N*2):
>>>     rep.append('Rep1')
>>>     rep.append('Rep2')
>>> # Add a `Genotype` column as the second variable
>>> wt = np.repeat('W', N).tolist()
>>> mt = np.repeat('M', N).tolist()
>>> wt2 = np.repeat('W', N).tolist()
>>> mt2 = np.repeat('M', N).tolist()
>>> genotype = wt + mt + wt2 + mt2
>>> # Add an `id` column for paired data plotting.
>>> id = list(range(0, N*2))
>>> id_col = id + id
>>> # Combine all columns into a DataFrame.
>>> df_delta2 = pd.DataFrame({'ID'        : id_col,
>>>                   'Rep'      : rep,
>>>                    'Genotype'  : genotype,
>>>                    'Treatment': treatment,
>>>                    'Y'         : y
>>>                 })
>>> unpaired_delta2 = dabest.load(data = df_delta2, x = ["Genotype", "Genotype"], y = "Y", delta2 = True, experiment = "Treatment")
>>> unpaired_delta2.mean_diff.plot()
property bca_high

The bias-corrected and accelerated confidence interval upper limit.

property bca_low

The bias-corrected and accelerated confidence interval lower limit.

property bootstraps

Return the bootstrapped deltas from all the experiment groups.

property bootstraps_delta_delta

Return the delta-delta values calculated from the bootstrapped deltas.

property difference

Return the delta-delta value calculated based on the raw data.

to_dict()

Returns the attributes of the DeltaDelta object as a dictionary.

Plotting Data

class dabest._classes.EffectSizeDataFrame(dabest, effect_size, is_paired, ci=95, proportional=False, resamples=5000, permutation_count=5000, random_seed=12345, x1_level=None, x2=None, delta2=False, experiment_label=None, mini_meta=False)

Parses the data from a Dabest object, enabling plotting and printing capability for the effect size of interest.

plot(color_col=None, raw_marker_size=6, es_marker_size=9, swarm_label=None, barchart_label=None, contrast_label=None, delta2_label=None, swarm_ylim=None, barchart_ylim=None, contrast_ylim=None, delta2_ylim=None, custom_palette=None, swarm_desat=0.5, halfviolin_desat=1, halfviolin_alpha=0.8, face_color=None, bar_label=None, bar_desat=0.5, bar_width=0.5, bar_ylim=None, ci=None, ci_type='bca', err_color=None, float_contrast=True, show_pairs=True, show_delta2=True, show_mini_meta=True, group_summaries=None, group_summaries_offset=0.1, fig_size=None, dpi=100, ax=None, swarmplot_kwargs=None, barplot_kwargs=None, violinplot_kwargs=None, slopegraph_kwargs=None, sankey_kwargs=None, reflines_kwargs=None, group_summary_kwargs=None, legend_kwargs=None)

Creates an estimation plot for the effect size of interest.

Parameters:

  • color_col (string, default None) – Column to be used for colors.

  • raw_marker_size (float, default 6) – The diameter (in points) of the marker dots plotted in the swarmplot.

  • es_marker_size (float, default 9) – The size (in points) of the effect size points on the difference axes.

  • swarm_label (strings, default None) – Set labels for the y-axis of the swarmplot and the contrast plot, respectively. If swarm_label is not specified, it defaults to “value”, unless a column name was passed to y. If contrast_label is not specified, it defaults to the effect size being plotted. If delta2_label is not specifed, it defaults to “delta - delta”

  • contrast_label (strings, default None) – Set labels for the y-axis of the swarmplot and the contrast plot, respectively. If swarm_label is not specified, it defaults to “value”, unless a column name was passed to y. If contrast_label is not specified, it defaults to the effect size being plotted. If delta2_label is not specifed, it defaults to “delta - delta”

  • delta2_label (strings, default None) – Set labels for the y-axis of the swarmplot and the contrast plot, respectively. If swarm_label is not specified, it defaults to “value”, unless a column name was passed to y. If contrast_label is not specified, it defaults to the effect size being plotted. If delta2_label is not specifed, it defaults to “delta - delta”

  • swarm_ylim (tuples, default None) – The desired y-limits of the raw data (swarmplot) axes, the difference axes and the delta-delta axes respectively, as a tuple. These will be autoscaled to sensible values if they are not specified. The delta2 axes and contrast axes should have the same limits for y. When show_delta2 is True, if both of the contrast_ylim and delta2_ylim are not None, then they must be specified with the same values; when show_delta2 is True and only one of them is specified, then the other will automatically be assigned with the same value. Specifying delta2_ylim does not have any effect when show_delta2 is False.

  • contrast_ylim (tuples, default None) – The desired y-limits of the raw data (swarmplot) axes, the difference axes and the delta-delta axes respectively, as a tuple. These will be autoscaled to sensible values if they are not specified. The delta2 axes and contrast axes should have the same limits for y. When show_delta2 is True, if both of the contrast_ylim and delta2_ylim are not None, then they must be specified with the same values; when show_delta2 is True and only one of them is specified, then the other will automatically be assigned with the same value. Specifying delta2_ylim does not have any effect when show_delta2 is False.

  • delta2_ylim (tuples, default None) – The desired y-limits of the raw data (swarmplot) axes, the difference axes and the delta-delta axes respectively, as a tuple. These will be autoscaled to sensible values if they are not specified. The delta2 axes and contrast axes should have the same limits for y. When show_delta2 is True, if both of the contrast_ylim and delta2_ylim are not None, then they must be specified with the same values; when show_delta2 is True and only one of them is specified, then the other will automatically be assigned with the same value. Specifying delta2_ylim does not have any effect when show_delta2 is False.

  • custom_palette (dict, list, or matplotlib color palette, default None) – This keyword accepts a dictionary with {‘group’:’color’} pairings, a list of RGB colors, or a specified matplotlib palette. This palette will be used to color the swarmplot. If color_col is not specified, then each group will be colored in sequence according to the default palette currently used by matplotlib. Please take a look at the seaborn commands color_palette and cubehelix_palette to generate a custom palette. Both these functions generate a list of RGB colors. See: https://seaborn.pydata.org/generated/seaborn.color_palette.html https://seaborn.pydata.org/generated/seaborn.cubehelix_palette.html The named colors of matplotlib can be found here: https://matplotlib.org/examples/color/named_colors.html

  • swarm_desat (float, default 1) – Decreases the saturation of the colors in the swarmplot by the desired proportion. Uses seaborn.desaturate() to acheive this.

  • halfviolin_desat (float, default 0.5) – Decreases the saturation of the colors of the half-violin bootstrap curves by the desired proportion. Uses seaborn.desaturate() to acheive this.

  • halfviolin_alpha (float, default 0.8) – The alpha (transparency) level of the half-violin bootstrap curves.

  • float_contrast (boolean, default True) – Whether or not to display the halfviolin bootstrapped difference distribution alongside the raw data.

  • show_pairs (boolean, default True) – If the data is paired, whether or not to show the raw data as a swarmplot, or as slopegraph, with a line joining each pair of observations.

  • show_delta2 (boolean, default True) – If delta-delta or mini-meta delta is calculated, whether or not to show the delta-delta plot or mini-meta plot.

  • show_mini_meta (boolean, default True) – If delta-delta or mini-meta delta is calculated, whether or not to show the delta-delta plot or mini-meta plot.

  • group_summaries (['mean_sd', 'median_quartiles', 'None'], default None.) – Plots the summary statistics for each group. If ‘mean_sd’, then the mean and standard deviation of each group is plotted as a notched line beside each group. If ‘median_quantiles’, then the median and 25th and 75th percentiles of each group is plotted instead. If ‘None’, the summaries are not shown.

  • group_summaries_offset (float, default 0.1) – If group summaries are displayed, they will be offset from the raw data swarmplot groups by this value.

  • fig_size (tuple, default None) – The desired dimensions of the figure as a (length, width) tuple.

  • dpi (int, default 100) – The dots per inch of the resulting figure.

  • ax (matplotlib.Axes, default None) – Provide an existing Axes for the plots to be created. If no Axes is specified, a new matplotlib Figure will be created.

  • swarmplot_kwargs (dict, default None) – Pass any keyword arguments accepted by the seaborn swarmplot command here, as a dict. If None, the following keywords are passed to sns.swarmplot : {‘size’:raw_marker_size}.

  • violinplot_kwargs (dict, default None) – Pass any keyword arguments accepted by the matplotlib ` pyplot.violinplot` command here, as a dict. If None, the following keywords are passed to violinplot : {‘widths’:0.5, ‘vert’:True, ‘showextrema’:False, ‘showmedians’:False}.

  • slopegraph_kwargs (dict, default None) – This will change the appearance of the lines used to join each pair of observations when show_pairs=True. Pass any keyword arguments accepted by matplotlib plot() function here, as a dict. If None, the following keywords are passed to plot() : {‘linewidth’:1, ‘alpha’:0.5}.

  • sankey_kwargs (dict, default None) – Whis will change the appearance of the sankey diagram used to depict paired proportional data when show_pairs=True and proportional=True. Pass any keyword arguments accepted by plot_tools.sankeydiag() function here, as a dict. If None, the following keywords are passed to sankey diagram: {“width”: 0.5, “align”: “center”, “alpha”: 0.4, “bar_width”: 0.1, “rightColor”: False}

  • reflines_kwargs (dict, default None) – This will change the appearance of the zero reference lines. Pass any keyword arguments accepted by the matplotlib Axes hlines command here, as a dict. If None, the following keywords are passed to Axes.hlines : {‘linestyle’:’solid’, ‘linewidth’:0.75, ‘zorder’:2, ‘color’ : default y-tick color}.

  • group_summary_kwargs (dict, default None) – Pass any keyword arguments accepted by the matplotlib.lines.Line2D command here, as a dict. This will change the appearance of the vertical summary lines for each group, if group_summaries is not ‘None’. If None, the following keywords are passed to matplotlib.lines.Line2D : {‘lw’:2, ‘alpha’:1, ‘zorder’:3}.

  • legend_kwargs (dict, default None) – Pass any keyword arguments accepted by the matplotlib Axes legend command here, as a dict. If None, the following keywords are passed to matplotlib.Axes.legend : {‘loc’:’upper left’, ‘frameon’:False}.

Returns:

  • A matplotlib.figure.Figure with 2 Axes, if ax = None.

  • The first axes (accessible with FigName.axes[0]) contains the rawdata swarmplot; the second axes (accessible with FigName.axes[1]) has the bootstrap distributions and effect sizes (with confidence intervals) plotted on it.

  • If ax is specified, the rawdata swarmplot is accessed at ax

  • itself, while the effect size axes is accessed at ax.contrast_axes.

  • See the last example below.

Examples

Create a Gardner-Altman estimation plot for the mean difference.

>>> my_data = dabest.load(df, idx=("Control 1", "Test 1"))
>>> fig1 = my_data.mean_diff.plot()

Create a Gardner-Altman plot for the Hedges’ g effect size.

>>> fig2 = my_data.hedges_g.plot()

Create a Cumming estimation plot for the mean difference.

>>> fig3 = my_data.mean_diff.plot(float_contrast=True)

Create a paired Gardner-Altman plot.

>>> my_data_paired = dabest.load(df, idx=("Control 1", "Test 1"),
...                id_col = "ID", paired='baseline')
>>> fig4 = my_data_paired.mean_diff.plot()

Create a multi-group Cumming plot.

>>> my_multi_groups = dabest.load(df, id_col = "ID",
...                             idx=(("Control 1", "Test 1"),
...                                 ("Control 2", "Test 2")))
>>> fig5 = my_multi_groups.mean_diff.plot()

Create a shared control Cumming plot.

>>> my_shared_control = dabest.load(df, id_col = "ID",
...                                 idx=("Control 1", "Test 1",
...                                          "Test 2", "Test 3"))
>>> fig6 = my_shared_control.mean_diff.plot()

Create a repeated meausures (against baseline) Slopeplot.

>>> my_rm_baseline = dabest.load(df, id_col = "ID", paired = "baseline",
...                                 idx=("Control 1", "Test 1",
...                                          "Test 2", "Test 3"))
>>> fig7 = my_rm_baseline.mean_diff.plot()

Create a repeated meausures (sequential) Slopeplot.

>>> my_rm_sequential = dabest.load(df, id_col = "ID", paired = "sequential",
...                                 idx=("Control 1", "Test 1",
...                                          "Test 2", "Test 3"))
>>> fig8 = my_rm_sequential.mean_diff.plot()

Creating estimation plots in individual panels of a figure.

>>> f, axx = plt.subplots(nrows=2, ncols=2, figsize=(15, 15))
>>> my_data.mean_diff.plot(ax=axx.flat[0])
>>> my_data_paired.mean_diff.plot(ax=axx.flat[1])
>>> my_shared_control.mean_diff.plot(ax=axx.flat[2])
>>> my_shared_control.mean_diff.plot(ax=axx.flat[3], float_contrast=False)
property lqrt

Returns all pairwise Lq-Likelihood Ratio Type test results as a pandas DataFrame.

For more information on LqRT tests, see https://arxiv.org/abs/1911.11922

Permutation Tests

class dabest._classes.PermutationTest(control, test, effect_size, is_paired, permutation_count=5000, random_seed=12345, **kwargs)

A class to compute and report permutation tests.

Parameters:

  • control (array-like) –

  • test (array-like) – These should be numerical iterables.

  • effect_size (string.) – Any one of the following are accepted inputs: ‘mean_diff’, ‘median_diff’, ‘cohens_d’, ‘hedges_g’, or ‘cliffs_delta’

  • is_paired (string, default None) –

  • permutation_count (int, default 10000) – The number of permutations (reshuffles) to perform.

  • random_seed (int, default 12345) – random_seed is used to seed the random number generator during bootstrap resampling. This ensures that the generated permutations are replicable.

Returns:

  • A PermutationTest object.

  • difference (float) – The effect size of the difference between the control and the test.

  • effect_size (string) – The type of effect size reported.

Notes

The basic concept of permutation tests is the same as that behind bootstrapping. In an “exact” permutation test, all possible resuffles of the control and test labels are performed, and the proportion of effect sizes that equal or exceed the observed effect size is computed. This is the probability, under the null hypothesis of zero difference between test and control groups, of observing the effect size: the p-value of the Student’s t-test.

Exact permutation tests are impractical: computing the effect sizes for all reshuffles quickly exceeds trivial computational loads. A control group and a test group both with 10 observations each would have a total of \(20!\) or \(2.43 \times {10}^{18}\) reshuffles. Therefore, in practice, “approximate” permutation tests are performed, where a sufficient number of reshuffles are performed (5,000 or 10,000), from which the p-value is computed.

More information can be found here.

Example

>>> import numpy as np
>>> from scipy.stats import norm
>>> import dabest
>>> control = norm.rvs(loc=0, size=30, random_state=12345)
>>> test = norm.rvs(loc=0.5, size=30, random_state=12345)
>>> perm_test = dabest.PermutationTest(control, test,
...                                    effect_size="mean_diff",
...                                    paired=None)
>>> perm_test
5000 permutations were taken. The pvalue is 0.0758.