Dabest object

Main class for estimating statistics and generating plots.

Dabest

 Dabest (data, idx, x, y, paired, id_col, ci, resamples, random_seed,
         proportional, delta2, experiment, experiment_label, x1_level,
         mini_meta)

Class for estimation statistics and plots.

Example: mean_diff

control = norm.rvs(loc=0, size=30, random_state=12345)
test    = norm.rvs(loc=0.5, size=30, random_state=12345)
my_df   = pd.DataFrame({"control": control,
                            "test": test})
my_dabest_object = dabest.load(my_df, idx=("control", "test"))
my_dabest_object.mean_diff
DABEST v2024.03.29
==================
                  
Good afternoon!
The current time is Tue Mar 19 15:33:25 2024.

The unpaired mean difference between control and test is 0.5 [95%CI -0.0412, 1.0].
The p-value of the two-sided permutation t-test is 0.0758, calculated for legacy purposes only. 

5000 bootstrap samples were taken; the confidence interval is bias-corrected and accelerated.
Any p-value reported is the probability of observing theeffect size (or greater),
assuming the null hypothesis of zero difference is true.
For each p-value, 5000 reshuffles of the control and test labels were performed.

To get the results of all valid statistical tests, use `.mean_diff.statistical_tests`

This is simply the mean of the control group subtracted from the mean of the test group.

\[\text{Mean difference} = \overline{x}_{Test} - \overline{x}_{Control}\]

where \(\overline{x}\) is the mean for the group \(x\).

Example: median_diff

control = norm.rvs(loc=0, size=30, random_state=12345)
test    = norm.rvs(loc=0.5, size=30, random_state=12345)
my_df   = pd.DataFrame({"control": control,
                            "test": test})
my_dabest_object = dabest.load(my_df, idx=("control", "test"))
my_dabest_object.median_diff
DABEST v2024.03.29
==================
                  
Good afternoon!
The current time is Tue Mar 19 15:33:26 2024.

The unpaired median difference between control and test is 0.5 [95%CI -0.0758, 0.991].
The p-value of the two-sided permutation t-test is 0.103, calculated for legacy purposes only. 

5000 bootstrap samples were taken; the confidence interval is bias-corrected and accelerated.
Any p-value reported is the probability of observing theeffect size (or greater),
assuming the null hypothesis of zero difference is true.
For each p-value, 5000 reshuffles of the control and test labels were performed.

To get the results of all valid statistical tests, use `.median_diff.statistical_tests`

This is the median difference between the control group and the test group.

If the comparison(s) are unpaired, median_diff is computed with the following equation:

\[\text{Median difference} = \widetilde{x}_{Test} - \widetilde{x}_{Control}\]

where \(\widetilde{x}\) is the median for the group \(x\).

If the comparison(s) are paired, median_diff is computed with the following equation:

\[\text{Median difference} = \widetilde{x}_{Test - Control}\]

Things to note

Using median difference as the statistic in bootstrapping may result in a biased estimate and cause problems with BCa confidence intervals. Consider using mean difference instead.

When plotting, consider using percentile confidence intervals instead of BCa confidence intervals by specifying ci_type = 'percentile' in .plot().

For detailed information, please refer to Issue 129.

Example: cohens_d

control = norm.rvs(loc=0, size=30, random_state=12345)
test    = norm.rvs(loc=0.5, size=30, random_state=12345)
my_df   = pd.DataFrame({"control": control,
                            "test": test})
my_dabest_object = dabest.load(my_df, idx=("control", "test"))
my_dabest_object.cohens_d
DABEST v2024.03.29
==================
                  
Good afternoon!
The current time is Tue Mar 19 15:33:27 2024.

The unpaired Cohen's d between control and test is 0.471 [95%CI -0.0843, 0.976].
The p-value of the two-sided permutation t-test is 0.0758, calculated for legacy purposes only. 

5000 bootstrap samples were taken; the confidence interval is bias-corrected and accelerated.
Any p-value reported is the probability of observing theeffect size (or greater),
assuming the null hypothesis of zero difference is true.
For each p-value, 5000 reshuffles of the control and test labels were performed.

To get the results of all valid statistical tests, use `.cohens_d.statistical_tests`

Cohen’s d is simply the mean of the control group subtracted from the mean of the test group.

If paired is None, then the comparison(s) are unpaired; otherwise the comparison(s) are paired.

If the comparison(s) are unpaired, Cohen’s d is computed with the following equation:

\[d = \frac{\overline{x}_{Test} - \overline{x}_{Control}} {\text{pooled standard deviation}}\]

For paired comparisons, Cohen’s d is given by

\[d = \frac{\overline{x}_{Test} - \overline{x}_{Control}} {\text{average standard deviation}}\]

where \(\overline{x}\) is the mean of the respective group of observations, \({Var}_{x}\) denotes the variance of that group,

\[\text{pooled standard deviation} = \sqrt{ \frac{(n_{control} - 1) * {Var}_{control} + (n_{test} - 1) * {Var}_{test} } {n_{control} + n_{test} - 2} }\]

and

\[\text{average standard deviation} = \sqrt{ \frac{{Var}_{control} + {Var}_{test}} {2}}\]

The sample variance (and standard deviation) uses N-1 degrees of freedoms. This is an application of Bessel’s correction, and yields the unbiased sample variance.

References:

https://en.wikipedia.org/wiki/Effect_size#Cohen's_d

https://en.wikipedia.org/wiki/Bessel%27s_correction

https://en.wikipedia.org/wiki/Standard_deviation#Corrected_sample_standard_deviation

Example: cohens_h

control = randint.rvs(0, 2, size=30, random_state=12345)
test    = randint.rvs(0, 2, size=30, random_state=12345)
my_df   = pd.DataFrame({"control": control,
                            "test": test})
my_dabest_object = dabest.load(my_df, idx=("control", "test"))
my_dabest_object.cohens_h
DABEST v2024.03.29
==================
                  
Good afternoon!
The current time is Tue Mar 19 15:33:29 2024.

The unpaired Cohen's h between control and test is 0.0 [95%CI -0.613, 0.429].
The p-value of the two-sided permutation t-test is 0.799, calculated for legacy purposes only. 

5000 bootstrap samples were taken; the confidence interval is bias-corrected and accelerated.
Any p-value reported is the probability of observing theeffect size (or greater),
assuming the null hypothesis of zero difference is true.
For each p-value, 5000 reshuffles of the control and test labels were performed.

To get the results of all valid statistical tests, use `.cohens_h.statistical_tests`

Cohen’s h uses the information of proportion in the control and test groups to calculate the distance between two proportions.

It can be used to describe the difference between two proportions as “small”, “medium”, or “large”.

It can be used to determine if the difference between two proportions is “meaningful”.

A directional Cohen’s h is computed with the following equation:

\[h = 2 * \arcsin{\sqrt{proportion_{Test}}} - 2 * \arcsin{\sqrt{proportion_{Control}}}\]

For a non-directional Cohen’s h, the equation is:

\[h = |2 * \arcsin{\sqrt{proportion_{Test}}} - 2 * \arcsin{\sqrt{proportion_{Control}}}|\]

References:

https://en.wikipedia.org/wiki/Cohen%27s_h

Example: hedges_g

control = norm.rvs(loc=0, size=30, random_state=12345)
test    = norm.rvs(loc=0.5, size=30, random_state=12345)
my_df   = pd.DataFrame({"control": control,
                            "test": test})
my_dabest_object = dabest.load(my_df, idx=("control", "test"))
my_dabest_object.hedges_g
DABEST v2024.03.29
==================
                  
Good afternoon!
The current time is Tue Mar 19 15:33:30 2024.

The unpaired Hedges' g between control and test is 0.465 [95%CI -0.0832, 0.963].
The p-value of the two-sided permutation t-test is 0.0758, calculated for legacy purposes only. 

5000 bootstrap samples were taken; the confidence interval is bias-corrected and accelerated.
Any p-value reported is the probability of observing theeffect size (or greater),
assuming the null hypothesis of zero difference is true.
For each p-value, 5000 reshuffles of the control and test labels were performed.

To get the results of all valid statistical tests, use `.hedges_g.statistical_tests`

Hedges’ g is cohens_d corrected for bias via multiplication with the following correction factor:

\[\frac{ \Gamma( \frac{a} {2} )} {\sqrt{ \frac{a} {2} } \times \Gamma( \frac{a - 1} {2} )}\]

where

\[a = {n}_{control} + {n}_{test} - 2\]

and \(\Gamma(x)\) is the Gamma function.

References:

https://en.wikipedia.org/wiki/Effect_size#Hedges'_g

https://journals.sagepub.com/doi/10.3102/10769986006002107

Example: cliffs_delta

control = norm.rvs(loc=0, size=30, random_state=12345)
test    = norm.rvs(loc=0.5, size=30, random_state=12345)
my_df   = pd.DataFrame({"control": control,
                            "test": test})
my_dabest_object = dabest.load(my_df, idx=("control", "test"))
my_dabest_object.cliffs_delta
DABEST v2024.03.29
==================
                  
Good afternoon!
The current time is Tue Mar 19 15:33:41 2024.

The unpaired Cliff's delta between control and test is 0.28 [95%CI -0.0244, 0.533].
The p-value of the two-sided permutation t-test is 0.061, calculated for legacy purposes only. 

5000 bootstrap samples were taken; the confidence interval is bias-corrected and accelerated.
Any p-value reported is the probability of observing theeffect size (or greater),
assuming the null hypothesis of zero difference is true.
For each p-value, 5000 reshuffles of the control and test labels were performed.

To get the results of all valid statistical tests, use `.cliffs_delta.statistical_tests`

Cliff’s delta is a measure of ordinal dominance, ie. how often the values from the test sample are larger than values from the control sample.

\[\text{Cliff's delta} = \frac{\#({x}_{test} > {x}_{control}) - \#({x}_{test} < {x}_{control})} {{n}_{Test} \times {n}_{Control}}\]

where \(\#\) denotes the number of times a value from the test sample exceeds (or is lesser than) values in the control sample.

Cliff’s delta ranges from -1 to 1; it can also be thought of as a measure of the degree of overlap between the two samples. An attractive aspect of this effect size is that it does not make an assumptions about the underlying distributions that the samples were drawn from.

References:

https://en.wikipedia.org/wiki/Effect_size#Effect_size_for_ordinal_data

https://psycnet.apa.org/record/1994-08169-001

Example: delta_g

random.seed(12345) # Fix the seed so the results are replicable.
N=20
y = norm.rvs(loc=3, scale=0.4, size=N*4)
y[N:2*N] = y[N:2*N]+1
y[2*N:3*N] = y[2*N:3*N]-0.5
t1 = repeat('Placebo', N*2).tolist()
t2 = repeat('Drug', N*2).tolist()
treatment = t1 + t2
rep = []
for i in range(N*2):
    rep.append('Rep1')
    rep.append('Rep2')
wt = repeat('W', N).tolist()
mt = repeat('M', N).tolist()
wt2 = repeat('W', N).tolist()
mt2 = repeat('M', N).tolist()
genotype = wt + mt + wt2 + mt2
id = list(range(0, N*2))
id_col = id + id
df_delta2 = pd.DataFrame({'ID'        : id_col,
                          'Rep'      : rep,
                          'Genotype'  : genotype,
                          'Treatment': treatment,
                          'Y'         : y})
unpaired_delta2 = dabest.load(data = df_delta2, x = ["Genotype", "Genotype"], y = "Y", delta2 = True, experiment = "Treatment")
unpaired_delta2.delta_g
DABEST v2024.03.29
==================
                  
Good afternoon!
The current time is Tue Mar 19 15:33:45 2024.

The unpaired deltas' g between W Placebo and M Placebo is 1.74 [95%CI 1.1, 2.31].
The p-value of the two-sided permutation t-test is 0.0, calculated for legacy purposes only. 

The unpaired deltas' g between W Drug and M Drug is 1.33 [95%CI 0.611, 1.96].
The p-value of the two-sided permutation t-test is 0.0, calculated for legacy purposes only. 

The deltas' g between Placebo and Drug is -0.651 [95%CI -1.59, 0.165].
The p-value of the two-sided permutation t-test is 0.0694, calculated for legacy purposes only. 

5000 bootstrap samples were taken; the confidence interval is bias-corrected and accelerated.
Any p-value reported is the probability of observing the effect size (or greater),
assuming the null hypothesis of zero difference is true.
For each p-value, 5000 reshuffles of the control and test labels were performed.

To get the results of all valid statistical tests, use `.delta_g.statistical_tests`

Deltas’ g is an effect size that only applied on experiments with a 2-by-2 arrangement where two independent variables, A and B, each have two categorical values, 1 and 2, which calculates hedges_g for delta-delta statistics.

\[\Delta_{1} = \overline{X}_{A_{2}, B_{1}} - \overline{X}_{A_{1}, B_{1}}\]

\[\Delta_{2} = \overline{X}_{A_{2}, B_{2}} - \overline{X}_{A_{1}, B_{2}}\]

where \(\overline{X}_{A_{i}, B_{j}}\) is the mean of the sample with A = i and B = j, \(\Delta\) is the mean difference between two samples.

A delta-delta value is then calculated as the mean difference between the two primary deltas:

\[\Delta_{\Delta} = \Delta_{2} - \Delta_{1}\]

and the standard deviation of the delta-delta value is calculated from a pooled variance of the 4 samples:

\[s_{\Delta_{\Delta}} = \sqrt{\frac{(n_{A_{2}, B_{1}}-1)s_{A_{2}, B_{1}}^2+(n_{A_{1}, B_{1}}-1)s_{A_{1}, B_{1}}^2+(n_{A_{2}, B_{2}}-1)s_{A_{2}, B_{2}}^2+(n_{A_{1}, B_{2}}-1)s_{A_{1}, B_{2}}^2}{(n_{A_{2}, B_{1}} - 1) + (n_{A_{1}, B_{1}} - 1) + (n_{A_{2}, B_{2}} - 1) + (n_{A_{1}, B_{2}} - 1)}}\]

where \(s\) is the standard deviation and \(n\) is the sample size.

A deltas’ g value is then calculated as delta-delta value divided by pooled standard deviation \(s_{\Delta_{\Delta}}\):

\(\Delta_{g} = \frac{\Delta_{\Delta}}{s_{\Delta_{\Delta}}}\)