.. _Tutorial: ======== Tutorial ======== Load Libraries -------------- .. code:: ipython3 >>> import numpy as np >>> import pandas as pd >>> import dabest >>> print("We're using DABEST v{}".format(dabest.__version__)) .. parsed-literal:: We're using DABEST v0.2.4 Create dataset for demo ----------------------- Here, we create a dataset to illustrate how :keyword:`dabest` functions. In this dataset, each column corresponds to a group of observations. .. code:: ipython3 >>> from scipy.stats import norm # Used in generation of populations. >>> np.random.seed(9999) # Fix the seed so the results are replicable. >>> Ns = 20 # The number of samples taken from each population >>> # Create samples >>> c1 = norm.rvs(loc=3, scale=0.4, size=Ns) >>> c2 = norm.rvs(loc=3.5, scale=0.75, size=Ns) >>> c3 = norm.rvs(loc=3.25, scale=0.4, size=Ns) >>> t1 = norm.rvs(loc=3.5, scale=0.5, size=Ns) >>> t2 = norm.rvs(loc=2.5, scale=0.6, size=Ns) >>> t3 = norm.rvs(loc=3, scale=0.75, size=Ns) >>> t4 = norm.rvs(loc=3.5, scale=0.75, size=Ns) >>> t5 = norm.rvs(loc=3.25, scale=0.4, size=Ns) >>> t6 = norm.rvs(loc=3.25, scale=0.4, size=Ns) >>> # Add a `gender` column for coloring the data. >>> females = np.repeat('Female', Ns/2).tolist() >>> males = np.repeat('Male', Ns/2).tolist() >>> gender = females + males >>> # Add an `id` column for paired data plotting. >>> id_col = pd.Series(range(1, Ns+1)) >>> # Combine samples and gender into a DataFrame. >>> df = pd.DataFrame({'Control 1' : c1, 'Test 1' : t1, ... 'Control 2' : c2, 'Test 2' : t2, ... 'Control 3' : c3, 'Test 3' : t3, ... 'Test 4' : t4, 'Test 5' : t5, ... 'Test 6' : t6, ... 'Gender' : gender, 'ID' : id_col ... }) Note that we have 9 samples (3 Control samples and 6 Test samples). Our dataset also has a non-numerical column indicating gender, and another column indicating the identity of each observation. This is known as a 'wide' dataset. See this `writeup `__ for more details. .. code:: ipython3 df.head() # Displays the first 5 rows of our dataset. .. raw:: html
Control 1 Test 1 Control 2 Test 2 Control 3 Test 3 Test 4 Test 5 Test 6 Gender ID
2.793984 3.420875 3.324661 1.707467 3.816940 1.796581 4.440050 2.937284 3.486127 Female 1
3.236759 3.467972 3.685186 1.121846 3.750358 3.944566 3.723494 2.837062 2.338094 Female 2
3.019149 4.377179 5.616891 3.301381 2.945397 2.832188 3.214014 3.111950 3.270897 Female 3
2.804638 4.564780 2.773152 2.534018 3.575179 3.048267 4.968278 3.743378 3.151188 Female 4
2.858019 3.220058 2.550361 2.796365 3.692138 3.276575 2.662104 2.977341 2.328601 Female 5
Loading Data ------------ Before we create estimation plots and obtain confidence intervals for our effect sizes, we need to load the data and the relevant groups. We simply supply the DataFrame to :keyword:`dabest.load()`. We also must supply the two groups you want to compare in the :keyword:`idx` argument as a tuple or list. .. code:: ipython3 >>> two_groups_unpaired = dabest.load(df, idx=("Control 1", "Test 1")) Calling this `Dabest` object gives you a gentle greeting, as well as the comparisons that can be computed. .. code:: ipython3 >>> two_groups_unpaired .. parsed-literal:: DABEST v0.2.1 ============= Good afternoon! The current time is Mon Mar 11 16:19:24 2019. Effect size(s) with 95% confidence intervals will be computed for: 1. Test 1 minus Control 1 5000 resamples will be used to generate the effect size bootstraps. Changing statistical parameters ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ If the dataset contains paired data (ie. repeated observations), specify this with the :keyword:`paired` keyword. You must also pass a column in the dataset that indicates the identity of each observation, using the :keyword:`id_col` keyword. .. code:: ipython3 >>> two_groups_paired = dabest.load(df, idx=("Control 1", "Test 1"), ... paired=True, id_col="ID") .. code:: ipython3 >>> two_groups_paired .. parsed-literal:: DABEST v0.2.1 ============= Good afternoon! The current time is Mon Mar 11 16:19:25 2019. Paired effect size(s) with 95% confidence intervals will be computed for: 1. Test 1 minus Control 1 5000 resamples will be used to generate the effect size bootstraps. You can also change the width of the confidence interval that will be produced. .. code:: ipython3 >>> two_groups_unpaired_ci90 = dabest.load(df, ... idx=("Control 1", "Test 1"), ... ci=90) .. code:: ipython3 >>> two_groups_unpaired_ci90 .. parsed-literal:: DABEST v0.2.1 ============= Good afternoon! The current time is Mon Mar 11 16:19:25 2019. Effect size(s) with 90% confidence intervals will be computed for: 1. Test 1 minus Control 1 5000 resamples will be used to generate the effect size bootstraps. Effect sizes ------------ :keyword:`dabest` now features a range of effect sizes: - the mean difference (:keyword:`mean_diff`) - the median difference (:keyword:`median_diff`) - `Cohen's *d* `__ (:keyword:`cohens_d`) - `Hedges' *g* `__ (:keyword:`hedges_g`) - `Cliff's delta `__ (:keyword:`cliffs_delta`) Each of these are attributes of the `Dabest` object. .. code:: ipython3 >>> two_groups_unpaired.mean_diff .. parsed-literal:: DABEST v0.2.1 ============= Good afternoon! The current time is Mon Mar 11 16:19:25 2019. The unpaired mean difference between Control 1 and Test 1 is 0.48 [95%CI 0.205, 0.774]. The two-sided p-value of the Mann-Whitney test is 0.000813. 5000 bootstrap samples were taken; the confidence interval is bias-corrected and accelerated. The p-value(s) reported are the likelihood(s) of observing the effect size(s), if the null hypothesis of zero difference is true. To get the results of all valid statistical tests, use `.mean_diff.statistical_tests` For each comparison, the type of effect size is reported (here, it's the "unpaired mean difference"). The confidence interval is reported as: [*confidenceIntervalWidth* *LowerBound*, *UpperBound*], and is generated through bootstrap resampling. See :ref:`Bootstrap Confidence Intervals` for more details. By default, DABEST will report the two-sided p-value of the most conservative test that is appropriate for the effect size. This is the statistical test that does not assume normality of the underlying populations, and does not assume that both of them do not share the same variance (ie. heteroscadacity). You can access the results as a ``pandas DataFrame``. .. code:: ipython3 >>> two_groups_unpaired.mean_diff.results .. raw:: html
control test effect_size is_paired difference ci bca_low bca_high bca_interval_idx pct_low pct_high pct_interval_idx bootstraps resamples random_seed pvalue_welch statistic_welch pvalue_students_t statistic_students_t pvalue_mann_whitney statistic_mann_whitney
0 Control 1 Test 1 mean difference False 0.48029 95 0.205161 0.773647 (145, 4893) 0.197427 0.758752 (125, 4875) [-0.05989473868674011, -0.018608309424335, 0.0... 5000 12345 0.002094 -3.308806 0.002057 -3.308806 0.000813 83.0
You can use ``.mean_diff.statistical_tests`` to obtain the p-values and test statistics for all relavant statistical tests. .. code:: ipython3 >>> two_groups_unpaired.mean_diff.statistical_tests .. raw:: html
control test effect_size is_paired difference ci bca_low bca_high pvalue_welch statistic_welch pvalue_students_t statistic_students_t pvalue_mann_whitney statistic_mann_whitney
0 Control 1 Test 1 mean difference False 0.48029 95 0.205161 0.773647 0.002094 -3.308806 0.002057 -3.308806 0.000813 83.0
Let's compute the Hedges' g for our comparison. .. code:: ipython3 >>> two_groups_unpaired.hedges_g .. parsed-literal:: DABEST v0.2.1 ============= Good afternoon! The current time is Mon Mar 11 16:19:26 2019. The unpaired Hedges' g between Control 1 and Test 1 is 1.03 [95%CI 0.317, 1.62]. The two-sided p-value of the Mann-Whitney test is 0.000813. 5000 bootstrap samples were taken; the confidence interval is bias-corrected and accelerated. The p-value(s) reported are the likelihood(s) of observing the effect size(s), if the null hypothesis of zero difference is true. To get the results of all valid statistical tests, use `.hedges_g.statistical_tests` .. code:: ipython3 >>> two_groups_unpaired.hedges_g.results .. raw:: html
control test effect_size is_paired difference ci bca_low bca_high bca_interval_idx pct_low ... pct_interval_idx bootstraps resamples random_seed pvalue_welch statistic_welch pvalue_students_t statistic_students_t pvalue_mann_whitney statistic_mann_whitney
0 Control 1 Test 1 Hedges' g False 1.025525 95 0.316506 1.616235 (42, 4725) 0.44486 ... (125, 4875) [-0.1491709040527835, -0.0504066101302326, 0.0... 5000 12345 0.002094 -3.308806 0.002057 -3.308806 0.000813 83.0

1 rows × 21 columns