diff --git a/nbs/.gitignore b/nbs/.gitignore new file mode 100644 index 00000000..ad293093 --- /dev/null +++ b/nbs/.gitignore @@ -0,0 +1,2 @@ +/.quarto/ +**/*.quarto_ipynb diff --git a/nbs/blog/posts/a-dabest2-preprint/a-dabest2-preprint.ipynb b/nbs/blog/posts/a-dabest2-preprint/a-dabest2-preprint.ipynb index 3d13476b..86ecadc9 100644 --- a/nbs/blog/posts/a-dabest2-preprint/a-dabest2-preprint.ipynb +++ b/nbs/blog/posts/a-dabest2-preprint/a-dabest2-preprint.ipynb @@ -5,9 +5,12 @@ "id": "3aaa3ea8", "metadata": {}, "source": [ - "# Preprint: Getting over ANOVA\n", + "---\n", + "date: \"2026-02-25\"\n", + "title: 'Preprint: Getting over ANOVA'\n", + "subtitle: 'A description of the updated DABEST packages.'\n", "\n", - "- order: 3" + "---" ] }, { @@ -43,7 +46,13 @@ ] } ], - "metadata": {}, + "metadata": { + "kernelspec": { + "display_name": "python3", + "language": "python", + "name": "python3" + } + }, "nbformat": 4, "nbformat_minor": 5 } diff --git a/nbs/blog/posts/bootstraps/bootstraps.ipynb b/nbs/blog/posts/bootstraps/bootstraps.ipynb index 8a5f73c9..23689c38 100644 --- a/nbs/blog/posts/bootstraps/bootstraps.ipynb +++ b/nbs/blog/posts/bootstraps/bootstraps.ipynb @@ -5,11 +5,12 @@ "id": "1a3ec507", "metadata": {}, "source": [ - "# Bootstrap Confidence Intervals\n", - "\n", - "> Explanation of the bootstrap method and its application in hypothesis testing using **DABEST**.\n", - "\n", - "- order: 3" + "---\n", + "date: \"2023-03-26\"\n", + "title: 'Bootstrap Confidence Intervals'\n", + "subtitle: 'Explanation of the bootstrap method and its application in hypothesis testing using **DABEST**.'\n", + "---\n", + "\n" ] }, { diff --git a/nbs/blog/posts/dabesties-over-the-years/dabesties-over-the-years.ipynb b/nbs/blog/posts/dabesties-over-the-years/dabesties-over-the-years.ipynb new file mode 100644 index 00000000..608a2d1a --- /dev/null +++ b/nbs/blog/posts/dabesties-over-the-years/dabesties-over-the-years.ipynb @@ -0,0 +1,201 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "b1f0c2a4", + "metadata": {}, + "source": [ + "---\n", + "date: \"2026-08-03\"\n", + "title: 'DABESTies over the years'\n", + "subtitle: 'Who are our DABESTies? 7 years later, here’s what we found.'\n", + "---" + ] + }, + { + "cell_type": "markdown", + "id": "70e5798f", + "metadata": {}, + "source": [ + "

Written by: Nicole Lee

" + ] + }, + { + "cell_type": "markdown", + "id": "c3d1e5a7", + "metadata": {}, + "source": [ + "In 1978, Kenneth Rothman argued in the *New England Journal of Medicine* [\\[1\\]](#1) that confidence intervals should replace significance tests. He spent years trying to make that happen. As an editor at the *American Journal of Public Health*, he required authors to report confidence intervals instead of p-values, reducing sole reliance on p-values from 63% to 5% [\\[2\\]](#2). When he founded [*Epidemiology*](https://www.ovid.com/jnls/epidem) in 1990, he went further and banned p-values altogether. On paper, it looked like a success. But when Fidler and colleagues [\\[2\\]](#2) examined those papers, they found that while authors had dutifully reported confidence intervals, very few actually used them to interpret their results. The policy was eventually abandoned." + ] + }, + { + "cell_type": "markdown", + "id": "e5f3a7c9", + "metadata": {}, + "source": [ + "That history was very much on our minds. We wanted to make it easier for people to think about their data by giving them a practical way to visualize estimation statistics. That led us to publish *Moving beyond P values: data analysis with estimation graphics* [\\[3\\]](#3) in *Nature Methods*, alongside DABEST [\\[4\\]](#4), the software that generates those figures." + ] + }, + { + "cell_type": "markdown", + "id": "978aff1c", + "metadata": {}, + "source": [ + "Our paper also became part of a broader conversation about how estimation statistics should be reported. Just six weeks after our paper was published, *eNeuro* launched an initiative encouraging authors to adopt estimation statistics, alongside a perspective by Calin-Jageman and Cumming [\\[5\\]](#5) explaining the rationale. In early 2020, Editor-in-Chief Christophe Bernard asked Reviewing Editors \\[[6](#6), [7](#7)\\] to identify papers that could benefit from converting to estimation statistics, and 100 were flagged. Across the journal, 52 papers published that year included estimation statistics. By any reasonable measure, the initiative worked, but what caught our attention was how *eNeuro* framed that success. The follow-up editorial [\\[7\\]](#7) presents estimation as something authors can layer on top of conventional significance testing rather than in place of it, and Bernard goes as far as saying that in some cases the p-value stops being necessary at all. " + ] + }, + { + "cell_type": "markdown", + "id": "f6a4b8d1", + "metadata": {}, + "source": [ + "Seven years after we released DABEST, we wanted to know who our DABESTies are, and what they actually did with it. We pulled every work citing the 2019 paper from OpenAlex [\\[8\\]](#8). After removing papers from our own lab, preprints and duplicate records, we were left with 1,266 citations from 2018 through 2025, with another 89 so far in 2026 **(Figure 1)**. DABEST has now been cited across 660 journals, 25 research fields, and ~638 institutions." + ] + }, + { + "cell_type": "markdown", + "id": "b8c6d1f3", + "metadata": {}, + "source": [ + "![](figure-1.svg)" + ] + }, + { + "cell_type": "markdown", + "id": "c9d7e2a4", + "metadata": {}, + "source": [ + "

Figure 1. Works citing DABEST per year excluding our lab’s work, preprints, and duplicate records.

" + ] + }, + { + "cell_type": "markdown", + "id": "d1e8f3b5", + "metadata": {}, + "source": [ + "The two biggest fields are medicine and neuroscience, with 322 and 317 citing papers respectively, followed by biochemistry and molecular biology (217), environmental science (114) and agriculture (113) **(Figure 2)**. Psychology, engineering, computer science and the social sciences all appear further down the list." + ] + }, + { + "cell_type": "markdown", + "id": "e2f9a4c6", + "metadata": {}, + "source": [ + "That is a much wider user base than we could ever have surveyed directly, and it is the reason we went to the papers themselves instead. If we wanted to know what people were doing with DABEST, the only reliable place to look was at what they had published." + ] + }, + { + "cell_type": "markdown", + "id": "f3a1b5d7", + "metadata": {}, + "source": [ + "![](figure-2.svg)" + ] + }, + { + "cell_type": "markdown", + "id": "a4b2c6e8", + "metadata": {}, + "source": [ + "

Figure 2. The field breakdown of the 1,355 works citing DABEST, after removing our own lab's papers, preprints and duplicate records.

" + ] + }, + { + "cell_type": "markdown", + "id": "b5c3d7f9", + "metadata": {}, + "source": [ + "So we read the 100 most-cited research papers that cited our 2019 paper, excluding reviews, editorials, tutorials and software papers. For each paper, we recorded where the estimation plot appeared and where p-values were reported **(Figure 3)**. Estimation plots reached the main figures in 74 papers, appeared only in the supplement in 19, and were absent in 7. Nearly all 100 of them reported p-values as well." + ] + }, + { + "cell_type": "markdown", + "id": "c6d4e8a1", + "metadata": {}, + "source": [ + "![](figure-3.svg)" + ] + }, + { + "cell_type": "markdown", + "id": "d7e5f9b2", + "metadata": {}, + "source": [ + "

Figure 3. How the 100 scored papers were selected and sorted. The counts in the top two boxes come from the earlier OpenAlex pull, before preprints and duplicates were removed. The six categories at the bottom were built to capture how each paper used DABEST, by recording whether the estimation plot reached a main figure or stayed in the supplement, and how far the p-values travelled alongside it.

" + ] + }, + { + "cell_type": "markdown", + "id": "e8f6a1c3", + "metadata": {}, + "source": [ + "The more revealing picture emerges when the two answers are placed side by side. The single most common arrangement, covering 68 of the 100 papers, was an estimation plot in a main figure with p-values reported in the main text right alongside it. Only two papers combined a main-figure estimation plot with no p-values anywhere at all **(Figure 4)**." + ] + }, + { + "cell_type": "markdown", + "id": "f9a7b2d4", + "metadata": {}, + "source": [ + "![](figure-4.svg)" + ] + }, + { + "cell_type": "markdown", + "id": "a1b8c3e5", + "metadata": {}, + "source": [ + "

Figure 4. Where the estimation plot and the p-values landed across the 100 most-cited research articles. The three colored groups are the categories referred to in Figure 3.

" + ] + }, + { + "cell_type": "markdown", + "id": "f6a4b8d2", + "metadata": {}, + "source": [ + "Today, estimation plots have clearly found a place in the literature, presenting effect sizes alongside the raw data, making both the magnitude of an effect and the underlying observation immediately visible. Most authors still reported p-values alongside them, while only a handful (2%) relied on estimation statistics alone **(Figure 4)**. We cannot say from this data whether statistical thinking has changed more broadly, only that for many researchers the two approaches now coexist." + ] + }, + { + "cell_type": "markdown", + "id": "a7b5c9e3", + "metadata": {}, + "source": [ + "## References" + ] + }, + { + "cell_type": "markdown", + "id": "b8c6d1f4", + "metadata": {}, + "source": [ + "\n", + "`[1]`: Rothman, Kenneth J. [“A Show of Confidence.”](https://doi.org/10.1056/NEJM197812142992410) *The New England Journal of Medicine*, vol. 299, no. 24, 1978, pp. 1362–63.\n", + "\n", + "\n", + "`[2]`: Fidler, Fiona, et al. [“Editors Can Lead Researchers to Confidence Intervals, but Can’t Make Them Think: Statistical Reform Lessons from Medicine.”](https://doi.org/10.1111/j.0963-7214.2004.01502008.x) *Psychological Science*, vol. 15, no. 2, 2004, pp. 119–26.\n", + "\n", + "\n", + "`[3]`: Ho, Joses, et al. [“Moving beyond P Values: Data Analysis with Estimation Graphics.”](https://doi.org/10.1038/s41592-019-0470-3) *Nature Methods*, vol. 16, no. 7, 2019, pp. 565–66.\n", + "\n", + "\n", + "`[4]`: ACCLAB. [*DABEST-Python: Data Analysis with Bootstrapped ESTimation*](https://github.com/ACCLAB/DABEST-python). *GitHub*. Accessed 3 Aug. 2026.\n", + "\n", + "\n", + "`[5]`: Calin-Jageman, Robert J., and Geoff Cumming. [“Estimation for Better Inference in Neuroscience.”](https://doi.org/10.1523/ENEURO.0205-19.2019) *eNeuro*, vol. 6, no. 4, 2019, article ENEURO.0205-19.2019.\n", + "\n", + "\n", + "`[6]`: Bernard, Christophe. [“Changing the Way We Report, Interpret, and Discuss Our Results to Rebuild Trust in Our Research.”](https://doi.org/10.1523/ENEURO.0259-19.2019) *eNeuro*, vol. 6, no. 4, 2019, article ENEURO.0259-19.2019.\n", + "\n", + "\n", + "`[7]`: Bernard, Christophe. [“Estimation Statistics, One Year Later.”](https://doi.org/10.1523/ENEURO.0091-21.2021) *eNeuro*, vol. 8, no. 2, 2021, article ENEURO.0091-21.2021.\n", + "\n", + "\n", + "`[8]`: [“Moving beyond P Values: Data Analysis with Estimation Graphics.”](https://openalex.org/W2952837230) *OpenAlex*, OurResearch. Accessed 3 Aug. 2026." + ] + } + ], + "metadata": {}, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/nbs/blog/posts/dabesties-over-the-years/figure-1.svg b/nbs/blog/posts/dabesties-over-the-years/figure-1.svg new file mode 100644 index 00000000..93760aa4 --- /dev/null +++ b/nbs/blog/posts/dabesties-over-the-years/figure-1.svg @@ -0,0 +1,336 @@ + + + + + + + + 2026-07-29T01:57:12.318102 + image/svg+xml + + + Matplotlib v3.11.0, https://matplotlib.org/ + + + + + + + + + + + + + + + + + + + + + + + + + + + + 2018 + + + + + + + + + + 2019 + + + + + + + + + + 2020 + + + + + + + + + + 2021 + + + + + + + + + + 2022 + + + + + + + + + + 2023 + + + + + + + + + + 2024 + + + + + + + + + + 2025 + + + + + + + + + + 2026 + + + + Publication year + + + + + + + + + + + + + + 0 + + + + + + + + + + 50 + + + + + + + + + + 100 + + + + + + + + + + 150 + + + + + + + + + + 200 + + + + + + + + + + 250 + + + + + + + + + + 300 + + + + Published research works citing DABEST + + + + + + + + + + + + + + + + 89 + (partial, + to Jul 2026) + + + 1 + + + 51 + + + 190 + + + 271 + + + 196 + + + 184 + + + 187 + + + 186 + + + + + + bioRxiv preprint, Jul 2018 + Nature Methods, Jun 2019 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/nbs/blog/posts/dabesties-over-the-years/figure-2.svg b/nbs/blog/posts/dabesties-over-the-years/figure-2.svg new file mode 100644 index 00000000..dd8401e2 --- /dev/null +++ b/nbs/blog/posts/dabesties-over-the-years/figure-2.svg @@ -0,0 +1,176 @@ + + + + + + + + 2026-07-30T01:28:21.839982 + image/svg+xml + + + Matplotlib v3.10.6, https://matplotlib.org/ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + Medicine + + + 322 (23.8%) + + + + + Neuroscience + + + 317 (23.4%) + + + + + Biochemistry, + Genetics and + Molecular + Biology + + + 217 (16.0%) + + + + + Environmental + Science + + + 114 (8.4%) + + + + + Agriculture & + Bio. Sciences + + + 113 (8.3%) + + + + + + + Psychology + + + 51 (3.8%) + + + + Other (19 + fields) + + + 221 (16.3%) + + + + 1,355 + + + citing works + + + + diff --git a/nbs/blog/posts/dabesties-over-the-years/figure-3.svg b/nbs/blog/posts/dabesties-over-the-years/figure-3.svg new file mode 100644 index 00000000..8d54d76d --- /dev/null +++ b/nbs/blog/posts/dabesties-over-the-years/figure-3.svg @@ -0,0 +1,337 @@ + + + + + + + + 2026-07-29T01:57:23.117955 + image/svg+xml + + + Matplotlib v3.11.0, https://matplotlib.org/ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + Retrieve citing works + + + 1,874 records, OpenAlex + + + Exclude own-lab papers + + + 1,855 external + + + Rank by citations; keep research articles + + + top 100 retained; reviews, editorials and tool papers excluded + + + Score figures by hand + + + placement and p-value reach + + + Estimation plot in main figure + + + no p-values + anywhere + + + p-values only in + supplement + + + p-values also in + main article + + + Estimation plot in supplement only + + + p-values in main + article + + + p-values in + supplement or + none + + + No estimation plot + anywhere + + + p-values reported + + + + + + + + + diff --git a/nbs/blog/posts/dabesties-over-the-years/figure-4.svg b/nbs/blog/posts/dabesties-over-the-years/figure-4.svg new file mode 100644 index 00000000..5a76a8c1 --- /dev/null +++ b/nbs/blog/posts/dabesties-over-the-years/figure-4.svg @@ -0,0 +1,278 @@ + + + + + + + + 2026-07-29T01:57:28.175071 + image/svg+xml + + + Matplotlib v3.11.0, https://matplotlib.org/ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + 0 + + + + + + + + + + 10 + + + + + + + + + + 20 + + + + + + + + + + 30 + + + + + + + + + + 40 + + + + + + + + + + 50 + + + + + + + + + + 60 + + + + + + + + + + 70 + + + + + + + + + + 80 + + + + % of the 100 most-cited research articles citing DABEST + + + + + + + + + + + + + + + + + + + + + + + + + + 2% (n=2) + + + 4% (n=4) + + + 68% (n=68) + + + 16% (n=16) + + + 3% (n=3) + + + 7% (n=7) + + + no p-values anywhere + + + p-values only in supplement + + + p-values also in main article + + + Estimation plot in main figure + + + p-values in main article + + + p-values in supplement or none + + + Estimation plot in supplement only + + + p-values reported + + + No estimation plot anywhere + + + + + + + + + diff --git a/nbs/blog/posts/nmeth-announce/nmeth-announce.ipynb b/nbs/blog/posts/nmeth-announce/nmeth-announce.ipynb new file mode 100644 index 00000000..fcba3b6b --- /dev/null +++ b/nbs/blog/posts/nmeth-announce/nmeth-announce.ipynb @@ -0,0 +1,69 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "3c916f98", + "metadata": {}, + "source": [ + "---\n", + "date: \"2026-08-04\"\n", + "title: 'Nature Methods: Getting over ANOVA'\n", + "---" + ] + }, + { + "cell_type": "markdown", + "id": "31c0d657", + "metadata": {}, + "source": [ + "" + ] + }, + { + "cell_type": "markdown", + "id": "1637313a", + "metadata": {}, + "source": [ + "‘Getting over ANOVA’ is the title of our paper on multi-group data, out today in Nature Methods.\n", + "\n", + "\n", + "\n", + "The break-up is overdue. ANOVA asks whether all the groups are the same—a question nobody wants answered—and then sends you off to a pile of post-hoc tests nobody wants either. So what replaces it? We need methods that answer what we actually want to know: which groups differ, in which direction, and by how much. And they should not just report it, but also show it.\n", + "\n", + "The new paper describes a software package, DABEST 2.0, that brings estimation graphics to multi-group data: repeated measures, two-factor interactions via delta-delta effects, binary outcomes, and internal replicates via mini-meta. Some graphics can directly replace an ANOVA method. Each graphic shows the raw data, the effect size, and the uncertainty. \n", + "\n", + "Building software to visualize multi-group effect sizes has been a collaborative effort by the DABEST team, and I'm proud of what we built. DABEST is open source and available in Python, R, and through a web app. Data analysis should be easy to practice, and give you direct answers to the questions your experiments were designed to ask.\n", + "\n", + "Shout out to the team: Zinan Lu, Jonathan Anns, Yishan Mai, ROU ZHANG, CFA, Kahseng Lian, Nicole Lee, Shan Hashir, Zhuoyu Wang, Yixuan Li, A. Rosa Castillo, Joses Ho, Hyungwon Choi, Sangyu Xu and Adam Claridge-Chang.\n", + "\n", + "Also posted on [LinkedIn](https://www.linkedin.com/posts/adam-claridge-chang-9a00819_dabest-estimationstatistics-datavisualization-share-7490052099582373888-nvys/?utm_source=share&utm_medium=member_desktop&rcm=ACoAAGAPWI0BNuczQdmuHJWNaXg6kmJqSUTR-gI).\n", + "\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "id": "efdc1c76", + "metadata": {}, + "source": [ + "![](preprint_fig.png)" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "python3", + "language": "python", + "name": "python3" + }, + "widgets": { + "application/vnd.jupyter.widget-state+json": { + "state": {}, + "version_major": 2, + "version_minor": 0 + } + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/nbs/blog/posts/nmeth-announce/sankey.jpg b/nbs/blog/posts/nmeth-announce/sankey.jpg new file mode 100644 index 00000000..77b3912a Binary files /dev/null and b/nbs/blog/posts/nmeth-announce/sankey.jpg differ diff --git a/nbs/blog/posts/robust-beautiful/robust-beautiful.ipynb b/nbs/blog/posts/robust-beautiful/robust-beautiful.ipynb index 6672337e..cc634af3 100644 --- a/nbs/blog/posts/robust-beautiful/robust-beautiful.ipynb +++ b/nbs/blog/posts/robust-beautiful/robust-beautiful.ipynb @@ -5,9 +5,10 @@ "id": "5b2f1637", "metadata": {}, "source": [ - "# Robust and Beautiful Statistical Visualization\n", - "\n", - "- order: 3" + "---\n", + "date: \"2024-01-11\"\n", + "title: 'Robust and Beautiful Statistical Visualization'\n", + "---" ] }, {