Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions nbs/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
/.quarto/
**/*.quarto_ipynb
15 changes: 12 additions & 3 deletions nbs/blog/posts/a-dabest2-preprint/a-dabest2-preprint.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,12 @@
"id": "3aaa3ea8",
"metadata": {},
"source": [
"# Preprint: Getting over ANOVA\n",
"---\n",
"date: \"2026-02-25\"\n",
"title: 'Preprint: Getting over ANOVA'\n",
"subtitle: 'A description of the updated DABEST packages.'\n",
"\n",
"- order: 3"
"---"
]
},
{
Expand Down Expand Up @@ -43,7 +46,13 @@
]
}
],
"metadata": {},
"metadata": {
"kernelspec": {
"display_name": "python3",
"language": "python",
"name": "python3"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
11 changes: 6 additions & 5 deletions nbs/blog/posts/bootstraps/bootstraps.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -5,11 +5,12 @@
"id": "1a3ec507",
"metadata": {},
"source": [
"# Bootstrap Confidence Intervals\n",
"\n",
"> Explanation of the bootstrap method and its application in hypothesis testing using **DABEST**.\n",
"\n",
"- order: 3"
"---\n",
"date: \"2023-03-26\"\n",
"title: 'Bootstrap Confidence Intervals'\n",
"subtitle: 'Explanation of the bootstrap method and its application in hypothesis testing using **DABEST**.'\n",
"---\n",
"\n"
]
},
{
Expand Down
201 changes: 201 additions & 0 deletions nbs/blog/posts/dabesties-over-the-years/dabesties-over-the-years.ipynb
Original file line number Diff line number Diff line change
@@ -0,0 +1,201 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "b1f0c2a4",
"metadata": {},
"source": [
"---\n",
"date: \"2026-08-03\"\n",
"title: 'DABESTies over the years'\n",
"subtitle: 'Who are our DABESTies? 7 years later, here’s what we found.'\n",
"---"
]
},
{
"cell_type": "markdown",
"id": "70e5798f",
"metadata": {},
"source": [
"<p style=\"color:#6c757d; font-style:normal; font-weight:400;\">Written by: <a href=\"https://github.com/mnicolee\" style=\"color:#6c757d; text-decoration:underline;\">Nicole Lee</a></p>"
]
},
{
"cell_type": "markdown",
"id": "c3d1e5a7",
"metadata": {},
"source": [
"In 1978, Kenneth Rothman argued in the *New England Journal of Medicine* [\\[1\\]](#1) that confidence intervals should replace significance tests. He spent years trying to make that happen. As an editor at the *American Journal of Public Health*, he required authors to report confidence intervals instead of p-values, reducing sole reliance on p-values from 63% to 5% [\\[2\\]](#2). When he founded [*Epidemiology*](https://www.ovid.com/jnls/epidem) in 1990, he went further and banned p-values altogether. On paper, it looked like a success. But when Fidler and colleagues [\\[2\\]](#2) examined those papers, they found that while authors had dutifully reported confidence intervals, very few actually used them to interpret their results. The policy was eventually abandoned."
]
},
{
"cell_type": "markdown",
"id": "e5f3a7c9",
"metadata": {},
"source": [
"That history was very much on our minds. We wanted to make it easier for people to think about their data by giving them a practical way to visualize estimation statistics. That led us to publish *Moving beyond P values: data analysis with estimation graphics* [\\[3\\]](#3) in *Nature Methods*, alongside DABEST [\\[4\\]](#4), the software that generates those figures."
]
},
{
"cell_type": "markdown",
"id": "978aff1c",
"metadata": {},
"source": [
"Our paper also became part of a broader conversation about how estimation statistics should be reported. Just six weeks after our paper was published, *eNeuro* launched an initiative encouraging authors to adopt estimation statistics, alongside a perspective by Calin-Jageman and Cumming [\\[5\\]](#5) explaining the rationale. In early 2020, Editor-in-Chief Christophe Bernard asked Reviewing Editors \\[[6](#6), [7](#7)\\] to identify papers that could benefit from converting to estimation statistics, and 100 were flagged. Across the journal, 52 papers published that year included estimation statistics. By any reasonable measure, the initiative worked, but what caught our attention was how *eNeuro* framed that success. The follow-up editorial [\\[7\\]](#7) presents estimation as something authors can layer on top of conventional significance testing rather than in place of it, and Bernard goes as far as saying that in some cases the p-value stops being necessary at all. "
]
},
{
"cell_type": "markdown",
"id": "f6a4b8d1",
"metadata": {},
"source": [
"Seven years after we released DABEST, we wanted to know who our DABESTies are, and what they actually did with it. We pulled every work citing the 2019 paper from OpenAlex [\\[8\\]](#8). After removing papers from our own lab, preprints and duplicate records, we were left with 1,266 citations from 2018 through 2025, with another 89 so far in 2026 **(Figure 1)**. DABEST has now been cited across 660 journals, 25 research fields, and ~638 institutions."
]
},
{
"cell_type": "markdown",
"id": "b8c6d1f3",
"metadata": {},
"source": [
"![](figure-1.svg)"
]
},
{
"cell_type": "markdown",
"id": "c9d7e2a4",
"metadata": {},
"source": [
"<p style=\"color:#6c757d;\"><b>Figure 1.</b> Works citing DABEST per year excluding our lab’s work, preprints, and duplicate records.</p>"
]
},
{
"cell_type": "markdown",
"id": "d1e8f3b5",
"metadata": {},
"source": [
"The two biggest fields are medicine and neuroscience, with 322 and 317 citing papers respectively, followed by biochemistry and molecular biology (217), environmental science (114) and agriculture (113) **(Figure 2)**. Psychology, engineering, computer science and the social sciences all appear further down the list."
]
},
{
"cell_type": "markdown",
"id": "e2f9a4c6",
"metadata": {},
"source": [
"That is a much wider user base than we could ever have surveyed directly, and it is the reason we went to the papers themselves instead. If we wanted to know what people were doing with DABEST, the only reliable place to look was at what they had published."
]
},
{
"cell_type": "markdown",
"id": "f3a1b5d7",
"metadata": {},
"source": [
"![](figure-2.svg)"
]
},
{
"cell_type": "markdown",
"id": "a4b2c6e8",
"metadata": {},
"source": [
"<p style=\"color:#6c757d;\"><b>Figure 2.</b> The field breakdown of the 1,355 works citing DABEST, after removing our own lab's papers, preprints and duplicate records.</p>"
]
},
{
"cell_type": "markdown",
"id": "b5c3d7f9",
"metadata": {},
"source": [
"So we read the 100 most-cited research papers that cited our 2019 paper, excluding reviews, editorials, tutorials and software papers. For each paper, we recorded where the estimation plot appeared and where p-values were reported **(Figure 3)**. Estimation plots reached the main figures in 74 papers, appeared only in the supplement in 19, and were absent in 7. Nearly all 100 of them reported p-values as well."
]
},
{
"cell_type": "markdown",
"id": "c6d4e8a1",
"metadata": {},
"source": [
"![](figure-3.svg)"
]
},
{
"cell_type": "markdown",
"id": "d7e5f9b2",
"metadata": {},
"source": [
"<p style=\"color:#6c757d;\"><b>Figure 3.</b> How the 100 scored papers were selected and sorted. The counts in the top two boxes come from the earlier OpenAlex pull, before preprints and duplicates were removed. The six categories at the bottom were built to capture how each paper used DABEST, by recording whether the estimation plot reached a main figure or stayed in the supplement, and how far the p-values travelled alongside it.</p>"
]
},
{
"cell_type": "markdown",
"id": "e8f6a1c3",
"metadata": {},
"source": [
"The more revealing picture emerges when the two answers are placed side by side. The single most common arrangement, covering 68 of the 100 papers, was an estimation plot in a main figure with p-values reported in the main text right alongside it. Only two papers combined a main-figure estimation plot with no p-values anywhere at all **(Figure 4)**."
]
},
{
"cell_type": "markdown",
"id": "f9a7b2d4",
"metadata": {},
"source": [
"![](figure-4.svg)"
]
},
{
"cell_type": "markdown",
"id": "a1b8c3e5",
"metadata": {},
"source": [
"<p style=\"color:#6c757d;\"><b>Figure 4.</b> Where the estimation plot and the p-values landed across the 100 most-cited research articles. The three colored groups are the categories referred to in Figure 3.</p>"
]
},
{
"cell_type": "markdown",
"id": "f6a4b8d2",
"metadata": {},
"source": [
"Today, estimation plots have clearly found a place in the literature, presenting effect sizes alongside the raw data, making both the magnitude of an effect and the underlying observation immediately visible. Most authors still reported p-values alongside them, while only a handful (2%) relied on estimation statistics alone **(Figure 4)**. We cannot say from this data whether statistical thinking has changed more broadly, only that for many researchers the two approaches now coexist."
]
},
{
"cell_type": "markdown",
"id": "a7b5c9e3",
"metadata": {},
"source": [
"## References"
]
},
{
"cell_type": "markdown",
"id": "b8c6d1f4",
"metadata": {},
"source": [
"<a id='1'></a>\n",
"`[1]`: Rothman, Kenneth J. [“A Show of Confidence.”](https://doi.org/10.1056/NEJM197812142992410) *The New England Journal of Medicine*, vol. 299, no. 24, 1978, pp. 1362–63.\n",
"\n",
"<a id='2'></a>\n",
"`[2]`: Fidler, Fiona, et al. [“Editors Can Lead Researchers to Confidence Intervals, but Can’t Make Them Think: Statistical Reform Lessons from Medicine.”](https://doi.org/10.1111/j.0963-7214.2004.01502008.x) *Psychological Science*, vol. 15, no. 2, 2004, pp. 119–26.\n",
"\n",
"<a id='3'></a>\n",
"`[3]`: Ho, Joses, et al. [“Moving beyond P Values: Data Analysis with Estimation Graphics.”](https://doi.org/10.1038/s41592-019-0470-3) *Nature Methods*, vol. 16, no. 7, 2019, pp. 565–66.\n",
"\n",
"<a id='4'></a>\n",
"`[4]`: ACCLAB. [*DABEST-Python: Data Analysis with Bootstrapped ESTimation*](https://github.com/ACCLAB/DABEST-python). *GitHub*. Accessed 3 Aug. 2026.\n",
"\n",
"<a id='5'></a>\n",
"`[5]`: Calin-Jageman, Robert J., and Geoff Cumming. [“Estimation for Better Inference in Neuroscience.”](https://doi.org/10.1523/ENEURO.0205-19.2019) *eNeuro*, vol. 6, no. 4, 2019, article ENEURO.0205-19.2019.\n",
"\n",
"<a id='6'></a>\n",
"`[6]`: Bernard, Christophe. [“Changing the Way We Report, Interpret, and Discuss Our Results to Rebuild Trust in Our Research.”](https://doi.org/10.1523/ENEURO.0259-19.2019) *eNeuro*, vol. 6, no. 4, 2019, article ENEURO.0259-19.2019.\n",
"\n",
"<a id='7'></a>\n",
"`[7]`: Bernard, Christophe. [“Estimation Statistics, One Year Later.”](https://doi.org/10.1523/ENEURO.0091-21.2021) *eNeuro*, vol. 8, no. 2, 2021, article ENEURO.0091-21.2021.\n",
"\n",
"<a id='8'></a>\n",
"`[8]`: [“Moving beyond P Values: Data Analysis with Estimation Graphics.”](https://openalex.org/W2952837230) *OpenAlex*, OurResearch. Accessed 3 Aug. 2026."
]
}
],
"metadata": {},
"nbformat": 4,
"nbformat_minor": 5
}
Loading
Loading