Skip to content

Updates to citations, and minor fixes to blog post #228

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
mnicolee merged 1 commit into from
Aug 5, 2026
Merged

Updates to citations, and minor fixes to blog post #228

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 6 additions & 6 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -132,15 +132,15 @@ Please refer to the official [tutorial](https://acclab.github.io/DABEST-python/)

## How to cite

**Getting over ANOVA: Estimation graphics for multi-group comparisons**
**Getting over ANOVA: estimation graphics for multi-group comparisons**

*Zinan Lu, Jonathan Anns, Yishan Mai, Rou Zhang, Kahseng Lian, Nicole MynYi Lee, Shan Hashir, Lucas Wang Zhuoyu, A. Rosa Castillo Gonzalez, Joses Ho, Hyungwon Choi, Sangyu Xu, Adam Claridge-Chang*
*Zinan Lu, Jonathan Anns, Yishan Mai, Rou Zhang, Kahseng Lian, Nicole MynYi Lee, Shan Hashir, Lucas Zhuoyu Wang, Yixuan Li, A. Rosa Castillo Gonzalez, Joses Ho, Hyungwon Choi, Sangyu Xu, Adam Claridge-Chang*

bioRxiv preprint 2026. [10.64898/2026.01.26.701654](http://dx.doi.org/10.64898/2026.01.26.701654)
Nature Methods 2026, 1548-7105. [10.1038/s41592-026-03187-7](https://doi.org/10.1038/s41592-026-03187-7)

[PDF](https://www.biorxiv.org/content/10.64898/2026.01.26.701654v1.full.pdf)
[Paywalled publisher site](https://www.nature.com/articles/s41592-026-03187-7)

**Moving beyond P values: Everyday data analysis with estimation plots**
**Moving beyond P values: data analysis with estimation graphics**

*Joses Ho, Tayfun Tumkaya, Sameer Aryal, Hyungwon Choi, Adam Claridge-Chang*

Expand All @@ -164,7 +164,7 @@ If you have any specific comments and ideas for new features that you would like

## Acknowledgements

We would like to thank alpha testers from the [Claridge-Chang lab](https://www.claridgechang.net/): [Sangyu Xu](https://github.com/sangyu), [Xianyuan Zhang](https://github.com/XYZfar), [Farhan Mohammad](https://github.com/farhan8igib), Jurga Mituzaitė, and Stanislav Ott.
We would like to thank alpha testers from the [Claridge-Chang lab](https://www.claridgechang.net/): [Sangyu Xu](https://github.com/sangyu), [Xianyuan Zhang](https://github.com/XYZfar), [Farhan Mohammad](https://github.com/farhan8igib), Jurga Mituzaitė, Stanislav Ott, [Tayfun Tumkaya](https://github.com/ttumkaya), [Jonathan Anns](https://github.com/JAnns98), [Nicole Lee](https://github.com/mnicolee) and [Yishan Mai](https://github.com/maiyishan).

## Testing

Expand Down
12 changes: 10 additions & 2 deletions nbs/03-citation.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,9 +15,17 @@
"source": [
"\n",
"\n",
"If your publication features a graphic generated with this software library, please cite the following publication.\n",
"If your publication features a graphic generated with this software library, please cite the following publications.\n",
"\n",
"**Moving beyond P values: Everyday data analysis with estimation plots**\n",
"**Getting over ANOVA: estimation graphics for multi-group comparisons**\n",
"Zinan Lu, Jonathan Anns, Yishan Mai, Rou Zhang, Kahseng Lian, Nicole MynYi Lee, Shan Hashir, Lucas Zhuoyu Wang, Yixuan Li, A. Rosa Castillo Gonzalez, Joses Ho, Hyungwon Choi, Sangyu Xu, Adam Claridge-Chang\n",
"\n",
"`Nature Methods` 2026, 1548-7105. [doi:10.1038/s41592-026-03187-7](https://doi.org/10.1038/s41592-026-03187-7)\n",
"\n",
"[Paywalled publisher site](https://www.nature.com/articles/s41592-026-03187-7)\n",
"\n",
"\n",
"**Moving beyond P values: data analysis with estimation graphics**\n",
"Joses Ho, Tayfun Tumkaya, Sameer Aryal, Hyungwon Choi, Adam Claridge-Chang\n",
"\n",
"`Nature Methods` 2019, 1548-7105. [doi:10.1038/s41592-019-0470-3](https://doi.org/10.1038/s41592-019-0470-3)\n",
Expand Down
49 changes: 21 additions & 28 deletions nbs/blog/posts/dabesties-over-the-years/dabesties-over-the-years.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,9 @@
"source": [
"---\n",
"date: \"2026-08-03\"\n",
"title: 'DABESTies over the years'\n",
"subtitle: 'Who are our DABESTies? 7 years later, here’s what we found.'\n",
"title: 'Tracking DABEST adoption'\n",
"subtitle: 'Seven years after DABEST was introduced in 2019, we looked back at its adoption across the literature.'\n",
"image: figure-2.svg\n",
"---"
]
},
Expand All @@ -25,47 +26,47 @@
"id": "c3d1e5a7",
"metadata": {},
"source": [
"In 1978, Kenneth Rothman argued in the *New England Journal of Medicine* [\\[1\\]](#1) that confidence intervals should replace significance tests. He spent years trying to make that happen. As an editor at the *American Journal of Public Health*, he required authors to report confidence intervals instead of p-values, reducing sole reliance on p-values from 63% to 5% [\\[2\\]](#2). When he founded [*Epidemiology*](https://www.ovid.com/jnls/epidem) in 1990, he went further and banned p-values altogether. On paper, it looked like a success. But when Fidler and colleagues [\\[2\\]](#2) examined those papers, they found that while authors had dutifully reported confidence intervals, very few actually used them to interpret their results. The policy was eventually abandoned."
"Knowing whether a result is statistically significant is not the same as understanding what the data shows. In 1978, Kenneth Rothman argued in the *New England Journal of Medicine* [\\[1\\]](#1) that confidence intervals should replace significance tests. He spent years trying to make that happen. As an editor at the *American Journal of Public Health*, he required authors to report confidence intervals instead of p-values, reducing sole reliance on p-values from 63% to 5% [\\[2\\]](#2). When he founded [*Epidemiology*](https://www.ovid.com/jnls/epidem) in 1990, he went further and banned p-values altogether. On paper, it looked like a success. But when Fidler and colleagues [\\[2\\]](#2) examined those papers, they found that while authors had dutifully reported confidence intervals, very few actually used them to interpret their results. The policy was eventually abandoned."
]
},
{
"cell_type": "markdown",
"id": "e5f3a7c9",
"metadata": {},
"source": [
"That history was very much on our minds. We wanted to make it easier for people to think about their data by giving them a practical way to visualize estimation statistics. That led us to publish *Moving beyond P values: data analysis with estimation graphics* [\\[3\\]](#3) in *Nature Methods*, alongside DABEST [\\[4\\]](#4), the software that generates those figures."
"That history was very much on our minds when we first published *Moving beyond P values: data analysis with estimation graphics* [\\[3\\]](#3) in *Nature Methods*, alongside DABEST [\\[4\\]](#4), the software that generates those figures. We wanted to make it easier for people to think about their data by giving them a practical way to visualize estimation statistics."
]
},
{
"cell_type": "markdown",
"id": "978aff1c",
"metadata": {},
"source": [
"Our paper also became part of a broader conversation about how estimation statistics should be reported. Just six weeks after our paper was published, *eNeuro* launched an initiative encouraging authors to adopt estimation statistics, alongside a perspective by Calin-Jageman and Cumming [\\[5\\]](#5) explaining the rationale. In early 2020, Editor-in-Chief Christophe Bernard asked Reviewing Editors \\[[6](#6), [7](#7)\\] to identify papers that could benefit from converting to estimation statistics, and 100 were flagged. Across the journal, 52 papers published that year included estimation statistics. By any reasonable measure, the initiative worked, but what caught our attention was how *eNeuro* framed that success. The follow-up editorial [\\[7\\]](#7) presents estimation as something authors can layer on top of conventional significance testing rather than in place of it, and Bernard goes as far as saying that in some cases the p-value stops being necessary at all. "
"Our paper also became part of a broader conversation about how estimation statistics should be reported. Just six weeks after our paper was published, *eNeuro* launched an initiative encouraging authors to adopt estimation statistics as something to layer on top of existing practice rather than replace it [\\[5\\]](#5), alongside a perspective by Calin-Jageman and Cumming [\\[6\\]](#6) explaining the rationale. In early 2020, Editor-in-Chief Christophe Bernard asked Reviewing Editors [\\[7\\]](#7) to identify papers that could benefit from converting to estimation statistics, and 100 were flagged. Across the journal, 52 papers published that year included estimation statistics. The initiative clearly succeeded in increasing the use of estimation statistics, although Bernard noted that in some cases p-values may no longer be necessary."
]
},
{
"cell_type": "markdown",
"id": "f6a4b8d1",
"metadata": {},
"source": [
"Seven years after we released DABEST, we wanted to know who our DABESTies are, and what they actually did with it. We pulled every work citing the 2019 paper from OpenAlex [\\[8\\]](#8). After removing papers from our own lab, preprints and duplicate records, we were left with 1,266 citations from 2018 through 2025, with another 89 so far in 2026 **(Figure 1)**. DABEST has now been cited across 660 journals, 25 research fields, and ~638 institutions."
"Seven years after we released DABEST, we wanted to know if anything had changed. What did the people who picked it up actually do with it? We pulled every work citing the 2019 paper from OpenAlex [\\[8\\]](#8). After removing papers from our own lab, preprints and duplicate records, we were left with 1,266 citations from 2018 through 2025, with another 89 so far in 2026 **(Figure 1)**. DABEST has now been cited across 660 journals, 25 research fields, and ~638 institutions."
]
},
{
"cell_type": "markdown",
"id": "b8c6d1f3",
"metadata": {},
"source": [
"![](figure-1.svg)"
"![](figure-1.svg){fig-align=\"center\" width=\"100%\"}"
]
},
{
"cell_type": "markdown",
"id": "c9d7e2a4",
"metadata": {},
"source": [
"<p style=\"color:#6c757d;\"><b>Figure 1.</b> Works citing DABEST per year excluding our lab’s work, preprints, and duplicate records.</p>"
"<p style=\"color:#6c757d; font-size:calc(1em - 2px); line-height:1.2;\"><b>Figure 1.</b> Works citing DABEST per year excluding our lab’s work, preprints, and duplicate records.</p>"
]
},
{
Expand All @@ -76,76 +77,68 @@
"The two biggest fields are medicine and neuroscience, with 322 and 317 citing papers respectively, followed by biochemistry and molecular biology (217), environmental science (114) and agriculture (113) **(Figure 2)**. Psychology, engineering, computer science and the social sciences all appear further down the list."
]
},
{
"cell_type": "markdown",
"id": "e2f9a4c6",
"metadata": {},
"source": [
"That is a much wider user base than we could ever have surveyed directly, and it is the reason we went to the papers themselves instead. If we wanted to know what people were doing with DABEST, the only reliable place to look was at what they had published."
]
},
{
"cell_type": "markdown",
"id": "f3a1b5d7",
"metadata": {},
"source": [
"![](figure-2.svg)"
"![](figure-2.svg){fig-align=\"center\" width=\"100%\"}"
]
},
{
"cell_type": "markdown",
"id": "a4b2c6e8",
"metadata": {},
"source": [
"<p style=\"color:#6c757d;\"><b>Figure 2.</b> The field breakdown of the 1,355 works citing DABEST, after removing our own lab's papers, preprints and duplicate records.</p>"
"<p style=\"color:#6c757d; font-size:calc(1em - 2px); line-height:1.2;\"><b>Figure 2.</b> The field breakdown of the 1,355 works citing DABEST, after removing our own lab's papers, preprints and duplicate records.</p>"
]
},
{
"cell_type": "markdown",
"id": "b5c3d7f9",
"id": "e2f9a4c6",
"metadata": {},
"source": [
"So we read the 100 most-cited research papers that cited our 2019 paper, excluding reviews, editorials, tutorials and software papers. For each paper, we recorded where the estimation plot appeared and where p-values were reported **(Figure 3)**. Estimation plots reached the main figures in 74 papers, appeared only in the supplement in 19, and were absent in 7. Nearly all 100 of them reported p-values as well."
"That is a much wider user base than we could ever have surveyed directly. To understand how people were actually using DABEST, we turned to the literature."
]
},
{
"cell_type": "markdown",
"id": "c6d4e8a1",
"metadata": {},
"source": [
"![](figure-3.svg)"
"![](figure-3.svg){fig-align=\"center\" width=\"100%\"}"
]
},
{
"cell_type": "markdown",
"id": "d7e5f9b2",
"metadata": {},
"source": [
"<p style=\"color:#6c757d;\"><b>Figure 3.</b> How the 100 scored papers were selected and sorted. The counts in the top two boxes come from the earlier OpenAlex pull, before preprints and duplicates were removed. The six categories at the bottom were built to capture how each paper used DABEST, by recording whether the estimation plot reached a main figure or stayed in the supplement, and how far the p-values travelled alongside it.</p>"
"<p style=\"color:#6c757d; font-size:calc(1em - 2px); line-height:1.2;\"><b>Figure 3.</b> How the 100 scored papers were selected and sorted. The counts in the top two boxes come from the earlier OpenAlex pull, before preprints and duplicates were removed. The six categories at the bottom were built to capture how each paper used DABEST, by recording whether the estimation plot reached a main figure or stayed in the supplement, and how far the p-values travelled alongside it.</p>"
]
},
{
"cell_type": "markdown",
"id": "e8f6a1c3",
"id": "b5c3d7f9",
"metadata": {},
"source": [
"The more revealing picture emerges when the two answers are placed side by side. The single most common arrangement, covering 68 of the 100 papers, was an estimation plot in a main figure with p-values reported in the main text right alongside it. Only two papers combined a main-figure estimation plot with no p-values anywhere at all **(Figure 4)**."
"We read the 100 most-cited research papers that cited our 2019 paper, excluding reviews, editorials, tutorials and software papers. For each paper, we recorded two things: where the estimation plot appeared and where p-values were reported **(Figure 3)**. Estimation plots reached the main figures in 74 papers, appeared only in the supplement in 19, and were absent in 7. Nearly all 100 of them reported p-values as well. The single most common arrangement, covering 68 of the 100 papers, was an estimation plot in a main figure with p-values reported in the main text right alongside it. Only two papers combined a main-figure estimation plot with no p-values anywhere at all **(Figure 4)**."
]
},
{
"cell_type": "markdown",
"id": "f9a7b2d4",
"metadata": {},
"source": [
"![](figure-4.svg)"
"![](figure-4.svg){fig-align=\"center\" width=\"100%\"}"
]
},
{
"cell_type": "markdown",
"id": "a1b8c3e5",
"metadata": {},
"source": [
"<p style=\"color:#6c757d;\"><b>Figure 4.</b> Where the estimation plot and the p-values landed across the 100 most-cited research articles. The three colored groups are the categories referred to in Figure 3.</p>"
"<p style=\"color:#6c757d; font-size:calc(1em - 2px); line-height:1.2;\"><b>Figure 4.</b> Where the estimation plot and the p-values landed across the 100 most-cited research articles. The three colored groups are the categories referred to in Figure 3.</p>"
]
},
{
Expand Down Expand Up @@ -182,10 +175,10 @@
"`[4]`: ACCLAB. [*DABEST-Python: Data Analysis with Bootstrapped ESTimation*](https://github.com/ACCLAB/DABEST-python). *GitHub*. Accessed 3 Aug. 2026.\n",
"\n",
"<a id='5'></a>\n",
"`[5]`: Calin-Jageman, Robert J., and Geoff Cumming. [“Estimation for Better Inference in Neuroscience.”](https://doi.org/10.1523/ENEURO.0205-19.2019) *eNeuro*, vol. 6, no. 4, 2019, article ENEURO.0205-19.2019.\n",
"`[5]`: Bernard, Christophe. [“Changing the Way We Report, Interpret, and Discuss Our Results to Rebuild Trust in Our Research.”](https://doi.org/10.1523/ENEURO.0259-19.2019) *eNeuro*, vol. 6, no. 4, 2019, article ENEURO.0259-19.2019.\n",
"\n",
"<a id='6'></a>\n",
"`[6]`: Bernard, Christophe. [“Changing the Way We Report, Interpret, and Discuss Our Results to Rebuild Trust in Our Research.”](https://doi.org/10.1523/ENEURO.0259-19.2019) *eNeuro*, vol. 6, no. 4, 2019, article ENEURO.0259-19.2019.\n",
"`[6]`: Calin-Jageman, Robert J., and Geoff Cumming. [“Estimation for Better Inference in Neuroscience.”](https://doi.org/10.1523/ENEURO.0205-19.2019) *eNeuro*, vol. 6, no. 4, 2019, article ENEURO.0205-19.2019.\n",
"\n",
"<a id='7'></a>\n",
"`[7]`: Bernard, Christophe. [“Estimation Statistics, One Year Later.”](https://doi.org/10.1523/ENEURO.0091-21.2021) *eNeuro*, vol. 8, no. 2, 2021, article ENEURO.0091-21.2021.\n",
Expand Down
Loading
Loading