ClimateAgent

ClimateAgent: Multi-Agent Orchestration for Complex Climate Data Science Workflows

Published in Transactions on Machine Learning Research (TMLR), 2026. [Paper and reviews]

ClimateAgent turns a climate-science question into an executable workflow: it plans the analysis, obtains climate data, generates and runs Python code, corrects failed steps, and produces a report with visualizations.

This repository contains:

• the current ClimateAgent implementation;
• Climate-Agent-Bench-85, with 85 climate-analysis tasks;
• 85 reference reports and their reference figures;
• evaluation scripts and a non-agentic GPT-5 baseline; and
• an optional Next.js application for browsing report examples.

Overview

The Orchestrate-Agent manages context and routes subtasks, the Plan-Agent decomposes each request, the Data-Agent retrieves and validates climate data, and the Coding-Agent generates, debugs, and visualizes the analysis before the final report is returned to the user.

Start here

Goal	Location
Run the primary ClimateAgent workflow	app/report-generate/new-approach/
Browse Climate-Agent-Bench-85 tasks	baselines/prompts/
Inspect the 85 reference reports and figures	experiments_reference/
Evaluate generated reports and code	eval_scripts/
Run the non-agentic GPT-5 baseline	baselines/gpt-5/
Launch the report browser	app/

Climate-Agent-Bench-85

The benchmark tasks and reference outputs are committed to this repository.

Code	Domain	Tasks
AR	Atmospheric rivers	15
DR	Drought	15
EP	Extreme precipitation	15
HW	Heat waves	10
SST	Sea surface temperature	15
TC	Tropical cyclones	15
	Total	85

Each task prompt is stored as baselines/prompts/<DOMAIN>/task<NUMBER>.txt. Its reference output is stored under experiments_reference/task_<DOMAIN>_<NUMBER>/ as report.md and image.png.

Some baseline files retain the earlier internal name Climate-Workflow-85. It refers to the benchmark published as Climate-Agent-Bench-85 in the TMLR paper.

Quick start: run ClimateAgent

Prerequisites:

• Python 3 and an activated Python environment;
• an OpenAI API key; and
• data-provider credentials or access when required by the selected task.

From the repository root:

cd app/report-generate/new-approach
python -m pip install --upgrade pip
python -m pip install -r ../requirements.txt

Create app/report-generate/new-approach/.env:

OPENAI_API_KEY="your-api-key"

Run one benchmark task:

python main.py --prompt ./baselines/prompts/AR/task1.txt

The primary entry point is main.py. It invokes the orchestrator in agents/orchestrating.py, which coordinates planning, data acquisition, programming, execution, retries, visualization, and report generation.

Each run creates:

experiments/task_<timestamp>/
├── context.json
├── data/
├── code/
├── code_output/
│   └── final_report.md
└── log/

For resume options, context-ablation runs, and the experimental Deep Agent variants, see the new-approach guide.

Evaluate a run

After generating an experiment:

cd app/report-generate/new-approach/eval_scripts
python evaluate_system.py \
  --task_dir ../experiments/task_<timestamp> \
  --code-evaluate

The evaluation tools support single-run evaluation, batch evaluation, paired comparison with reference reports, aggregation, and token-usage summaries.

Optional report browser

The Next.js application renders the committed MDX report examples.

pnpm install --frozen-lockfile
pnpm dev

Open http://localhost:3000.

Repository guide

Application and documentation

Folder	Responsibility
app/	Next.js application plus the Python report-generation subsystem.
app/components/	Shared UI components for navigation, report lists, and MDX rendering.
app/report/	Report index, dynamic report pages, MDX parsing utilities, and committed example posts.
app/og/	Dynamic Open Graph image route for the web application.
app/rss/	RSS feed route for published report pages.
public/	Static images served by the Next.js application.
docs/assets/	Images used by the current repository documentation.
docs/archive/	Superseded documentation retained for historical reference.

ClimateAgent and benchmark

Folder	Responsibility	Status
app/report-generate/new-approach/	Current report-generation implementation and command-line entry points.	Primary
new-approach/agents/	Orchestration, planning, ECMWF/CDS download, programming, visualization, user-data inspection, and token tracking.	Primary
new-approach/baselines/prompts/	The 85 tracked Climate-Agent-Bench-85 task prompts, grouped by climate domain.	Benchmark data
new-approach/experiments_reference/	The 85 tracked reference reports and figures used by evaluation.	Reference data
new-approach/eval_scripts/	Current single-task, batch, paired, aggregation, and token-usage evaluation tools.	Evaluation
new-approach/baselines/gpt-5/	Non-agentic best-of-N GPT-5 baseline, benchmark runner, evaluation helpers, and saved baseline artifacts.	Baseline
new-approach/data_download_agent/	ECMWF metadata extraction and download support used by the current data agents, plus standalone helper code.	Supporting
new-approach/deep_agent/	Tool-driven Deep Agent compatibility layer; related entry points and runners live directly under new-approach/.	Experimental
new-approach/evaluation/	Committed historical evaluation snapshots for a subset of extreme-precipitation tasks.	Historical results
new-approach/data/	Local downloaded climate-data workspace; only its README is tracked.	Runtime data
new-approach/experiments/	Generated task directories, user-provided runtime data, code, reports, and traces.	Generated; Git-ignored
new-approach/log/	Runtime planning and execution logs.	Generated; Git-ignored
app/report-generate/code/	Earlier fixed atmospheric-river pipeline scripts.	Legacy
app/report-generate/AR/	Images produced by the earlier atmospheric-river workflow.	Legacy artifacts

Raw climate files such as NetCDF, GRIB, pickle, and CSV outputs are excluded by .gitignore. A fresh clone therefore includes benchmark prompts and reference reports, but not downloaded climate datasets or local experiment runs.

Citation

If you use ClimateAgent or Climate-Agent-Bench-85, please cite:

@article{li2026climateagent,
  title   = {ClimateAgent: Multi-Agent Orchestration for Complex Climate Data Science Workflows},
  author  = {Chenyue Li and Hyeonjae Kim and Wen Deng and Mengxi Jin and HUANG Wen and Mengqian Lu and Binhang Yuan},
  journal = {Transactions on Machine Learning Research},
  issn    = {2835-8856},
  year    = {2026},
  url     = {https://openreview.net/forum?id=XLWvXNumGa}
}

Legacy documentation

The previous root README described an early prototype whose entry-point files no longer exist. It and the historical diagrams not reused above are archived under docs/archive/legacy-readme/ and are not current usage instructions.

License

Except for third-party material identified in THIRD_PARTY_NOTICES.md, this repository is licensed under the Apache License 2.0.