Delete committed dataset orphans (Track X) - #276
Conversation
These files are committed but read by nothing in this repository: every lecture that uses these datasets reads them from QuantEcon/data-lectures (verified per file on 2026-09-01), and the remaining files are written by the lectures themselves with %%file / %%writefile before being read. Translations sit outside the data-lectures audit's scanned set, so this is the translation half of the orphan sweep tracked at QuantEcon/workspace-lectures#57. An authenticated content sweep of 407 repositories (283 QuantEcon org repos plus the forks of the holding repos, by basename, positive and negative controls in the same pass) on 2026-09-01 found no reader of these paths. Mirrors the English repo's Track B deletions (QuantEcon/lecture-python.myst#1035 and #1037): the ols trio, fp.dta, NEWQDATA.csv, both hansen_singleton data files and their make_data.py builders — the builders now live in data-lectures builders/. finite_markov writes web_graph_data.txt with %%file, so neither copy is consumed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
short_path regenerates graph.txt with %%file before reading it, so the committed bytes are never consumed. The 2026-09-01 organisation sweep found no reader of this repo's copy. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Local runs write lectures/graph.txt and lectures/web_graph_data.txt into the working tree; ignoring them prevents the deleted copies from being re-committed by a later git add -A. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
Removes lecture-specific datasets and helper artifacts that are no longer read from this repository (lectures now fetch them from QuantEcon/data-lectures, or generate them at build time), reducing repository bloat and aligning with the upstream Track X orphan sweep.
Changes:
- Deletes committed orphan datasets and graph data files under
lectures/andlectures/_static/lecture_specific/. - Removes the committed Hansen–Singleton dataset builder scripts and their frozen CSV snapshots (now sourced externally).
- Adds
.gitignoreentries to prevent re-committing root-level files regenerated by lecture execution.
Reviewed changes
Copilot reviewed 9 out of 16 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| lectures/web_graph_data.txt | Remove committed graph edge list that is no longer consumed from this repo. |
| lectures/graph.txt | Remove committed graph data file that is no longer consumed from this repo. |
| lectures/_static/lecture_specific/phillips_drifts_volatilities/NEWQDATA.csv | Remove vendored dataset snapshot (lecture fetches from data-lectures). |
| lectures/_static/lecture_specific/pandas_panel/countries.csv | Remove vendored dataset snapshot (lecture fetches from data-lectures). |
| lectures/_static/lecture_specific/hansen_singleton_1983/make_data.py | Remove local dataset builder script (builder moved upstream). |
| lectures/_static/lecture_specific/hansen_singleton_1983/hansen_singleton_1983_data.csv | Remove frozen CSV snapshot (lecture fetches from data-lectures). |
| lectures/_static/lecture_specific/hansen_singleton_1982/make_data.py | Remove local dataset builder script (builder moved upstream). |
| lectures/_static/lecture_specific/hansen_singleton_1982/hansen_singleton_1982_data.csv | Remove frozen CSV snapshot (lecture fetches from data-lectures). |
| lectures/_static/lecture_specific/finite_markov/web_graph_data.txt | Remove vendored graph data snapshot (generated during lecture execution). |
| .gitignore | Add ignore rules for regenerated root-level files. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
✅ Deploy Preview for astonishing-narwhal-a8fc64 ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
|
Validation (2026-09-01, preview |
Deletes the fifteen committed files no lecture in this edition reads, mirroring the English repo's Track B deletions (QuantEcon/lecture-python.myst#1035, #1037) and its Track X PR of the same title: the
olstrio,mle/fp.dta,NEWQDATA.csv, bothhansen_singletondata files and theirmake_data.pybuilders (the builders now live in data-lecturesbuilders/), thepandas_paneltrio, both copies ofweb_graph_data.txt, and the shadowedgraph.txt(separate commit) — plus.gitignoreentries for the two%%file-regenerated root files.Translations sit outside the data-lectures audit's scanned set, so this is the translation half of the orphan sweep: the same files the English repo deletes in its PR of the same title, verified here file by file on 2026-09-01 — every lecture that uses these datasets reads them from
QuantEcon/data-lectures, and the rest are written by the lectures themselves with%%file/%%writefilebefore being read. The translation sync is.md-only, so nothing will re-add them. Every data read inols.md,mle.md,pandas_panel.md,phillips_drifts_volatilities.md,hansen_singleton_1982.mdandhansen_singleton_1983.mdpoints atQuantEcon/data-lectures;finite_markov.mdandshort_path.mdwrite their files with%%file.Published surface and settle. Thirteen of the fifteen serve at 200 on
quantecon.github.io/lecture-python.zh-cn/today (the two root files do not). This repo publishes on apublish*tag (last:publish-2026aug24b) with no_build/htmlprune, so they clear on the settle policy — next Monday clean rebuild, then the next publish.Rows for QuantEcon/workspace-lectures#40 (published URL, measured today with a 200 page control and a 404 never-existed control on the same host; expect 404 after the settle):
/_static/lecture_specific/ols/maketable1.dta/_static/lecture_specific/ols/maketable2.dta/_static/lecture_specific/ols/maketable4.dta/_static/lecture_specific/mle/fp.dta/_static/lecture_specific/phillips_drifts_volatilities/NEWQDATA.csv/_static/lecture_specific/hansen_singleton_1982/hansen_singleton_1982_data.csv/_static/lecture_specific/hansen_singleton_1982/make_data.py/_static/lecture_specific/hansen_singleton_1983/hansen_singleton_1983_data.csv/_static/lecture_specific/hansen_singleton_1983/make_data.py/_static/lecture_specific/pandas_panel/countries.csv/_static/lecture_specific/pandas_panel/employ.csv/_static/lecture_specific/pandas_panel/realwage.csv/_static/lecture_specific/finite_markov/web_graph_data.txtDeletion-time reader sweep (2026-09-01). Authenticated tarball-and-grep pass over 407 repositories: all 283 QuantEcon org repos (70 private, 26 archived;
numfocus,quantecon-book-dpandtest-cliare empty and unfetchable) plus 130 forks of the six holding repos (2 returned 404 and are recorded as not-swept rather than clean). Matched by basename, not URL, with each hit's three preceding lines folded in because URLs get split across source lines. Controls fired in the same pass: a never-existing basename returned 0 hits and 0 copies, and both known readers were found — the Englishpython_advanced_features.mdexercise link totest_table.csv, anddevopseng99/project.lecture-wasm's runtime fetch of intro'sgraph.txt. Result: no repository reads any path this PR deletes. Every other co-occurrence of these basenames is a%%fileembed, prose, a.gitignoreline, or review text; the pre-MySTmaster/source/…reads in old course repos have been dead since the MyST migration and are unaffected.Part of the Track X orphan sweep, QuantEcon/workspace-lectures#57 (one PR per holding repo, identical titles). Source list: https://quantecon.github.io/data-lectures/audit.json
orphans, 2026-08-31 run.🤖 Generated with Claude Code