Skip to content

Delete committed dataset orphans (Track X) - #276

Merged
mmcky merged 3 commits into
mainfrom
track-x-orphans
Sep 1, 2026
Merged

Delete committed dataset orphans (Track X)#276
mmcky merged 3 commits into
mainfrom
track-x-orphans

Conversation

@mmcky

@mmcky mmcky commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

Deletes the fifteen committed files no lecture in this edition reads, mirroring the English repo's Track B deletions (QuantEcon/lecture-python.myst#1035, #1037) and its Track X PR of the same title: the ols trio, mle/fp.dta, NEWQDATA.csv, both hansen_singleton data files and their make_data.py builders (the builders now live in data-lectures builders/), the pandas_panel trio, both copies of web_graph_data.txt, and the shadowed graph.txt (separate commit) — plus .gitignore entries for the two %%file-regenerated root files.

Translations sit outside the data-lectures audit's scanned set, so this is the translation half of the orphan sweep: the same files the English repo deletes in its PR of the same title, verified here file by file on 2026-09-01 — every lecture that uses these datasets reads them from QuantEcon/data-lectures, and the rest are written by the lectures themselves with %%file / %%writefile before being read. The translation sync is .md-only, so nothing will re-add them. Every data read in ols.md, mle.md, pandas_panel.md, phillips_drifts_volatilities.md, hansen_singleton_1982.md and hansen_singleton_1983.md points at QuantEcon/data-lectures; finite_markov.md and short_path.md write their files with %%file.

Published surface and settle. Thirteen of the fifteen serve at 200 on quantecon.github.io/lecture-python.zh-cn/ today (the two root files do not). This repo publishes on a publish* tag (last: publish-2026aug24b) with no _build/html prune, so they clear on the settle policy — next Monday clean rebuild, then the next publish.

Rows for QuantEcon/workspace-lectures#40 (published URL, measured today with a 200 page control and a 404 never-existed control on the same host; expect 404 after the settle):

path today
/_static/lecture_specific/ols/maketable1.dta 200, 14,466 B
/_static/lecture_specific/ols/maketable2.dta 200, 11,210 B
/_static/lecture_specific/ols/maketable4.dta 200, 12,024 B
/_static/lecture_specific/mle/fp.dta 200, 1,024,085 B
/_static/lecture_specific/phillips_drifts_volatilities/NEWQDATA.csv 200, 13,738 B
/_static/lecture_specific/hansen_singleton_1982/hansen_singleton_1982_data.csv 200, 11,662 B
/_static/lecture_specific/hansen_singleton_1982/make_data.py 200, 4,999 B
/_static/lecture_specific/hansen_singleton_1983/hansen_singleton_1983_data.csv 200, 26,084 B
/_static/lecture_specific/hansen_singleton_1983/make_data.py 200, 5,374 B
/_static/lecture_specific/pandas_panel/countries.csv 200, 49,605 B
/_static/lecture_specific/pandas_panel/employ.csv 200, 1,725,234 B
/_static/lecture_specific/pandas_panel/realwage.csv 200, 121,589 B
/_static/lecture_specific/finite_markov/web_graph_data.txt 200, 296 B

Deletion-time reader sweep (2026-09-01). Authenticated tarball-and-grep pass over 407 repositories: all 283 QuantEcon org repos (70 private, 26 archived; numfocus, quantecon-book-dp and test-cli are empty and unfetchable) plus 130 forks of the six holding repos (2 returned 404 and are recorded as not-swept rather than clean). Matched by basename, not URL, with each hit's three preceding lines folded in because URLs get split across source lines. Controls fired in the same pass: a never-existing basename returned 0 hits and 0 copies, and both known readers were found — the English python_advanced_features.md exercise link to test_table.csv, and devopseng99/project.lecture-wasm's runtime fetch of intro's graph.txt. Result: no repository reads any path this PR deletes. Every other co-occurrence of these basenames is a %%file embed, prose, a .gitignore line, or review text; the pre-MyST master/source/… reads in old course repos have been dead since the MyST migration and are unaffected.

Part of the Track X orphan sweep, QuantEcon/workspace-lectures#57 (one PR per holding repo, identical titles). Source list: https://quantecon.github.io/data-lectures/audit.json orphans, 2026-08-31 run.

🤖 Generated with Claude Code

mmcky and others added 3 commits September 1, 2026 10:05
These files are committed but read by nothing in this repository: every lecture that uses these datasets reads them from QuantEcon/data-lectures (verified per file on 2026-09-01), and the remaining files are written by the lectures themselves with %%file / %%writefile before being read. Translations sit outside the data-lectures audit's scanned set, so this is the translation half of the orphan sweep tracked at QuantEcon/workspace-lectures#57. An authenticated content sweep of 407 repositories (283 QuantEcon org repos plus the forks of the holding repos, by basename, positive and negative controls in the same pass) on 2026-09-01 found no reader of these paths.

Mirrors the English repo's Track B deletions (QuantEcon/lecture-python.myst#1035 and #1037): the ols trio, fp.dta, NEWQDATA.csv, both hansen_singleton data files and their make_data.py builders — the builders now live in data-lectures builders/. finite_markov writes web_graph_data.txt with %%file, so neither copy is consumed.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
short_path regenerates graph.txt with %%file before reading it, so the committed bytes are never consumed. The 2026-09-01 organisation sweep found no reader of this repo's copy.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Local runs write lectures/graph.txt and lectures/web_graph_data.txt into the working tree; ignoring them prevents the deleted copies from being re-committed by a later git add -A.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Copilot AI lite review requested due to automatic review settings September 1, 2026 00:08

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Removes lecture-specific datasets and helper artifacts that are no longer read from this repository (lectures now fetch them from QuantEcon/data-lectures, or generate them at build time), reducing repository bloat and aligning with the upstream Track X orphan sweep.

Changes:

  • Deletes committed orphan datasets and graph data files under lectures/ and lectures/_static/lecture_specific/.
  • Removes the committed Hansen–Singleton dataset builder scripts and their frozen CSV snapshots (now sourced externally).
  • Adds .gitignore entries to prevent re-committing root-level files regenerated by lecture execution.

Reviewed changes

Copilot reviewed 9 out of 16 changed files in this pull request and generated no comments.

Show a summary per file
File Description
lectures/web_graph_data.txt Remove committed graph edge list that is no longer consumed from this repo.
lectures/graph.txt Remove committed graph data file that is no longer consumed from this repo.
lectures/_static/lecture_specific/phillips_drifts_volatilities/NEWQDATA.csv Remove vendored dataset snapshot (lecture fetches from data-lectures).
lectures/_static/lecture_specific/pandas_panel/countries.csv Remove vendored dataset snapshot (lecture fetches from data-lectures).
lectures/_static/lecture_specific/hansen_singleton_1983/make_data.py Remove local dataset builder script (builder moved upstream).
lectures/_static/lecture_specific/hansen_singleton_1983/hansen_singleton_1983_data.csv Remove frozen CSV snapshot (lecture fetches from data-lectures).
lectures/_static/lecture_specific/hansen_singleton_1982/make_data.py Remove local dataset builder script (builder moved upstream).
lectures/_static/lecture_specific/hansen_singleton_1982/hansen_singleton_1982_data.csv Remove frozen CSV snapshot (lecture fetches from data-lectures).
lectures/_static/lecture_specific/finite_markov/web_graph_data.txt Remove vendored graph data snapshot (generated during lecture execution).
.gitignore Add ignore rules for regenerated root-level files.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@netlify

netlify Bot commented Sep 1, 2026

Copy link
Copy Markdown

Deploy Preview for astonishing-narwhal-a8fc64 ready!

Name Link
🔨 Latest commit 9e35f0b
🔍 Latest deploy log https://app.netlify.com/projects/astonishing-narwhal-a8fc64/deploys/6a96177c6ea2ad000997425d
😎 Deploy Preview https://deploy-preview-276--astonishing-narwhal-a8fc64.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@github-actions

github-actions Bot commented Sep 1, 2026

Copy link
Copy Markdown

@github-actions
github-actions Bot temporarily deployed to pull request September 1, 2026 00:15 Inactive
@mmcky

mmcky commented Sep 1, 2026

Copy link
Copy Markdown
Contributor Author

Validation (2026-09-01, preview 6a961919…--astonishing-narwhal): root graph.txt and root web_graph_data.txt → 404; ols.html (131,480 B) and mle.html (164,972 B) → 200. Every consuming page reads data-lectures and none references a local _static copy: ols.html 5 refs, mle.html 1, pandas_panel.html 3, phillips_drifts_volatilities.html 1, hansen_singleton_1982.html 1, hansen_singleton_1983.html 1, finite_markov.html shows the %%file cell. This edition has no short_path lecture, so its graph.txt was a pure stray. Build log: 0 warnings mentioning a deleted basename. Expected exception: the thirteen deleted _static paths still serve at 200 on the preview — it restored the 2026-08-31T05:23Z build-cache, built from main. (This is the stale-_build behaviour from QuantEcon/workspace-lectures#41: the preview restored the build-cache artifact built from main before this branch existed, and this repo has no jb clean . --html prune yet, so the restored copies are re-shipped. It says nothing about the PR; on the live site they clear on the settle policy recorded in the PR body.)

@mmcky
mmcky merged commit 23dbb85 into main Sep 1, 2026
8 checks passed
@mmcky
mmcky deleted the track-x-orphans branch September 1, 2026 00:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants