Restore the validation chart provenance gate#91
Open
vahid-ahmadi wants to merge 1 commit into
Open
Conversation
make_charts.py --check has been failing on main since #89: the page carries 10 .vchart SVGs and BUILDERS knew about 7. The three added charts — the OBR computed-share breakdown and the two forecast-skill charts — were authored by hand, so nothing tied their numbers to a source. Root cause is that no workflow ever ran --check. The gate existed and was never enforced, so it stopped holding without anyone noticing. - Builders for all three, reading from source: the two skill charts from papers/boe-svar/figures/rolling_evaluation.json (relative RMSE vs the drift benchmark and its Diebold-Mariano p-values), the scorecard breakdown from a new transcribed chart_data.json entry with per-value source pointers, matching how obr_reform and frbus_residuals are already handled. - site-provenance.yml runs --check on every PR and push, so the gate holds. Every rendered number is unchanged — verified by diffing the numeric labels of all three charts against HEAD. Remaining markup differences are sub-0.1px coordinate rounding and the aria-labelledby ids, which now follow the chart id. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The bug
validation/figures/make_charts.py --checkhas been failing onmainsince #89 — it expects 7.vchartSVGs and the page has 10. The three charts #89 added (obr-computed-share,svar-skill-all,svar-winrate) were authored by hand, so nothing tied their numbers to a source.Root cause: no workflow ever ran
--check. The gate existed and was never enforced, so it stopped holding without anyone noticing. That is the more important half of this fix.What
svar-skill-all/svar-winrate←papers/boe-svar/figures/rolling_evaluation.json(relative_rmse_vs_drift_by_variableanddm_pvalue_vs_drift_by_variable). The hand-authored values check out exactly: win counts 4,4,3,2,3,3,2,2 and significant wins 2,1,1,1,1,0,0,0.obr-computed-share← a newchart_data.jsonentry with per-value source pointers, matching howobr_reformandfrbus_residualsare already handled (the scorecard lives in the OBR repo, so it is transcribed rather than read)..github/workflows/site-provenance.ymlruns--checkon every PR and push.No number changed
Verified by diffing every numeric label in all three charts against
HEAD— identical. Remaining markup differences are sub-0.1px coordinate rounding and thearia-labelledbyids, which now follow the chart id (obr-cs-t→obr-computed-share-t).🤖 Generated with Claude Code