Teach Skill to publish valid Plans

[raghubetina]

Summary

• teach create-full-stack-app that an explicit request to create or publish an app authorizes exactly one singleton private GitHub Publication after observed valid analysis
• keep diagnostics-only work at analysis and local plan compile --output as a separate development path
• define authentication, ambiguous outcome, nonterminal status, terminal fork, and token-handling recovery boundaries
• add behavioral evals and black-box CLI contract coverage for replay, reconciliation, privacy, personal-account ownership, Project/source identity, and every outer lifecycle status

Contract and evidence boundary

This pins the reviewed, merged CLI revision 2d792f20424ae4fcc312d05be6201efb86b1f93b and its full runtime-source digest 7157b01e556d1c8a9eadf591995e251fe96b703bd612d15d991a304cea794e37. CI proves that the pin is reachable from CLI main, detaches to the exact revision, and then exercises the contract. This deliberately keeps the prepared contract distinct from the earlier local Compilation evidence pin.

The Publication endpoint and joined server-to-GitHub path remain unreleased and lack a completed staging smoke. This change does not claim npm availability, a live endpoint, or a real GitHub repository mutation.

Depends on the now-merged firstdraft/cli#14.

Review

Local review on Opus/high completed five passes. All worthwhile feedback was incorporated; the final pass reported no action items.

Verification

• npm ci --ignore-scripts
• sh script/check — 17 tests passed
• node script/check-cli-contract.mjs /Users/sandbox2/code/firstdraft/cli — source runner and packed executable passed against the exact merged commit
• independent Ruby implementation over 15 runtime files — 7157b01e556d1c8a9eadf591995e251fe96b703bd612d15d991a304cea794e37
• uv run --with pyyaml python /Users/sandbox2/.codex/skills/.system/skill-creator/scripts/quick_validate.py skills/create-full-stack-app — Skill is valid
• npm audit --audit-level=high — 0 vulnerabilities
• git diff --check

[raghubetina]

Technical review

sh script/check reproduces at 17 tests passed.

The digest is exactly right

I reimplemented cliRuntimeDigest in Ruby rather than reusing the JavaScript, and ran it against firstdraft/cli checked out at the pinned commit:

files hashed: 15
recomputed:   7157b01e556d1c8a9eadf591995e251fe96b703bd612d15d991a304cea794e37
pinned:       7157b01e556d1c8a9eadf591995e251fe96b703bd612d15d991a304cea794e37

Independent implementation, same value, so the length-prefixed framing is precise enough that a second reading of the spec reproduces it. That is the third time this digest definition has survived that test in this series.

The digest identifies the right bytes. My finding is about whether anyone will be able to find those bytes later.

Finding: the pin is a branch head, and CI checks it out

527c7875 is the current head of firstdraft/cli#14, which is open:

$ gh api repos/firstdraft/cli/compare/main...527c7875… --jq .status
ahead

ahead means reachable from the branch and not an ancestor of main. Compare with the pin being replaced:

$ gh api repos/firstdraft/cli/compare/main...121272cd… --jq .status
behind

behind is an ancestor, which is permanent. So this change moves from a durable pin to a fragile one.

That matters more than usual because of where the value appears. Of the five tracked-source occurrences, one is a checkout:

# .github/workflows/ci.yml:33
ref: 527c7875dc83f2d391978b352d88c340db79fd59

If cli#14 is rebased before merging, that commit stops being reachable and this repository's CI fails at checkout with a reference error. Not a stale link in a document, a red build with a confusing cause.

Rebasing before merge is the norm across these repositories rather than a hypothetical. Working through this series I have watched it happen to firstdraft #264, #265, #270, #273, #277, #278, #279, and #280, and cli#12 itself was rebased between my review and its merge. On that record the pin is more likely to rot than not.

This is also the same failure that produced cli#12. There the pin was c831d505, a pre-rebase head of firstdraft#265, and the fix was repinning to a merged revision. skills#13 then went further and added a preflight to RELEASING.md, with a caveat I thought was better than my finding. The lesson landed for npm names; the same shape has reappeared for Git revisions.

The check is one API call, and it is the same one that would have caught c831d505:

compare/main...SHA  ->  identical or behind = permanent
                        ahead or diverged   = will vanish on rebase

assertRevisionTokens in repository.test.mjs already asserts the exact set of revisions each file may contain, so there is an obvious place to also assert that each one is reachable. That would make this class of mistake impossible to merge rather than something to remember.

On the deliberate part

The body is explicit that this depends on cli#14 and that the endpoint is unreleased, so the dependency is disclosed rather than hidden, and repinning after the CLI merges is clearly the plan. Nothing here is a surprise.

The gap between "we will repin" and "CI keeps working until we do" is what I am pointing at. If the intent is to land this before cli#14, the workflow ref is the one occurrence that needs to be resilient, because the others degrade into stale documentation while that one degrades into a broken build.

The rest holds up

The contract check grew from 7 lines to 841 and now covers replay, reconciliation, privacy, personal-account ownership, Project and source identity, and each outer lifecycle status, exercised against both the source runner and the packed executable. Keeping the provisional publication contract distinct from the earlier local Compilation evidence pin is the right separation, and the body states plainly that there is no live endpoint, no npm availability, and no real repository mutation behind any of it.

One finding

Pin a reachable revision, or accept that ci.yml will break on rebase and land this after cli#14 merges.

[raghubetina]

Lesson: a commit SHA is immutable, which is not the same as permanent

These two properties get conflated constantly, and the difference is where a whole class of broken builds comes from.

Immutable means the bytes at that hash will never be different bytes. Guaranteed by the hash.

Permanent means the object will still be there when you go looking. Guaranteed by nothing, unless something references it.

Git keeps an object alive because a branch, a tag, or a reachable ancestor points at it. Cut every path to a commit and it becomes garbage, and eventually the collector agrees.

How a pull request branch head evaporates

You pin the head of an open PR, because a SHA feels like the careful choice:

- uses: actions/checkout@v7
  with:
    repository: myorg/other-repo
    ref: 527c7875dc83f2d391978b352d88c340db79fd59

Then the PR is rebased. Rebasing does not move commits, it creates new ones with the same changes and different parents. The originals are now unreferenced. The branch is deleted on merge, garbage collection runs, and your ref is gone.

Your build fails with a reference error in a repository nobody touched. The commit that "can never change" cannot be found.

Squash-merging does the same thing, and it is the default on a lot of repositories.

The one-call check

GET /repos/OWNER/REPO/compare/main...SHA

Read status:

• identical or behind: the commit is an ancestor of main, so main keeps it alive forever. Safe to pin.
• ahead or diverged: the commit is not on main. Something else is holding it, and that something is usually a branch that is about to disappear.

Locally the same question is git merge-base --is-ancestor SHA main, which exits 0 when the commit is permanent.

Ten seconds, and it is the difference between a citation and a time bomb.

The wider pattern

This is one instance of something more general: a reference is only as durable as the thing keeping its target alive.

• A Docker tag like node:24 moves. node:24.18.0 moves less. A sha256: digest does not move, until the registry prunes untagged manifests.
• A main-branch permalink to a file drifts, and the line numbers drift faster than the content.
• An S3 object under a lifecycle rule expires on schedule regardless of who is linking to it.
• A CDN URL with a content hash is stable until the deploy that stops referencing it.

In every case, ask what is keeping the target alive, and whether that thing outlives the reference.

When you must cite something impermanent

Sometimes the honest answer is that the thing you ran genuinely is not on main yet. Do not pretend otherwise by picking a nearby merged commit, because that is a small lie about what you tested.

Two options that stay honest:

Anchor by content. If you can hash what you depend on, cite the hash alongside the revision. A digest over the source tree still identifies the exact bytes even after the commit is collected, and anyone with a copy can verify it.

Make the dependency explicit and ordered. If the pin cannot be permanent yet, say so and sequence the merges, so the thing being pinned lands first and the pin is rewritten to its merged revision before anything depends on it.

The rule worth carrying: before pinning a SHA, ask whether main can reach it. If not, you have not pinned anything, you have written down a number that currently happens to resolve.

[raghubetina]

Addressed in 9813085 after merging firstdraft/cli#14 first.

• The permanent CLI revision is 2d792f20424ae4fcc312d05be6201efb86b1f93b; GitHub reports it as cli#14's rebase-merge commit and it is an ancestor of firstdraft/cli main.
• Every provisional 527c7875dc83f2d391978b352d88c340db79fd59 pin was replaced; rg finds no remaining occurrence.
• An independent Ruby implementation hashed the same 15 runtime files with the documented length-prefixed framing and reproduced 7157b01e556d1c8a9eadf591995e251fe96b703bd612d15d991a304cea794e37.
• Skill CI now checks out CLI main with full history, requires git merge-base --is-ancestor 2d792f20424ae4fcc312d05be6201efb86b1f93b HEAD, and only then detaches to that exact revision. The contract harness independently requires git rev-parse HEAD to equal the pin and verifies the runtime digest.
• test/repository.test.mjs requires the full-history checkout, ancestry guard, and exact detached checkout, so a future PR-head pin cannot replace this one unnoticed.

Verification after the correction:

• sh script/check — 17/17 passed
• node script/check-cli-contract.mjs /Users/sandbox2/code/firstdraft/cli — passed
• Skill structural validation — passed
• npm audit --audit-level=high — 0 vulnerabilities
• local Opus/high re-review — no remaining material defects
• hosted Node 22.0.0 and 24.18.0 checks — green

[raghubetina]

Follow-up on 9813085: the pin rotted, and the fix is better than the finding

Recording the outcome, since this stopped being a prediction.

cli#14 merged as 2d792f20, and the SHA this branch previously pinned is now unreachable from that repository's default branch:

$ gh api repos/firstdraft/cli/compare/main...527c7875… --jq .status
diverged

So the rebase-on-merge happened, 527c7875 was orphaned, and the old ref: would have failed at checkout on the next CI run.

The repin is correct. 2d792f20 compares as identical, so it is cli's current main tip and permanent.

The workflow change is the part worth crediting, because it is a stronger answer than the one I suggested:

repository: firstdraft/cli
ref: main
fetch-depth: 0

git merge-base --is-ancestor 2d792f20… HEAD
git checkout --detach 2d792f20…

Rather than checking out a bare SHA and hoping it resolves, this fetches main with full history, proves the pin is an ancestor, and only then detaches to it. A future rot fails at the assertion with an obvious cause instead of an opaque checkout error.

I had suggested asserting reachability inside repository.test.mjs. Enforcing it in the workflow is better: the test would confirm the value on the day it was written, while merge-base --is-ancestor re-confirms it on every run, against the live repository. repository.test.mjs then asserts the workflow contains all three steps, so the guard cannot be quietly dropped.

CI is green on both Node 22.0.0 and 24.18.0.

No further findings.