diff --git a/presets/ARCHITECTURE.md b/presets/ARCHITECTURE.md index c533976b8a..2ef78add27 100644 --- a/presets/ARCHITECTURE.md +++ b/presets/ARCHITECTURE.md @@ -59,6 +59,19 @@ Content resolution functions for composition: - **Bash**: `resolve_template_content()` in `scripts/bash/common.sh` (templates only; command/script composition is handled by the Python resolver) - **PowerShell**: `Resolve-TemplateContent` in `scripts/powershell/common.ps1` (templates only; command/script composition is handled by the Python resolver) +### Constitution lifecycle + +Initialization resolves `constitution-template` through the full stack and seeds +`.specify/memory/constitution.md` once. Existing files are preserved byte-for-byte. On subsequent +`/constitution` runs, the command resolves the current composed template at runtime and uses the live +constitution as the source of project-specific values and amendments. + +Preset installation, removal, enablement, disablement, and priority changes do not materialize +`constitution-template` by default. When the enabled preset registry contains `constitution-sync`, +those operations may reconcile the live file, but only if its provenance hash proves it is still +generated content. Missing files may be seeded when the preset is installed; authored or edited +constitutions are never overwritten. + ## Command Registration When a preset is installed with `type: "command"` entries, the `PresetManager` registers them into all detected agent directories using the shared `CommandRegistrar` from `src/specify_cli/agents.py`. diff --git a/presets/README.md b/presets/README.md index 29cce64248..539da08786 100644 --- a/presets/README.md +++ b/presets/README.md @@ -15,6 +15,16 @@ If no preset is installed, core templates are used — exactly the same behavior Template resolution happens **at runtime** — although preset files are copied into `.specify/presets//` during installation, Spec Kit walks the resolution stack on every template lookup rather than merging templates into a single location. +`constitution-template` follows the same runtime model. Project initialization seeds +`.specify/memory/constitution.md` once so downstream commands always have a constitution to read. +After that, installing, removing, enabling, disabling, or reprioritizing presets does not rewrite the +live constitution. Each `/constitution` run resolves the current composed `constitution-template`, +then applies existing project values and amendments to that scaffold. + +Teams that intentionally want preset stack changes to refresh an unchanged generated constitution can +install the bundled `constitution-sync` preset. It restores guarded install-time materialization in +addition to its command-time propagation behavior; authored constitutions remain protected. + For detailed resolution and command registration flows, see [ARCHITECTURE.md](ARCHITECTURE.md). ## Command Overrides diff --git a/presets/catalog.json b/presets/catalog.json index 196115ffb4..39bacb4157 100644 --- a/presets/catalog.json +++ b/presets/catalog.json @@ -1,6 +1,6 @@ { "schema_version": "1.0", - "updated_at": "2026-04-24T00:00:00Z", + "updated_at": "2026-08-04T00:00:00Z", "catalog_url": "https://raw.githubusercontent.com/github/spec-kit/main/presets/catalog.json", "presets": { "lean": { @@ -30,7 +30,7 @@ "name": "Constitution Template Sync", "id": "constitution-sync", "version": "1.0.0", - "description": "Opt-in: restores /constitution propagation of amended guidance into plan/spec/tasks templates and installed command files, for teams that treat materialized templates as reviewed artifacts.", + "description": "Opt-in: restores guarded install-time constitution seeding and /constitution propagation for teams that treat materialized templates as reviewed artifacts.", "author": "github", "repository": "https://github.com/github/spec-kit", "license": "MIT", diff --git a/presets/constitution-sync/README.md b/presets/constitution-sync/README.md index 5c4a9825b4..8eb01d7b7c 100644 --- a/presets/constitution-sync/README.md +++ b/presets/constitution-sync/README.md @@ -1,13 +1,15 @@ # Constitution Template Sync -An **opt-in** preset that restores `/constitution`'s ability to propagate amended guidance into your -project's own templates and command files. After you update the constitution, it aligns -`plan-template.md`, `spec-template.md`, `tasks-template.md`, project-local command files, and +An **opt-in** preset that restores materialized constitution workflows. It refreshes an unchanged +generated `.specify/memory/constitution.md` when constitution-providing presets are installed, +removed, enabled, disabled, or reprioritized. After `/constitution` updates the live file, it also +aligns `plan-template.md`, `spec-template.md`, `tasks-template.md`, project-local command files, and guidance docs so they reflect the current principles. This propagation used to be built into `/constitution`; it was dropped when the command moved to the -preset model. Installing this preset opts you back into it: you get the guidance materialized into -reviewed, committed artifacts instead of relying on runtime resolution alone. +preset model. Installing this preset opts you back into materialization: preset stack changes refresh +the generated constitution, and `/constitution` propagates its guidance into reviewed, committed +artifacts instead of relying on runtime resolution alone. > **What you're opting into.** Propagation was removed deliberately — it duplicates the constitution > as the source of truth and can fight the composition stack (materialized edits get shadowed or @@ -28,7 +30,12 @@ versioned preset a core team maintains. ## What it does -Ships a single `wrap`-strategy override of `speckit.constitution`. It composes on top of the +Its presence enables core's guarded install-time constitution reconciliation. Installing the preset +materializes the currently resolved `constitution-template`; later stack changes re-materialize it +only while the live file still matches its recorded generated-content hash. Human edits disable +automatic replacement. + +It also ships a single `wrap`-strategy override of `speckit.constitution`. It composes on top of the current core command (via `{CORE_TEMPLATE}`), so it stays forward-compatible with core changes, and appends a propagation pass that, after the constitution is written: @@ -43,6 +50,8 @@ appends a propagation pass that, after the constitution is written: - It does **not** disable runtime resolution. `plan`, `tasks`, and `analyze` still read the live constitution every run; this preset adds materialized copies on top — it does not replace the source of truth. +- It does **not** overwrite an authored or edited constitution. Install-time reconciliation only + replaces content whose provenance proves it is an unchanged generated file. - It does **not** edit versioned, package-owned files — templates or command files provided or wrapped by another preset or extension. Those are recomposed from the resolution stack, so it only ever writes into your project's own `.specify/templates/` scaffolds and command files that diff --git a/presets/constitution-sync/preset.yml b/presets/constitution-sync/preset.yml index 574faa9698..a54265f65a 100644 --- a/presets/constitution-sync/preset.yml +++ b/presets/constitution-sync/preset.yml @@ -4,7 +4,7 @@ preset: id: "constitution-sync" name: "Constitution Template Sync" version: "1.0.0" - description: "Opt-in: restores /constitution propagation of amended guidance into plan/spec/tasks templates and installed command files, for teams that treat materialized templates as reviewed artifacts." + description: "Opt-in: restores guarded install-time constitution seeding and /constitution propagation for teams that treat materialized templates as reviewed artifacts." author: "github" repository: "https://github.com/github/spec-kit" license: "MIT" diff --git a/scripts/bash/check-prerequisites.sh b/scripts/bash/check-prerequisites.sh index b9688d6742..c21edc41f0 100644 --- a/scripts/bash/check-prerequisites.sh +++ b/scripts/bash/check-prerequisites.sh @@ -12,6 +12,7 @@ # --require-tasks Require tasks.md to exist (for implementation phase) # --include-tasks Include tasks.md in AVAILABLE_DOCS list # --paths-only Only output path variables (no validation) +# --template NAME Include composed template content in JSON output # --help, -h Show help message # # OUTPUTS: @@ -26,9 +27,10 @@ JSON_MODE=false REQUIRE_TASKS=false INCLUDE_TASKS=false PATHS_ONLY=false +TEMPLATE_NAME="" -for arg in "$@"; do - case "$arg" in +while [[ $# -gt 0 ]]; do + case "$1" in --json) JSON_MODE=true ;; @@ -41,6 +43,14 @@ for arg in "$@"; do --paths-only) PATHS_ONLY=true ;; + --template) + shift + if [[ $# -eq 0 ]]; then + echo "ERROR: --template requires a template name" >&2 + exit 1 + fi + TEMPLATE_NAME="$1" + ;; --help|-h) cat << 'EOF' Usage: check-prerequisites.sh [OPTIONS] @@ -52,6 +62,7 @@ OPTIONS: --require-tasks Require tasks.md to exist (for implementation phase) --include-tasks Include tasks.md in AVAILABLE_DOCS list --paths-only Only output path variables (no prerequisite validation) + --template NAME Include composed template content in JSON output --help, -h Show this help message EXAMPLES: @@ -68,10 +79,11 @@ EOF exit 0 ;; *) - echo "ERROR: Unknown option '$arg'. Use --help for usage information." >&2 + echo "ERROR: Unknown option '$1'. Use --help for usage information." >&2 exit 1 ;; esac + shift done # Source common functions @@ -156,6 +168,16 @@ if $INCLUDE_TASKS && [[ -f "$TASKS" ]]; then docs+=("tasks.md") fi +TEMPLATE_CONTENT="" +if [[ -n "$TEMPLATE_NAME" ]]; then + if TEMPLATE_CONTENT=$(resolve_template_content "$TEMPLATE_NAME" "$REPO_ROOT"; status=$?; printf x; exit "$status"); then + TEMPLATE_CONTENT="${TEMPLATE_CONTENT%x}" + else + echo "ERROR: Could not resolve required $TEMPLATE_NAME from the template override stack for $REPO_ROOT" >&2 + exit 1 + fi +fi + # Output results if $JSON_MODE; then # Build JSON array of documents @@ -165,10 +187,18 @@ if $JSON_MODE; then else json_docs=$(printf '%s\n' "${docs[@]}" | jq -R . | jq -s .) fi - jq -cn \ - --arg feature_dir "$FEATURE_DIR" \ - --argjson docs "$json_docs" \ - '{FEATURE_DIR:$feature_dir,AVAILABLE_DOCS:$docs}' + if [[ -n "$TEMPLATE_NAME" ]]; then + jq -cn \ + --arg feature_dir "$FEATURE_DIR" \ + --argjson docs "$json_docs" \ + --arg template_content "$TEMPLATE_CONTENT" \ + '{FEATURE_DIR:$feature_dir,AVAILABLE_DOCS:$docs,TEMPLATE_CONTENT:$template_content}' + else + jq -cn \ + --arg feature_dir "$FEATURE_DIR" \ + --argjson docs "$json_docs" \ + '{FEATURE_DIR:$feature_dir,AVAILABLE_DOCS:$docs}' + fi else if [[ ${#docs[@]} -eq 0 ]]; then json_docs="[]" @@ -176,7 +206,12 @@ if $JSON_MODE; then json_docs=$(for d in "${docs[@]}"; do printf '"%s",' "$(json_escape "$d")"; done) json_docs="[${json_docs%,}]" fi - printf '{"FEATURE_DIR":"%s","AVAILABLE_DOCS":%s}\n' "$(json_escape "$FEATURE_DIR")" "$json_docs" + if [[ -n "$TEMPLATE_NAME" ]]; then + printf '{"FEATURE_DIR":"%s","AVAILABLE_DOCS":%s,"TEMPLATE_CONTENT":"%s"}\n' \ + "$(json_escape "$FEATURE_DIR")" "$json_docs" "$(json_escape "$TEMPLATE_CONTENT")" + else + printf '{"FEATURE_DIR":"%s","AVAILABLE_DOCS":%s}\n' "$(json_escape "$FEATURE_DIR")" "$json_docs" + fi fi else # Text output diff --git a/scripts/bash/common.sh b/scripts/bash/common.sh index dc60f9ff5d..33f90b8dbb 100644 --- a/scripts/bash/common.sh +++ b/scripts/bash/common.sh @@ -398,6 +398,101 @@ json_escape() { check_file() { [[ -f "$1" ]] && echo " ✓ $2" || echo " ✗ $2"; } check_dir() { [[ -d "$1" && -n $(ls -A "$1" 2>/dev/null) ]] && echo " ✓ $2" || echo " ✗ $2"; } +_python3_command() { + if command -v python3 >/dev/null 2>&1 && + python3 -c 'import sys; raise SystemExit(sys.version_info.major != 3)' >/dev/null 2>&1; then + printf '%s\n' "python3" + elif command -v python >/dev/null 2>&1 && + python -c 'import sys; raise SystemExit(sys.version_info.major != 3)' >/dev/null 2>&1; then + printf '%s\n' "python" + elif command -v py >/dev/null 2>&1 && + py -3 -c 'import sys' >/dev/null 2>&1; then + printf '%s\n' "py -3" + else + return 1 + fi +} + +_sorted_extension_ids() { + local ext_dir="$1" + local python_spec + if python_spec=$(_python3_command); then + local -a python_cmd + read -r -a python_cmd <<< "$python_spec" + local py_stderr sorted_ids + py_stderr=$(mktemp) + if sorted_ids=$(SPECKIT_EXTENSIONS="$ext_dir" "${python_cmd[@]}" -c " +import json, os, re, sys +from pathlib import Path + +root = Path(os.environ['SPECKIT_EXTENSIONS']) +registered = {} +registry = root / '.registry' +if os.path.lexists(registry): + if not registry.is_file(): + print('registry_invalid: not a regular file', file=sys.stderr) + sys.exit(1) + try: + data = json.loads(registry.read_text(encoding='utf-8')) + except Exception as exc: + print('registry_invalid: ' + str(exc), file=sys.stderr) + sys.exit(1) + if not isinstance(data, dict): + print('registry_invalid: root must be a mapping', file=sys.stderr) + sys.exit(1) + raw_extensions = data.get('extensions', {}) + if not isinstance(raw_extensions, dict): + print('registry_invalid: extensions must be a mapping', file=sys.stderr) + sys.exit(1) + registered = raw_extensions + +def priority(value): + if isinstance(value, bool): + return 10 + try: + parsed = int(value) + return parsed if parsed >= 1 else 10 + except (TypeError, ValueError, OverflowError): + return 10 + +ranked = [] +for ext_id, meta in registered.items(): + if isinstance(ext_id, str) and re.fullmatch(r'[a-z0-9-]+', ext_id) and isinstance(meta, dict) and bool(meta.get('enabled', True)): + ranked.append((priority(meta.get('priority')), ext_id)) +for path in root.iterdir(): + if path.is_dir() and re.fullmatch(r'[a-z0-9-]+', path.name) and path.name not in registered: + ranked.append((10, path.name)) +for _, ext_id in sorted(ranked): + print(ext_id) +" 2>"$py_stderr"); then + rm -f "$py_stderr" + printf '%s\n' "$sorted_ids" + return 0 + else + echo "Error: invalid extension registry $ext_dir/.registry" >&2 + rm -f "$py_stderr" + return 1 + fi + fi + + if [ -e "$ext_dir/.registry" ] || [ -L "$ext_dir/.registry" ]; then + if [ ! -f "$ext_dir/.registry" ] || [ ! -r "$ext_dir/.registry" ]; then + echo "Error: invalid extension registry $ext_dir/.registry" >&2 + return 1 + fi + echo "Error: Python 3 is required to honor the extension registry" >&2 + return 2 + fi + + local ext extension_id + for ext in "$ext_dir"/*/; do + [ -d "$ext" ] || continue + extension_id=$(basename "$ext") + case "$extension_id" in *[!a-z0-9-]*) continue ;; esac + printf '%s\n' "$extension_id" + done +} + # Resolve a template name to a file path using the priority stack: # 1. .specify/templates/overrides/ # 2. .specify/presets//templates/ (sorted by priority from .registry) @@ -408,6 +503,8 @@ resolve_template() { local repo_root="$2" local base="$repo_root/.specify/templates" + case "$template_name" in ""|*[!a-z0-9-]*) return 1 ;; esac + # Priority 1: Project overrides local override="$base/overrides/${template_name}.md" [ -f "$override" ] && echo "$override" && return 0 @@ -416,19 +513,32 @@ resolve_template() { local presets_dir="$repo_root/.specify/presets" if [ -d "$presets_dir" ]; then local registry_file="$presets_dir/.registry" - if [ -f "$registry_file" ] && command -v python3 >/dev/null 2>&1; then + local python_spec="" + local -a python_cmd=() + if python_spec=$(_python3_command); then + read -r -a python_cmd <<< "$python_spec" + fi + if [ -f "$registry_file" ] && [ "${#python_cmd[@]}" -gt 0 ]; then # Read preset IDs sorted by priority (lower number = higher precedence). # The python3 call is wrapped in an if-condition so that set -e does not # abort the function when python3 exits non-zero (e.g. invalid JSON). local sorted_presets="" - if sorted_presets=$(SPECKIT_REGISTRY="$registry_file" python3 -c " -import json, sys, os + if sorted_presets=$(SPECKIT_REGISTRY="$registry_file" "${python_cmd[@]}" -c " +import json, re, sys, os try: - with open(os.environ['SPECKIT_REGISTRY']) as f: + with open(os.environ['SPECKIT_REGISTRY'], encoding='utf-8') as f: data = json.load(f) presets = data.get('presets', {}) - for pid, meta in sorted(presets.items(), key=lambda x: x[1].get('priority', 10) if isinstance(x[1], dict) else 10): - if isinstance(meta, dict) and meta.get('enabled', True) is not False: + def priority(meta): + if not isinstance(meta, dict) or isinstance(meta.get('priority'), bool): + return 10 + try: + value = int(meta.get('priority', 10)) + return value if value >= 1 else 10 + except (TypeError, ValueError, OverflowError): + return 10 + for pid, meta in sorted(presets.items(), key=lambda x: (priority(x[1]), x[0])): + if isinstance(meta, dict) and bool(meta.get('enabled', True)) and re.fullmatch(r'[a-z0-9-]+', pid): print(pid) except Exception: sys.exit(1) @@ -438,6 +548,8 @@ except Exception: while IFS= read -r preset_id; do local candidate="$presets_dir/$preset_id/templates/${template_name}.md" [ -f "$candidate" ] && echo "$candidate" && return 0 + candidate="$presets_dir/$preset_id/${template_name}.md" + [ -f "$candidate" ] && echo "$candidate" && return 0 done <<< "$sorted_presets" fi # python3 succeeded but registry has no presets — nothing to search @@ -447,6 +559,8 @@ except Exception: [ -d "$preset" ] || continue local candidate="$preset/templates/${template_name}.md" [ -f "$candidate" ] && echo "$candidate" && return 0 + candidate="$preset/${template_name}.md" + [ -f "$candidate" ] && echo "$candidate" && return 0 done fi else @@ -455,6 +569,8 @@ except Exception: [ -d "$preset" ] || continue local candidate="$preset/templates/${template_name}.md" [ -f "$candidate" ] && echo "$candidate" && return 0 + candidate="$preset/${template_name}.md" + [ -f "$candidate" ] && echo "$candidate" && return 0 done fi fi @@ -462,13 +578,17 @@ except Exception: # Priority 3: Extension-provided templates local ext_dir="$repo_root/.specify/extensions" if [ -d "$ext_dir" ]; then - for ext in "$ext_dir"/*/; do - [ -d "$ext" ] || continue - # Skip hidden directories (e.g. .backup, .cache) - case "$(basename "$ext")" in .*) continue;; esac + local sorted_extensions="" + if ! sorted_extensions=$(_sorted_extension_ids "$ext_dir"); then + return 2 + fi + while IFS= read -r extension_id; do + [ -n "$extension_id" ] || continue + local ext="$ext_dir/$extension_id" local candidate="$ext/templates/${template_name}.md" + [ -f "$candidate" ] || candidate="$ext/${template_name}.md" [ -f "$candidate" ] && echo "$candidate" && return 0 - done + done <<< "$sorted_extensions" fi # Priority 4: Core templates @@ -492,6 +612,8 @@ resolve_template_content() { local repo_root="$2" local base="$repo_root/.specify/templates" + case "$template_name" in ""|*[!a-z0-9-]*) return 1 ;; esac + # Collect all layers (highest priority first) local -a layer_paths=() local -a layer_strategies=() @@ -499,133 +621,206 @@ resolve_template_content() { # Priority 1: Project overrides (always "replace") local override="$base/overrides/${template_name}.md" if [ -f "$override" ]; then - layer_paths+=("$override") - layer_strategies+=("replace") + if ! cat "$override"; then + echo "Error: failed to read template layer $override" >&2 + return 2 + fi + return 0 fi + local effective_base_found=false + # Priority 2: Installed presets (sorted by priority from .registry) local presets_dir="$repo_root/.specify/presets" if [ -d "$presets_dir" ]; then local registry_file="$presets_dir/.registry" local sorted_presets="" - if [ -f "$registry_file" ] && command -v python3 >/dev/null 2>&1; then - if sorted_presets=$(SPECKIT_REGISTRY="$registry_file" python3 -c " -import json, sys, os + local registry_parsed=false + local python_spec="" + local -a python_cmd=() + if python_spec=$(_python3_command); then + read -r -a python_cmd <<< "$python_spec" + fi + if [ -f "$registry_file" ] && [ "${#python_cmd[@]}" -gt 0 ]; then + if sorted_presets=$(SPECKIT_REGISTRY="$registry_file" "${python_cmd[@]}" -c " +import json, re, sys, os try: - with open(os.environ['SPECKIT_REGISTRY']) as f: + with open(os.environ['SPECKIT_REGISTRY'], encoding='utf-8') as f: data = json.load(f) presets = data.get('presets', {}) - for pid, meta in sorted(presets.items(), key=lambda x: x[1].get('priority', 10) if isinstance(x[1], dict) else 10): - if isinstance(meta, dict) and meta.get('enabled', True) is not False: + def priority(meta): + if not isinstance(meta, dict) or isinstance(meta.get('priority'), bool): + return 10 + try: + value = int(meta.get('priority', 10)) + return value if value >= 1 else 10 + except (TypeError, ValueError, OverflowError): + return 10 + for pid, meta in sorted(presets.items(), key=lambda x: (priority(x[1]), x[0])): + if isinstance(meta, dict) and bool(meta.get('enabled', True)) and re.fullmatch(r'[a-z0-9-]+', pid): print(pid) except Exception: sys.exit(1) " 2>/dev/null); then - if [ -n "$sorted_presets" ]; then - local yaml_warned=false - while IFS= read -r preset_id; do - # Read strategy and file path from preset manifest - local strategy="replace" - local manifest_file="" - local manifest="$presets_dir/$preset_id/preset.yml" - if [ -f "$manifest" ] && command -v python3 >/dev/null 2>&1; then - # Requires PyYAML; falls back to replace/convention if unavailable - local result - local py_stderr - py_stderr=$(mktemp) - result=$(SPECKIT_MANIFEST="$manifest" SPECKIT_TMPL="$template_name" python3 -c " + registry_parsed=true + fi + fi + if [ "$registry_parsed" = false ]; then + for preset in "$presets_dir"/*/; do + [ -d "$preset" ] || continue + local fallback_id + fallback_id=$(basename "$preset") + case "$fallback_id" in *[!a-z0-9-]*) continue ;; esac + sorted_presets+="${sorted_presets:+$'\n'}$fallback_id" + done + fi + + if [ -n "$sorted_presets" ]; then + while IFS= read -r preset_id; do + local strategy="replace" + local manifest_file="" + local manifest="$presets_dir/$preset_id/preset.yml" + local manifest_declared=false + if [ -f "$manifest" ]; then + if [ "${#python_cmd[@]}" -eq 0 ]; then + echo "Error: Python 3 and PyYAML are required to resolve preset template composition" >&2 + return 2 + fi + local result + local py_stderr + local parse_status + py_stderr=$(mktemp) + if result=$(SPECKIT_MANIFEST="$manifest" SPECKIT_TMPL="$template_name" "${python_cmd[@]}" -c " import sys, os try: import yaml except ImportError: print('yaml_missing', file=sys.stderr) - print('replace\t') - sys.exit(0) + sys.exit(2) try: - with open(os.environ['SPECKIT_MANIFEST']) as f: + with open(os.environ['SPECKIT_MANIFEST'], encoding='utf-8') as f: data = yaml.safe_load(f) - for t in data.get('provides', {}).get('templates', []): + if not isinstance(data, dict): + raise ValueError('manifest root must be a mapping') + if 'provides' not in data: + raise ValueError('manifest missing provides section') + provides = data['provides'] + if not isinstance(provides, dict): + raise ValueError('manifest provides must be a mapping') + if 'templates' not in provides: + raise ValueError('manifest provides missing templates') + templates = provides['templates'] + if not isinstance(templates, list): + raise ValueError('manifest templates must be a list') + if not templates: + raise ValueError('manifest must provide at least one template') + valid_types = ('template', 'command', 'script') + valid_strategies = ('replace', 'prepend', 'append', 'wrap') + for t in templates: + if not isinstance(t, dict): + raise ValueError('manifest template entries must be mappings') + if 'type' not in t or 'name' not in t or 'file' not in t: + raise ValueError('manifest template entry missing type, name, or file') + for field in ('type', 'name', 'file'): + if not isinstance(t[field], str): + raise ValueError('manifest template ' + field + ' must be a string') + if t['type'] not in valid_types: + raise ValueError('invalid manifest template type') + strategy = t.get('strategy', 'replace') + if not isinstance(strategy, str): + raise ValueError('manifest template strategy must be a string') + strategy = strategy.lower() + if strategy not in valid_strategies: + raise ValueError('invalid manifest template strategy') + if t['type'] == 'script' and strategy not in ('replace', 'wrap'): + raise ValueError('invalid manifest script strategy') + for t in templates: if t.get('name') == os.environ['SPECKIT_TMPL'] and t.get('type', 'template') == 'template': - print(t.get('strategy', 'replace') + '\t' + t.get('file', '')) + file_value = t.get('file', '') + strategy = t.get('strategy', 'replace') + print('found\t' + strategy + '\t' + file_value) sys.exit(0) - print('replace\t') -except Exception: - print('replace\t') -" 2>"$py_stderr") - local parse_status=$? - if [ $parse_status -eq 0 ] && [ -n "$result" ]; then - IFS=$'\t' read -r strategy manifest_file <<< "$result" - strategy=$(printf '%s' "$strategy" | tr '[:upper:]' '[:lower:]') - fi - if [ "$yaml_warned" = false ] && grep -q 'yaml_missing' "$py_stderr" 2>/dev/null; then - echo "Warning: PyYAML not available; composition strategies may be ignored" >&2 - yaml_warned=true - fi - rm -f "$py_stderr" - fi - # Try manifest file path first, then convention path - local candidate="" - if [ -n "$manifest_file" ]; then - # Reject absolute paths and parent traversal - case "$manifest_file" in - /*|*../*|../*) manifest_file="" ;; - esac - fi - if [ -n "$manifest_file" ]; then - local mf="$presets_dir/$preset_id/$manifest_file" - [ -f "$mf" ] && candidate="$mf" - fi - if [ -z "$candidate" ]; then - local cf="$presets_dir/$preset_id/templates/${template_name}.md" - [ -f "$cf" ] && candidate="$cf" - fi - if [ -n "$candidate" ]; then - layer_paths+=("$candidate") - layer_strategies+=("$strategy") + print('absent\treplace\t') +except Exception as exc: + print(f'manifest_invalid: {exc}', file=sys.stderr) + sys.exit(3) +" 2>"$py_stderr"); then + parse_status=0 + else + parse_status=$? + fi + if [ "$parse_status" -ne 0 ]; then + if [ "$parse_status" -eq 2 ]; then + echo "Error: PyYAML is required to resolve preset template composition" >&2 + else + echo "Error: invalid preset manifest $manifest" >&2 fi - done <<< "$sorted_presets" + rm -f "$py_stderr" + return 2 + fi + if [ -n "$result" ]; then + local declaration + IFS=$'\t' read -r declaration strategy manifest_file <<< "$result" + [ "$declaration" = "found" ] && manifest_declared=true + strategy=$(printf '%s' "$strategy" | tr '[:upper:]' '[:lower:]') + fi + rm -f "$py_stderr" fi - else - # python3 failed — fall back to unordered directory scan (replace only) - for preset in "$presets_dir"/*/; do - [ -d "$preset" ] || continue - local candidate="$preset/templates/${template_name}.md" - if [ -f "$candidate" ]; then - layer_paths+=("$candidate") - layer_strategies+=("replace") + + local candidate="" + if [ -n "$manifest_file" ]; then + case "$manifest_file" in + /*|*../*|../*) manifest_file="" ;; + esac + fi + if [ -n "$manifest_file" ]; then + local mf="$presets_dir/$preset_id/$manifest_file" + [ -f "$mf" ] && candidate="$mf" + fi + if [ -z "$candidate" ] && [ "$manifest_declared" = false ]; then + local cf="$presets_dir/$preset_id/templates/${template_name}.md" + [ -f "$cf" ] && candidate="$cf" + if [ -z "$candidate" ]; then + cf="$presets_dir/$preset_id/${template_name}.md" + [ -f "$cf" ] && candidate="$cf" fi - done - fi - else - # No python3 or registry — fall back to unordered directory scan (replace only) - for preset in "$presets_dir"/*/; do - [ -d "$preset" ] || continue - local candidate="$preset/templates/${template_name}.md" - if [ -f "$candidate" ]; then + fi + if [ -n "$candidate" ]; then layer_paths+=("$candidate") - layer_strategies+=("replace") + layer_strategies+=("$strategy") + if [ "$strategy" = "replace" ]; then + effective_base_found=true + break + fi fi - done + done <<< "$sorted_presets" fi fi # Priority 3: Extension-provided templates (always "replace") local ext_dir="$repo_root/.specify/extensions" - if [ -d "$ext_dir" ]; then - for ext in "$ext_dir"/*/; do - [ -d "$ext" ] || continue - case "$(basename "$ext")" in .*) continue;; esac + if [ "$effective_base_found" = false ] && [ -d "$ext_dir" ]; then + local sorted_extensions="" + if ! sorted_extensions=$(_sorted_extension_ids "$ext_dir"); then + return 2 + fi + while IFS= read -r extension_id; do + [ -n "$extension_id" ] || continue + local ext="$ext_dir/$extension_id" local candidate="$ext/templates/${template_name}.md" + [ -f "$candidate" ] || candidate="$ext/${template_name}.md" if [ -f "$candidate" ]; then layer_paths+=("$candidate") layer_strategies+=("replace") + effective_base_found=true + break fi - done + done <<< "$sorted_extensions" fi # Priority 4: Core templates (always "replace") local core="$base/${template_name}.md" - if [ -f "$core" ]; then + if [ "$effective_base_found" = false ] && [ -f "$core" ]; then layer_paths+=("$core") layer_strategies+=("replace") fi @@ -642,12 +837,18 @@ except Exception: # If the top (highest-priority) layer is replace, it wins entirely — # lower layers are irrelevant regardless of their strategies. if [ "${layer_strategies[0]}" = "replace" ]; then - cat "${layer_paths[0]}" + if ! cat "${layer_paths[0]}"; then + echo "Error: failed to read template layer ${layer_paths[0]}" >&2 + return 2 + fi return 0 fi if [ "$has_composition" = false ]; then - cat "${layer_paths[0]}" + if ! cat "${layer_paths[0]}"; then + echo "Error: failed to read template layer ${layer_paths[0]}" >&2 + return 2 + fi return 0 fi @@ -663,12 +864,16 @@ except Exception: done if [ $base_idx -lt 0 ]; then - return 1 # no base layer found + echo "Error: template '$template_name' has composing layers but no replace base" >&2 + return 2 fi # Read the base content; compose layers above the base (higher priority) local content - content=$(cat "${layer_paths[$base_idx]}"; printf x) + if ! content=$(cat "${layer_paths[$base_idx]}"; status=$?; printf x; exit "$status"); then + echo "Error: failed to read template layer ${layer_paths[$base_idx]}" >&2 + return 2 + fi content="${content%x}" for (( i=base_idx-1; i>=0; i-- )); do @@ -676,17 +881,26 @@ except Exception: local strat="${layer_strategies[$i]}" local layer_content # Preserve trailing newlines - layer_content=$(cat "$path"; printf x) + if ! layer_content=$(cat "$path"; status=$?; printf x; exit "$status"); then + echo "Error: failed to read template layer $path" >&2 + return 2 + fi layer_content="${layer_content%x}" case "$strat" in replace) content="$layer_content" ;; - prepend) content="$(printf '%s\n\n%s' "$layer_content" "$content")" ;; - append) content="$(printf '%s\n\n%s' "$content" "$layer_content")" ;; + prepend) + content=$(printf '%s\n\n%s' "$layer_content" "$content"; printf x) + content="${content%x}" + ;; + append) + content=$(printf '%s\n\n%s' "$content" "$layer_content"; printf x) + content="${content%x}" + ;; wrap) case "$layer_content" in *'{CORE_TEMPLATE}'*) ;; - *) echo "Error: wrap strategy missing {CORE_TEMPLATE} placeholder" >&2; return 1 ;; + *) echo "Error: wrap strategy missing {CORE_TEMPLATE} placeholder" >&2; return 2 ;; esac while [[ "$layer_content" == *'{CORE_TEMPLATE}'* ]]; do local before="${layer_content%%\{CORE_TEMPLATE\}*}" @@ -695,7 +909,7 @@ except Exception: done content="$layer_content" ;; - *) echo "Error: unknown strategy '$strat'" >&2; return 1 ;; + *) echo "Error: unknown strategy '$strat'" >&2; return 2 ;; esac done diff --git a/scripts/bash/create-new-feature.sh b/scripts/bash/create-new-feature.sh index c1b189dc08..abdb2194b1 100644 --- a/scripts/bash/create-new-feature.sh +++ b/scripts/bash/create-new-feature.sh @@ -339,12 +339,27 @@ if [ "$DRY_RUN" != true ]; then exit 1 fi + NEEDS_SPEC=false + SPEC_TEMPLATE_FOUND=false + SPEC_TEMPLATE_CONTENT="" + if [ ! -f "$SPEC_FILE" ]; then + NEEDS_SPEC=true + if SPEC_TEMPLATE_CONTENT=$(resolve_template_content "spec-template" "$REPO_ROOT"; status=$?; printf x; exit "$status"); then + SPEC_TEMPLATE_CONTENT="${SPEC_TEMPLATE_CONTENT%x}" + SPEC_TEMPLATE_FOUND=true + else + resolve_status=$? + if [ "$resolve_status" -ne 1 ]; then + exit "$resolve_status" + fi + fi + fi + mkdir -p "$FEATURE_DIR" - if [ ! -f "$SPEC_FILE" ]; then - TEMPLATE=$(resolve_template "spec-template" "$REPO_ROOT") || true - if [ -n "$TEMPLATE" ] && [ -f "$TEMPLATE" ]; then - cp "$TEMPLATE" "$SPEC_FILE" + if [ "$NEEDS_SPEC" = true ]; then + if [ "$SPEC_TEMPLATE_FOUND" = true ]; then + printf '%s' "$SPEC_TEMPLATE_CONTENT" > "$SPEC_FILE" else echo "Warning: Spec template not found; created empty spec file" >&2 touch "$SPEC_FILE" diff --git a/scripts/bash/resolve-template.sh b/scripts/bash/resolve-template.sh new file mode 100644 index 0000000000..da05d2df6d --- /dev/null +++ b/scripts/bash/resolve-template.sh @@ -0,0 +1,57 @@ +#!/usr/bin/env bash + +set -e + +SCRIPT_DIR="$(CDPATH="" cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)" +source "$SCRIPT_DIR/common.sh" + +JSON_MODE=false +TEMPLATE_NAME="" + +for arg in "$@"; do + case "$arg" in + --json) JSON_MODE=true ;; + --help|-h) + echo "Usage: $0 [--json]" + exit 0 + ;; + -*) + echo "ERROR: Unknown option '$arg'" >&2 + exit 1 + ;; + *) + if [[ -n "$TEMPLATE_NAME" ]]; then + echo "ERROR: Unexpected argument '$arg'" >&2 + exit 1 + fi + TEMPLATE_NAME="$arg" + ;; + esac +done + +if [[ -z "$TEMPLATE_NAME" ]]; then + echo "ERROR: Template name is required" >&2 + exit 1 +fi + +REPO_ROOT=$(get_repo_root) +if TEMPLATE_CONTENT=$(resolve_template_content "$TEMPLATE_NAME" "$REPO_ROOT"; status=$?; printf x; exit "$status"); then + TEMPLATE_CONTENT="${TEMPLATE_CONTENT%x}" +else + echo "ERROR: Could not resolve required $TEMPLATE_NAME from the template override stack for $REPO_ROOT" >&2 + exit 1 +fi + +if $JSON_MODE; then + if has_jq; then + jq -cn \ + --arg template_name "$TEMPLATE_NAME" \ + --arg template_content "$TEMPLATE_CONTENT" \ + '{TEMPLATE_NAME:$template_name,TEMPLATE_CONTENT:$template_content}' + else + printf '{"TEMPLATE_NAME":"%s","TEMPLATE_CONTENT":"%s"}\n' \ + "$(json_escape "$TEMPLATE_NAME")" "$(json_escape "$TEMPLATE_CONTENT")" + fi +else + printf '%s' "$TEMPLATE_CONTENT" +fi diff --git a/scripts/bash/setup-plan.sh b/scripts/bash/setup-plan.sh index e01dc44bce..03eaf713b0 100644 --- a/scripts/bash/setup-plan.sh +++ b/scripts/bash/setup-plan.sh @@ -43,21 +43,23 @@ if [[ -f "$IMPL_PLAN" ]]; then echo "Plan already exists at $IMPL_PLAN, skipping template copy" fi else - TEMPLATE=$(resolve_template "plan-template" "$REPO_ROOT") || true - if [[ -n "$TEMPLATE" ]] && [[ -f "$TEMPLATE" ]]; then - cp "$TEMPLATE" "$IMPL_PLAN" + if resolve_template_content "plan-template" "$REPO_ROOT" > "$IMPL_PLAN"; then if $JSON_MODE; then echo "Copied plan template to $IMPL_PLAN" >&2 else echo "Copied plan template to $IMPL_PLAN" fi else + resolve_status=$? + rm -f "$IMPL_PLAN" + if [ "$resolve_status" -ne 1 ]; then + exit "$resolve_status" + fi if $JSON_MODE; then echo "Warning: Plan template not found" >&2 else echo "Warning: Plan template not found" fi - # Create a basic plan file if template doesn't exist touch "$IMPL_PLAN" fi fi diff --git a/scripts/bash/setup-tasks.sh b/scripts/bash/setup-tasks.sh index 8c989060ba..a5a685cd0e 100644 --- a/scripts/bash/setup-tasks.sh +++ b/scripts/bash/setup-tasks.sh @@ -51,7 +51,9 @@ fi # Resolve tasks template through override stack TASKS_TEMPLATE=$(resolve_template "tasks-template" "$REPO_ROOT") || true -if [[ -z "$TASKS_TEMPLATE" ]] || [[ ! -f "$TASKS_TEMPLATE" ]]; then +if TASKS_TEMPLATE_CONTENT=$(resolve_template_content "tasks-template" "$REPO_ROOT"; status=$?; printf x; exit "$status"); then + TASKS_TEMPLATE_CONTENT="${TASKS_TEMPLATE_CONTENT%x}" +else echo "ERROR: Could not resolve required tasks-template from the template override stack for $REPO_ROOT" >&2 echo "Template 'tasks-template' was not found in any supported location (overrides, presets, extensions, or shared core). Add an override at .specify/templates/overrides/tasks-template.md, or run 'specify init' / reinstall shared infra to restore the core .specify/templates/tasks-template.md template." >&2 exit 1 @@ -69,7 +71,8 @@ if $JSON_MODE; then --arg feature_dir "$FEATURE_DIR" \ --argjson docs "$json_docs" \ --arg tasks_template "${TASKS_TEMPLATE:-}" \ - '{FEATURE_DIR:$feature_dir,AVAILABLE_DOCS:$docs,TASKS_TEMPLATE:$tasks_template}' + --arg tasks_template_content "$TASKS_TEMPLATE_CONTENT" \ + '{FEATURE_DIR:$feature_dir,AVAILABLE_DOCS:$docs,TASKS_TEMPLATE:$tasks_template,TASKS_TEMPLATE_CONTENT:$tasks_template_content}' else if [[ ${#docs[@]} -eq 0 ]]; then json_docs="[]" @@ -77,8 +80,8 @@ if $JSON_MODE; then json_docs=$(for d in "${docs[@]}"; do printf '"%s",' "$(json_escape "$d")"; done) json_docs="[${json_docs%,}]" fi - printf '{"FEATURE_DIR":"%s","AVAILABLE_DOCS":%s,"TASKS_TEMPLATE":"%s"}\n' \ - "$(json_escape "$FEATURE_DIR")" "$json_docs" "$(json_escape "${TASKS_TEMPLATE:-}")" + printf '{"FEATURE_DIR":"%s","AVAILABLE_DOCS":%s,"TASKS_TEMPLATE":"%s","TASKS_TEMPLATE_CONTENT":"%s"}\n' \ + "$(json_escape "$FEATURE_DIR")" "$json_docs" "$(json_escape "${TASKS_TEMPLATE:-}")" "$(json_escape "$TASKS_TEMPLATE_CONTENT")" fi else echo "FEATURE_DIR: $FEATURE_DIR" diff --git a/scripts/powershell/check-prerequisites.ps1 b/scripts/powershell/check-prerequisites.ps1 index 07ece76e21..c547d5f8c8 100644 --- a/scripts/powershell/check-prerequisites.ps1 +++ b/scripts/powershell/check-prerequisites.ps1 @@ -12,6 +12,7 @@ # -RequireTasks Require tasks.md to exist (for implementation phase) # -IncludeTasks Include tasks.md in AVAILABLE_DOCS list # -PathsOnly Only output path variables (no validation) +# -Template NAME Include composed template content in JSON output # -Help, -h Show help message [CmdletBinding()] @@ -20,6 +21,7 @@ param( [switch]$RequireTasks, [switch]$IncludeTasks, [switch]$PathsOnly, + [string]$Template, [switch]$Help ) @@ -37,6 +39,7 @@ OPTIONS: -RequireTasks Require tasks.md to exist (for implementation phase) -IncludeTasks Include tasks.md in AVAILABLE_DOCS list -PathsOnly Only output path variables (no prerequisite validation) + -Template NAME Include composed template content in JSON output -Help, -h Show this help message EXAMPLES: @@ -129,13 +132,26 @@ if ($IncludeTasks -and (Test-Path $paths.TASKS)) { $docs += 'tasks.md' } +$templateContent = $null +if ($Template) { + $templateContent = Resolve-TemplateContent -TemplateName $Template -RepoRoot $paths.REPO_ROOT + if ($null -eq $templateContent) { + [Console]::Error.WriteLine("ERROR: Could not resolve required $Template from the template override stack for $($paths.REPO_ROOT)") + exit 1 + } +} + # Output results if ($Json) { # JSON output - [PSCustomObject]@{ + $result = [ordered]@{ FEATURE_DIR = $paths.FEATURE_DIR AVAILABLE_DOCS = $docs - } | ConvertTo-Json -Compress + } + if ($Template) { + $result.TEMPLATE_CONTENT = $templateContent + } + [PSCustomObject]$result | ConvertTo-Json -Compress } else { # Text output Write-Output "FEATURE_DIR:$($paths.FEATURE_DIR)" diff --git a/scripts/powershell/common.ps1 b/scripts/powershell/common.ps1 index 7922e94032..585e884702 100644 --- a/scripts/powershell/common.ps1 +++ b/scripts/powershell/common.ps1 @@ -332,6 +332,82 @@ function Get-Python3Command { return $null } +function Get-NormalizedPriority { + param($Value) + + if ($Value -is [bool]) { return 10 } + if ($Value -is [string]) { + $integerText = $Value.Trim() + if ($integerText -cnotmatch '^[+-]?[0-9]+(?:_[0-9]+)*$') { return 10 } + $Value = $integerText.Replace('_', '') + } + try { + $parsedPriority = [System.Numerics.BigInteger]$Value + } catch { + return 10 + } + return $(if ($parsedPriority -ge 1) { $parsedPriority } else { 10 }) +} + +function Get-SortedExtensionIds { + param([Parameter(Mandatory=$true)][string]$ExtensionsDir) + + $registeredNames = @() + $ranked = @() + $registryFile = Join-Path $ExtensionsDir '.registry' + # Detect any filesystem entry at the registry path without following symlinks. + # Test-Path follows links and reports $false for a dangling symlink, so a + # broken .registry symlink would otherwise bypass this guard and let the + # directory scan below enable every on-disk extension. Enumerating the parent + # directory still observes a broken symlink as an entry. + $registryEntry = Get-ChildItem -LiteralPath $ExtensionsDir -Force -ErrorAction SilentlyContinue | + Where-Object { $_.Name -eq '.registry' } | + Select-Object -First 1 + if ($registryEntry) { + if (-not (Test-Path -LiteralPath $registryFile -PathType Leaf)) { + throw "Invalid extension registry ${registryFile}: not a regular file" + } + try { + $data = [System.IO.File]::ReadAllText($registryFile, [System.Text.Encoding]::UTF8) | ConvertFrom-Json + } catch { + throw "Invalid extension registry ${registryFile}: $($_.Exception.Message)" + } + if ($null -eq $data -or $data -isnot [PSCustomObject]) { + throw "Invalid extension registry ${registryFile}: root must be a mapping" + } + $extensionsProperty = $data.PSObject.Properties['extensions'] + if ($extensionsProperty) { + if ($extensionsProperty.Value -isnot [PSCustomObject]) { + throw "Invalid extension registry ${registryFile}: 'extensions' must be a mapping" + } + $extensions = $extensionsProperty.Value + } else { + $extensions = [PSCustomObject]@{} + } + $registeredNames = @($extensions.PSObject.Properties | ForEach-Object { $_.Name }) + foreach ($entry in $extensions.PSObject.Properties) { + if ($entry.Name -cnotmatch '^[a-z0-9-]+$' -or $entry.Value -isnot [PSCustomObject]) { + continue + } + $enabledProperty = $entry.Value.PSObject.Properties['enabled'] + if ($enabledProperty -and -not [bool]$enabledProperty.Value) { continue } + $priority = 10 + $priorityProperty = $entry.Value.PSObject.Properties['priority'] + if ($priorityProperty) { + $priority = Get-NormalizedPriority -Value $priorityProperty.Value + } + $ranked += [PSCustomObject]@{ Priority = $priority; Id = $entry.Name } + } + } + + foreach ($directory in Get-ChildItem -Path $ExtensionsDir -Directory -ErrorAction SilentlyContinue) { + if ($directory.Name -cmatch '^[a-z0-9-]+$' -and $directory.Name -cnotin $registeredNames) { + $ranked += [PSCustomObject]@{ Priority = 10; Id = $directory.Name } + } + } + return $ranked | Sort-Object Priority, Id | ForEach-Object { $_.Id } +} + # Resolve a template name to a file path using the priority stack: # 1. .specify/templates/overrides/ # 2. .specify/presets//templates/ (sorted by priority from .registry) @@ -343,6 +419,8 @@ function Resolve-Template { [Parameter(Mandatory=$true)][string]$RepoRoot ) + if ($TemplateName -cnotmatch '^[a-z0-9-]+$') { return $null } + $base = Join-Path $RepoRoot '.specify/templates' # Priority 1: Project overrides @@ -357,7 +435,7 @@ function Resolve-Template { $registryParsed = $false if (Test-Path $registryFile) { try { - $registryData = Get-Content $registryFile -Raw | ConvertFrom-Json + $registryData = [System.IO.File]::ReadAllText($registryFile, [System.Text.Encoding]::UTF8) | ConvertFrom-Json if ($null -eq $registryData -or $registryData -isnot [PSCustomObject]) { throw 'Registry root must be an object' } @@ -372,30 +450,20 @@ function Resolve-Template { param($Entry) if ($Entry.Value -is [PSCustomObject]) { $priorityProperty = $Entry.Value.PSObject.Properties['priority'] - if ($priorityProperty) { return $priorityProperty.Value } - } - return 10 - } - if ($presetEntries.Count -gt 1) { - $allNumeric = $true - $allStrings = $true - foreach ($entry in $presetEntries) { - $priority = & $priorityFor $entry - if ($null -eq $priority -or $priority -isnot [ValueType]) { - $allNumeric = $false - } - if ($null -eq $priority -or $priority -isnot [string]) { - $allStrings = $false + if ($priorityProperty) { + return Get-NormalizedPriority -Value $priorityProperty.Value } } - if (-not $allNumeric -and -not $allStrings) { - throw 'Registry priorities are not mutually orderable' - } + return 10 } $sortedPresets = $presetEntries | Where-Object { $_.Value -is [PSCustomObject] } | - Where-Object { $null -eq $_.Value.enabled -or $_.Value.enabled -ne $false } | - Sort-Object { & $priorityFor $_ } | + Where-Object { + $enabled = $_.Value.PSObject.Properties['enabled'] + -not $enabled -or [bool]$enabled.Value + } | + Where-Object { $_.Name -cmatch '^[a-z0-9-]+$' } | + Sort-Object @{ Expression = { & $priorityFor $_ } }, @{ Expression = { $_.Name } } | ForEach-Object { $_.Name } } $registryParsed = $true @@ -408,12 +476,16 @@ function Resolve-Template { foreach ($presetId in $sortedPresets) { $candidate = Join-Path $presetsDir "$presetId/templates/$TemplateName.md" if (Test-Path $candidate) { return $candidate } + $candidate = Join-Path $presetsDir "$presetId/$TemplateName.md" + if (Test-Path $candidate) { return $candidate } } } else { # Fallback: alphabetical directory order foreach ($preset in Get-ChildItem -Path $presetsDir -Directory -ErrorAction SilentlyContinue | Where-Object { $_.Name -notlike '.*' } | Sort-Object Name) { $candidate = Join-Path $preset.FullName "templates/$TemplateName.md" if (Test-Path $candidate) { return $candidate } + $candidate = Join-Path $preset.FullName "$TemplateName.md" + if (Test-Path $candidate) { return $candidate } } } } @@ -421,8 +493,11 @@ function Resolve-Template { # Priority 3: Extension-provided templates $extDir = Join-Path $RepoRoot '.specify/extensions' if (Test-Path $extDir) { - foreach ($ext in Get-ChildItem -Path $extDir -Directory -ErrorAction SilentlyContinue | Where-Object { $_.Name -notlike '.*' } | Sort-Object Name) { - $candidate = Join-Path $ext.FullName "templates/$TemplateName.md" + foreach ($extensionId in Get-SortedExtensionIds -ExtensionsDir $extDir) { + $candidate = Join-Path $extDir "$extensionId/templates/$TemplateName.md" + if (-not (Test-Path $candidate)) { + $candidate = Join-Path $extDir "$extensionId/$TemplateName.md" + } if (Test-Path $candidate) { return $candidate } } } @@ -443,6 +518,10 @@ function Resolve-TemplateContent { [Parameter(Mandatory=$true)][string]$RepoRoot ) + if ($TemplateName -cnotmatch '^[a-z0-9-]+$') { + return $null + } + $base = Join-Path $RepoRoot '.specify/templates' # Collect all layers (highest priority first) @@ -452,49 +531,77 @@ function Resolve-TemplateContent { # Priority 1: Project overrides (always "replace") $override = Join-Path $base "overrides/$TemplateName.md" if (Test-Path $override) { - $layerPaths += $override - $layerStrategies += 'replace' + return [System.IO.File]::ReadAllText( + $override, + [System.Text.Encoding]::UTF8 + ) } + $effectiveBaseFound = $false + # Priority 2: Installed presets (sorted by priority from .registry) $presetsDir = Join-Path $RepoRoot '.specify/presets' if (Test-Path $presetsDir) { $registryFile = Join-Path $presetsDir '.registry' $sortedPresets = @() + $registryParsed = $false if (Test-Path $registryFile) { try { - $registryData = Get-Content $registryFile -Raw | ConvertFrom-Json - $presets = $registryData.presets - if ($presets) { - $sortedPresets = $presets.PSObject.Properties | - Where-Object { $null -eq $_.Value.enabled -or $_.Value.enabled -ne $false } | - Sort-Object { if ($null -ne $_.Value.priority) { $_.Value.priority } else { 10 } } | + $registryData = [System.IO.File]::ReadAllText($registryFile, [System.Text.Encoding]::UTF8) | ConvertFrom-Json + if ($null -eq $registryData -or $registryData -isnot [PSCustomObject]) { + throw 'Registry root must be an object' + } + $presetsProperty = $registryData.PSObject.Properties['presets'] + if ($presetsProperty) { + $presets = $presetsProperty.Value + if ($null -eq $presets -or $presets -isnot [PSCustomObject]) { + throw 'Registry presets must be an object' + } + $presetEntries = @($presets.PSObject.Properties) + $priorityFor = { + param($Entry) + if ($Entry.Value -is [PSCustomObject]) { + $priorityProperty = $Entry.Value.PSObject.Properties['priority'] + if ($priorityProperty) { + return Get-NormalizedPriority -Value $priorityProperty.Value + } + } + return 10 + } + $sortedPresets = $presetEntries | + Where-Object { $_.Value -is [PSCustomObject] } | + Where-Object { + $enabled = $_.Value.PSObject.Properties['enabled'] + -not $enabled -or [bool]$enabled.Value + } | + Where-Object { $_.Name -cmatch '^[a-z0-9-]+$' } | + Sort-Object @{ Expression = { & $priorityFor $_ } }, @{ Expression = { $_.Name } } | ForEach-Object { $_.Name } } + $registryParsed = $true } catch { - $sortedPresets = @() + $registryParsed = $false } } - if ($sortedPresets.Count -gt 0) { - $pyCmd = Get-Python3Command - if (-not $pyCmd) { - # Check if any preset has strategy fields that would be ignored - foreach ($pid in $sortedPresets) { - $mf = Join-Path $presetsDir "$pid/preset.yml" - if ((Test-Path $mf) -and (Select-String -Path $mf -Pattern 'strategy:' -Quiet -ErrorAction SilentlyContinue)) { - Write-Warning "No Python 3 found; preset composition strategies will be ignored" - break - } - } - } - $yamlWarned = $false - foreach ($presetId in $sortedPresets) { + if (-not $registryParsed) { + $sortedPresets = Get-ChildItem -Path $presetsDir -Directory -ErrorAction SilentlyContinue | + Where-Object { $_.Name -cmatch '^[a-z0-9-]+$' } | + Sort-Object Name | + ForEach-Object { $_.Name } + } + + $pyCmd = @(Get-Python3Command) + foreach ($presetId in $sortedPresets) { # Read strategy and file path from preset manifest $strategy = 'replace' $manifestFilePath = '' + $manifestDeclared = $false $manifest = Join-Path $presetsDir "$presetId/preset.yml" - if ((Test-Path $manifest) -and $pyCmd) { + if ((Test-Path $manifest) -and -not $pyCmd) { + throw "Python 3 and PyYAML are required to resolve preset template composition" + } + if (Test-Path $manifest) { try { # Use Python to parse YAML manifest for strategy and file path $pyArgs = if ($pyCmd.Count -gt 1) { $pyCmd[1..($pyCmd.Count-1)] } else { @() } @@ -505,32 +612,71 @@ try: import yaml except ImportError: print('yaml_missing', file=sys.stderr) - print('replace\t') - sys.exit(0) + sys.exit(2) try: - with open(sys.argv[1]) as f: + with open(sys.argv[1], encoding='utf-8') as f: data = yaml.safe_load(f) - for t in data.get('provides', {}).get('templates', []): + if not isinstance(data, dict): + raise ValueError('manifest root must be a mapping') + if 'provides' not in data: + raise ValueError('manifest missing provides section') + provides = data['provides'] + if not isinstance(provides, dict): + raise ValueError('manifest provides must be a mapping') + if 'templates' not in provides: + raise ValueError('manifest provides missing templates') + templates = provides['templates'] + if not isinstance(templates, list): + raise ValueError('manifest templates must be a list') + if not templates: + raise ValueError('manifest must provide at least one template') + valid_types = ('template', 'command', 'script') + valid_strategies = ('replace', 'prepend', 'append', 'wrap') + for t in templates: + if not isinstance(t, dict): + raise ValueError('manifest template entries must be mappings') + if 'type' not in t or 'name' not in t or 'file' not in t: + raise ValueError('manifest template entry missing type, name, or file') + for field in ('type', 'name', 'file'): + if not isinstance(t[field], str): + raise ValueError('manifest template ' + field + ' must be a string') + if t['type'] not in valid_types: + raise ValueError('invalid manifest template type') + strategy = t.get('strategy', 'replace') + if not isinstance(strategy, str): + raise ValueError('manifest template strategy must be a string') + strategy = strategy.lower() + if strategy not in valid_strategies: + raise ValueError('invalid manifest template strategy') + if t['type'] == 'script' and strategy not in ('replace', 'wrap'): + raise ValueError('invalid manifest script strategy') + for t in templates: if t.get('name') == sys.argv[2] and t.get('type', 'template') == 'template': - print(t.get('strategy', 'replace') + '\t' + t.get('file', '')) + file_value = t.get('file', '') + strategy = t.get('strategy', 'replace') + print('found\t' + strategy + '\t' + file_value) sys.exit(0) - print('replace\t') -except Exception: - print('replace\t') + print('absent\treplace\t') +except Exception as exc: + print(f'manifest_invalid: {exc}', file=sys.stderr) + sys.exit(3) "@ $manifest $TemplateName 2>$pyStderrFile + if ($LASTEXITCODE -ne 0) { + if ($LASTEXITCODE -eq 2) { + throw "PyYAML is required to resolve preset template composition" + } + throw "Invalid preset manifest $manifest" + } if ($stratResult) { - $parts = $stratResult.Trim() -split "`t", 2 - $strategy = $parts[0].ToLowerInvariant() - if ($parts.Count -gt 1 -and $parts[1]) { $manifestFilePath = $parts[1] } - } - if (-not $yamlWarned -and (Test-Path $pyStderrFile) -and (Get-Content $pyStderrFile -Raw -ErrorAction SilentlyContinue) -match 'yaml_missing') { - Write-Warning "PyYAML not available; composition strategies may be ignored" - $yamlWarned = $true + $parts = $stratResult.Trim() -split "`t", 3 + $manifestDeclared = $parts[0] -eq 'found' + $strategy = $parts[1].ToLowerInvariant() + if ($parts.Count -gt 2 -and $parts[2]) { $manifestFilePath = $parts[2] } } Remove-Item $pyStderrFile -Force -ErrorAction SilentlyContinue } catch { - $strategy = 'replace' if ($pyStderrFile) { Remove-Item $pyStderrFile -Force -ErrorAction SilentlyContinue } + throw } } # Try manifest file path first, then convention path @@ -545,42 +691,45 @@ except Exception: $mf = Join-Path $presetsDir "$presetId/$manifestFilePath" if (Test-Path $mf) { $candidate = $mf } } - if (-not $candidate) { + if (-not $candidate -and -not $manifestDeclared) { $cf = Join-Path $presetsDir "$presetId/templates/$TemplateName.md" if (Test-Path $cf) { $candidate = $cf } + if (-not $candidate) { + $cf = Join-Path $presetsDir "$presetId/$TemplateName.md" + if (Test-Path $cf) { $candidate = $cf } + } } if ($candidate) { $layerPaths += $candidate $layerStrategies += $strategy + if ($strategy -eq 'replace') { + $effectiveBaseFound = $true + break + } } } - } else { - # Fallback: alphabetical directory order (no registry or parse failure) - foreach ($preset in Get-ChildItem -Path $presetsDir -Directory -ErrorAction SilentlyContinue | Where-Object { $_.Name -notlike '.*' }) { - $candidate = Join-Path $preset.FullName "templates/$TemplateName.md" - if (Test-Path $candidate) { - $layerPaths += $candidate - $layerStrategies += 'replace' - } - } - } } # Priority 3: Extension-provided templates (always "replace") $extDir = Join-Path $RepoRoot '.specify/extensions' - if (Test-Path $extDir) { - foreach ($ext in Get-ChildItem -Path $extDir -Directory -ErrorAction SilentlyContinue | Where-Object { $_.Name -notlike '.*' } | Sort-Object Name) { - $candidate = Join-Path $ext.FullName "templates/$TemplateName.md" + if (-not $effectiveBaseFound -and (Test-Path $extDir)) { + foreach ($extensionId in Get-SortedExtensionIds -ExtensionsDir $extDir) { + $candidate = Join-Path $extDir "$extensionId/templates/$TemplateName.md" + if (-not (Test-Path $candidate)) { + $candidate = Join-Path $extDir "$extensionId/$TemplateName.md" + } if (Test-Path $candidate) { $layerPaths += $candidate $layerStrategies += 'replace' + $effectiveBaseFound = $true + break } } } # Priority 4: Core templates (always "replace") $core = Join-Path $base "$TemplateName.md" - if (Test-Path $core) { + if (-not $effectiveBaseFound -and (Test-Path $core)) { $layerPaths += $core $layerStrategies += 'replace' } @@ -590,7 +739,7 @@ except Exception: # If the top (highest-priority) layer is replace, it wins entirely -- # lower layers are irrelevant regardless of their strategies. if ($layerStrategies[0] -eq 'replace') { - return (Get-Content $layerPaths[0] -Raw) + return [System.IO.File]::ReadAllText($layerPaths[0], [System.Text.Encoding]::UTF8) } # Check if any layer uses a non-replace strategy @@ -600,7 +749,7 @@ except Exception: } if (-not $hasComposition) { - return (Get-Content $layerPaths[0] -Raw) + return [System.IO.File]::ReadAllText($layerPaths[0], [System.Text.Encoding]::UTF8) } # Find the effective base: scan from highest priority (index 0) downward @@ -612,14 +761,22 @@ except Exception: break } } - if ($baseIdx -lt 0) { return $null } + if ($baseIdx -lt 0) { + throw "Template '$TemplateName' has composing layers but no replace base" + } - $content = Get-Content $layerPaths[$baseIdx] -Raw + $content = [System.IO.File]::ReadAllText( + $layerPaths[$baseIdx], + [System.Text.Encoding]::UTF8 + ) for ($i = $baseIdx - 1; $i -ge 0; $i--) { $path = $layerPaths[$i] $strat = $layerStrategies[$i] - $layerContent = Get-Content $path -Raw + $layerContent = [System.IO.File]::ReadAllText( + $path, + [System.Text.Encoding]::UTF8 + ) switch ($strat) { 'replace' { $content = $layerContent } diff --git a/scripts/powershell/create-new-feature.ps1 b/scripts/powershell/create-new-feature.ps1 index abe70f65ed..e7a68c4076 100644 --- a/scripts/powershell/create-new-feature.ps1 +++ b/scripts/powershell/create-new-feature.ps1 @@ -262,13 +262,16 @@ if (-not $DryRun) { exit 1 } + $needsSpec = -not (Test-Path -PathType Leaf $specFile) + $content = $null + if ($needsSpec) { + $content = Resolve-TemplateContent -TemplateName 'spec-template' -RepoRoot $repoRoot + } + New-Item -ItemType Directory -Path $featureDir -Force | Out-Null - if (-not (Test-Path -PathType Leaf $specFile)) { - $template = Resolve-Template -TemplateName 'spec-template' -RepoRoot $repoRoot - if ($template -and (Test-Path $template)) { - # Read the template content and write it to the spec file with UTF-8 encoding without BOM - $content = [System.IO.File]::ReadAllText($template) + if ($needsSpec) { + if ($null -ne $content) { $utf8NoBom = New-Object System.Text.UTF8Encoding($false) [System.IO.File]::WriteAllText($specFile, $content, $utf8NoBom) } else { diff --git a/scripts/powershell/resolve-template.ps1 b/scripts/powershell/resolve-template.ps1 new file mode 100644 index 0000000000..70aee0aca0 --- /dev/null +++ b/scripts/powershell/resolve-template.ps1 @@ -0,0 +1,38 @@ +#!/usr/bin/env pwsh + +param( + [Parameter(Position=0)] + [string]$TemplateName, + [switch]$Json, + [switch]$Help +) + +$ErrorActionPreference = 'Stop' + +if ($Help) { + Write-Output "Usage: resolve-template.ps1 [-Json]" + exit 0 +} + +if (-not $TemplateName) { + [Console]::Error.WriteLine("ERROR: Template name is required") + exit 1 +} + +. "$PSScriptRoot/common.ps1" + +$repoRoot = Get-RepoRoot +$templateContent = Resolve-TemplateContent -TemplateName $TemplateName -RepoRoot $repoRoot +if ($null -eq $templateContent) { + [Console]::Error.WriteLine("ERROR: Could not resolve required $TemplateName from the template override stack for $repoRoot") + exit 1 +} + +if ($Json) { + [PSCustomObject]@{ + TEMPLATE_NAME = $TemplateName + TEMPLATE_CONTENT = $templateContent + } | ConvertTo-Json -Compress +} else { + [Console]::Out.Write($templateContent) +} diff --git a/scripts/powershell/setup-plan.ps1 b/scripts/powershell/setup-plan.ps1 index 6ed0344dd9..52f615aaad 100644 --- a/scripts/powershell/setup-plan.ps1 +++ b/scripts/powershell/setup-plan.ps1 @@ -41,10 +41,8 @@ if (Test-Path $paths.IMPL_PLAN -PathType Leaf) { Write-Output "Plan already exists at $($paths.IMPL_PLAN), skipping template copy" } } else { - $template = Resolve-Template -TemplateName 'plan-template' -RepoRoot $paths.REPO_ROOT - if ($template -and (Test-Path $template)) { - # Read the template content and write it to the implementation plan file with UTF-8 encoding without BOM - $content = [System.IO.File]::ReadAllText($template) + $content = Resolve-TemplateContent -TemplateName 'plan-template' -RepoRoot $paths.REPO_ROOT + if ($null -ne $content) { $utf8NoBom = New-Object System.Text.UTF8Encoding($false) [System.IO.File]::WriteAllText($paths.IMPL_PLAN, $content, $utf8NoBom) # Emit the copy status like the bash twin (setup-plan.sh); route to stderr diff --git a/scripts/powershell/setup-tasks.ps1 b/scripts/powershell/setup-tasks.ps1 index 1d091360e7..828ff4a5b3 100644 --- a/scripts/powershell/setup-tasks.ps1 +++ b/scripts/powershell/setup-tasks.ps1 @@ -57,12 +57,17 @@ if (Test-Path $paths.QUICKSTART) { $docs += 'quickstart.md' } # Resolve tasks template through override stack $tasksTemplate = Resolve-Template -TemplateName 'tasks-template' -RepoRoot $paths.REPO_ROOT -if (-not $tasksTemplate -or -not (Test-Path -LiteralPath $tasksTemplate -PathType Leaf)) { +$tasksTemplateContent = Resolve-TemplateContent -TemplateName 'tasks-template' -RepoRoot $paths.REPO_ROOT +if ($null -eq $tasksTemplateContent) { [Console]::Error.WriteLine("ERROR: Could not resolve required tasks-template from the template override stack for $($paths.REPO_ROOT)") [Console]::Error.WriteLine("Template 'tasks-template' was not found in any supported location (overrides, presets, extensions, or shared core). Add an override at .specify/templates/overrides/tasks-template.md, or run 'specify init' / reinstall shared infra to restore the core .specify/templates/tasks-template.md template.") exit 1 } -$tasksTemplate = (Resolve-Path -LiteralPath $tasksTemplate).Path +if ($tasksTemplate -and (Test-Path -LiteralPath $tasksTemplate -PathType Leaf)) { + $tasksTemplate = (Resolve-Path -LiteralPath $tasksTemplate).Path +} else { + $tasksTemplate = '' +} # Output results if ($Json) { @@ -70,6 +75,7 @@ if ($Json) { FEATURE_DIR = $paths.FEATURE_DIR AVAILABLE_DOCS = $docs TASKS_TEMPLATE = $tasksTemplate + TASKS_TEMPLATE_CONTENT = $tasksTemplateContent } | ConvertTo-Json -Compress } else { Write-Output "FEATURE_DIR: $($paths.FEATURE_DIR)" diff --git a/scripts/python/check_prerequisites.py b/scripts/python/check_prerequisites.py index 50c31cb513..724d04194b 100644 --- a/scripts/python/check_prerequisites.py +++ b/scripts/python/check_prerequisites.py @@ -9,10 +9,22 @@ from pathlib import Path try: - from common import FeaturePaths, format_speckit_command, get_feature_paths + from common import ( + FeaturePaths, + TemplateResolutionError, + format_speckit_command, + get_feature_paths, + resolve_template_content, + ) except ImportError: # pragma: no cover - direct execution from unusual cwd sys.path.insert(0, str(Path(__file__).resolve().parent)) - from common import FeaturePaths, format_speckit_command, get_feature_paths + from common import ( + FeaturePaths, + TemplateResolutionError, + format_speckit_command, + get_feature_paths, + resolve_template_content, + ) def _json_line(payload: object) -> str: @@ -28,6 +40,7 @@ def _json_line(payload: object) -> str: --require-tasks Require tasks.md to exist (for implementation phase) --include-tasks Include tasks.md in AVAILABLE_DOCS list --paths-only Only output path variables (no prerequisite validation) + --template NAME Include composed template content in JSON output --help, -h Show this help message EXAMPLES: @@ -49,6 +62,7 @@ class Args: require_tasks: bool = False include_tasks: bool = False paths_only: bool = False + template_name: str | None = None def _parse_args(argv: list[str]) -> Args: @@ -56,8 +70,11 @@ def _parse_args(argv: list[str]) -> Args: require_tasks = False include_tasks = False paths_only = False + template_name = None - for arg in argv: + index = 0 + while index < len(argv): + arg = argv[index] if arg == "--json": json_mode = True elif arg == "--require-tasks": @@ -66,6 +83,15 @@ def _parse_args(argv: list[str]) -> Args: include_tasks = True elif arg == "--paths-only": paths_only = True + elif arg == "--template": + index += 1 + if index >= len(argv): + print( + "ERROR: --template requires a template name", + file=sys.stderr, + ) + raise SystemExit(1) + template_name = argv[index] elif arg in {"--help", "-h"}: sys.stdout.write(HELP_TEXT) raise SystemExit(0) @@ -75,12 +101,14 @@ def _parse_args(argv: list[str]) -> Args: file=sys.stderr, ) raise SystemExit(1) + index += 1 return Args( json_mode=json_mode, require_tasks=require_tasks, include_tasks=include_tasks, paths_only=paths_only, + template_name=template_name, ) @@ -194,9 +222,32 @@ def main(argv: list[str] | None = None) -> int: return 1 docs = _available_docs(paths, args.include_tasks) + template_content = None + if args.template_name: + try: + template_content = resolve_template_content( + args.template_name, paths.repo_root + ) + except TemplateResolutionError as exc: + print(f"ERROR: {exc}", file=sys.stderr) + return 1 + if template_content is None: + print( + f"ERROR: Could not resolve required {args.template_name} from " + f"the template override stack for {paths.repo_root}", + file=sys.stderr, + ) + return 1 + if args.json_mode: + payload: dict[str, object] = { + "FEATURE_DIR": str(paths.feature_dir), + "AVAILABLE_DOCS": docs, + } + if args.template_name: + payload["TEMPLATE_CONTENT"] = template_content sys.stdout.write( - _json_line({"FEATURE_DIR": str(paths.feature_dir), "AVAILABLE_DOCS": docs}) + _json_line(payload) ) else: _print_text_results(paths, args.include_tasks) diff --git a/scripts/python/common.py b/scripts/python/common.py index 72f61d3782..db958dc1cb 100644 --- a/scripts/python/common.py +++ b/scripts/python/common.py @@ -4,6 +4,7 @@ import json import os +import re import sys from dataclasses import dataclass from pathlib import Path @@ -182,12 +183,30 @@ def get_feature_paths( ) +_SAFE_COMPONENT_PATTERN = re.compile(r"[a-z0-9-]+") + + +def _is_safe_component(value: object) -> bool: + return ( + isinstance(value, str) + and _SAFE_COMPONENT_PATTERN.fullmatch(value) is not None + ) + + +def _normalize_priority(value: object) -> int: + if isinstance(value, bool): + return 10 + try: + priority = int(value) + except (TypeError, ValueError, OverflowError): + return 10 + return priority if priority >= 1 else 10 + + def _sorted_preset_ids(presets_dir: Path) -> list[str]: registry = presets_dir / ".registry" if registry.is_file(): - # Mirrors bash: any failure while reading or sorting the registry - # (invalid JSON, non-dict shapes, unorderable priority values) falls - # back to the directory scan below. + # Invalid JSON or registry shapes fall back to the directory scan below. try: data = json.loads(registry.read_text(encoding="utf-8")) presets = data.get("presets", {}) @@ -195,11 +214,18 @@ def _sorted_preset_ids(presets_dir: Path) -> list[str]: pid for pid, meta in sorted( presets.items(), - key=lambda kv: kv[1].get("priority", 10) - if isinstance(kv[1], dict) - else 10, + key=lambda kv: ( + _normalize_priority(kv[1].get("priority")) + if isinstance(kv[1], dict) + else 10, + kv[0], + ), + ) + if ( + _is_safe_component(pid) + and isinstance(meta, dict) + and bool(meta.get("enabled", True)) ) - if isinstance(meta, dict) and meta.get("enabled", True) is not False ] except Exception: pass @@ -207,12 +233,78 @@ def _sorted_preset_ids(presets_dir: Path) -> list[str]: return sorted( p.name for p in presets_dir.iterdir() - if p.is_dir() and not p.name.startswith(".") + if p.is_dir() and _is_safe_component(p.name) ) except OSError: return [] +def _sorted_extension_ids(extensions_dir: Path) -> list[str]: + registry = extensions_dir / ".registry" + registered_ids: set[str] = set() + extensions: dict[object, object] = {} + if os.path.lexists(registry): + if not registry.is_file(): + raise TemplateResolutionError( + f"Invalid extension registry {registry}: not a regular file" + ) + try: + data = json.loads(registry.read_text(encoding="utf-8")) + except (OSError, UnicodeError, json.JSONDecodeError) as exc: + raise TemplateResolutionError( + f"Failed to parse extension registry {registry}: {exc}" + ) from exc + if not isinstance(data, dict): + raise TemplateResolutionError( + f"Invalid extension registry {registry}: root must be a mapping" + ) + raw_extensions = data.get("extensions", {}) + if not isinstance(raw_extensions, dict): + raise TemplateResolutionError( + f"Invalid extension registry {registry}: " + "'extensions' must be a mapping" + ) + extensions = raw_extensions + registered_ids = { + ext_id for ext_id in extensions if isinstance(ext_id, str) + } + + ranked: list[tuple[int, str]] = [] + for ext_id, metadata in extensions.items(): + if ( + _is_safe_component(ext_id) + and isinstance(metadata, dict) + and bool(metadata.get("enabled", True)) + ): + ranked.append((_normalize_priority(metadata.get("priority")), ext_id)) + + try: + ranked.extend( + (10, path.name) + for path in extensions_dir.iterdir() + if ( + path.is_dir() + and _is_safe_component(path.name) + and path.name not in registered_ids + ) + ) + except OSError: + pass + return [ext_id for _, ext_id in sorted(ranked)] + + +def _conventional_template( + base_dir: Path, template_name: str +) -> Path | None: + for candidate in ( + base_dir / "templates" / f"{template_name}.md", + base_dir / f"{template_name}.md", + ): + if candidate.is_file(): + return candidate + return None + + def resolve_template(template_name: str, repo_root: Path) -> Path | None: """Resolve a template name to a file path using the priority stack. @@ -222,6 +314,9 @@ def resolve_template(template_name: str, repo_root: Path) -> Path | None: 3. .specify/extensions//templates/ (hidden directories skipped) 4. .specify/templates/ (core) """ + if not _is_safe_component(template_name): + return None + base = repo_root / ".specify" / "templates" override = base / "overrides" / f"{template_name}.md" @@ -231,21 +326,18 @@ def resolve_template(template_name: str, repo_root: Path) -> Path | None: presets_dir = repo_root / ".specify" / "presets" if presets_dir.is_dir(): for preset_id in _sorted_preset_ids(presets_dir): - candidate = presets_dir / preset_id / "templates" / f"{template_name}.md" - if candidate.is_file(): + candidate = _conventional_template( + presets_dir / preset_id, template_name + ) + if candidate is not None: return candidate ext_dir = repo_root / ".specify" / "extensions" if ext_dir.is_dir(): - try: - extensions = sorted(p for p in ext_dir.iterdir() if p.is_dir()) - except OSError: - extensions = [] - for ext in extensions: - if ext.name.startswith("."): - continue - candidate = ext / "templates" / f"{template_name}.md" - if candidate.is_file(): + for extension_id in _sorted_extension_ids(ext_dir): + ext = ext_dir / extension_id + candidate = _conventional_template(ext, template_name) + if candidate is not None: return candidate core = base / f"{template_name}.md" @@ -254,6 +346,175 @@ def resolve_template(template_name: str, repo_root: Path) -> Path | None: return None +class TemplateResolutionError(RuntimeError): + """Raised when template layers exist but cannot be composed safely.""" + + +# Mirror the canonical PresetManifest contract (see src/specify_cli/presets) +# so runtime resolution rejects the same structurally malformed manifests. +_VALID_TEMPLATE_TYPES = ("template", "command", "script") +_VALID_TEMPLATE_STRATEGIES = ("replace", "prepend", "append", "wrap") +_VALID_SCRIPT_STRATEGIES = ("replace", "wrap") + + +def _validate_manifest_template_entry(entry: object) -> None: + """Validate a single manifest template entry against the canonical rules.""" + if not isinstance(entry, dict): + raise ValueError("manifest template entries must be mappings") + if "type" not in entry or "name" not in entry or "file" not in entry: + raise ValueError("manifest template entry missing type, name, or file") + for field in ("type", "name", "file"): + if not isinstance(entry[field], str): + raise ValueError(f"manifest template {field} must be a string") + if entry["type"] not in _VALID_TEMPLATE_TYPES: + raise ValueError(f"invalid manifest template type '{entry['type']}'") + strategy = entry.get("strategy", "replace") + if not isinstance(strategy, str): + raise ValueError("manifest template strategy must be a string") + strategy = strategy.lower() + if strategy not in _VALID_TEMPLATE_STRATEGIES: + raise ValueError(f"invalid manifest template strategy '{strategy}'") + if entry["type"] == "script" and strategy not in _VALID_SCRIPT_STRATEGIES: + raise ValueError( + f"invalid manifest script strategy '{strategy}'" + ) + + +def _preset_template_layer( + preset_dir: Path, template_name: str +) -> tuple[Path, str] | None: + """Return the preset template path and composition strategy.""" + manifest_path = preset_dir / "preset.yml" + conventional = _conventional_template(preset_dir, template_name) + + try: + import yaml + except ImportError as exc: + if manifest_path.is_file(): + raise TemplateResolutionError( + "PyYAML is required to resolve preset template composition" + ) from exc + return (conventional, "replace") if conventional is not None else None + + if manifest_path.is_file(): + try: + manifest = yaml.safe_load(manifest_path.read_text(encoding="utf-8")) + if not isinstance(manifest, dict): + raise ValueError("manifest root must be a mapping") + if "provides" not in manifest: + raise ValueError("manifest missing provides section") + provides = manifest["provides"] + if not isinstance(provides, dict): + raise ValueError("manifest provides must be a mapping") + if "templates" not in provides: + raise ValueError("manifest provides missing templates") + templates = provides["templates"] + if not isinstance(templates, list): + raise ValueError("manifest templates must be a list") + if not templates: + raise ValueError("manifest must provide at least one template") + for entry in templates: + _validate_manifest_template_entry(entry) + for entry in templates: + if ( + entry.get("name") != template_name + or entry.get("type", "template") != "template" + ): + continue + file_value = entry.get("file", "") + strategy = entry.get("strategy", "replace") + relative = Path(file_value) + if ( + not relative + or relative.is_absolute() + or ".." in relative.parts + ): + return None + candidate = preset_dir / relative + if not candidate.is_file(): + return None + return candidate, strategy.lower() + except (OSError, UnicodeError, ValueError, yaml.YAMLError) as exc: + raise TemplateResolutionError( + f"Failed to parse preset manifest {manifest_path}: {exc}" + ) from exc + + return (conventional, "replace") if conventional is not None else None + + +def resolve_template_content(template_name: str, repo_root: Path) -> str | None: + """Resolve and compose template content through the project layer stack.""" + if not _is_safe_component(template_name): + return None + + layers: list[tuple[Path, str]] = [] + + def compose_from_base() -> str: + try: + content = layers[-1][0].read_bytes().decode("utf-8") + for path, strategy in reversed(layers[:-1]): + layer_content = path.read_bytes().decode("utf-8") + if strategy == "prepend": + content = f"{layer_content}\n\n{content}" + elif strategy == "append": + content = f"{content}\n\n{layer_content}" + elif strategy == "wrap": + placeholder = "{CORE_TEMPLATE}" + if placeholder not in layer_content: + raise TemplateResolutionError( + f"Wrap layer {path} is missing {placeholder}" + ) + content = layer_content.replace(placeholder, content) + else: + raise TemplateResolutionError( + f"Unknown template composition strategy '{strategy}' in {path}" + ) + except (OSError, UnicodeError) as exc: + raise TemplateResolutionError( + f"Failed to read template layer for '{template_name}': {exc}" + ) from exc + return content + + override = ( + repo_root + / ".specify" + / "templates" + / "overrides" + / f"{template_name}.md" + ) + if override.is_file(): + layers.append((override, "replace")) + return compose_from_base() + + presets_dir = repo_root / ".specify" / "presets" + for preset_id in _sorted_preset_ids(presets_dir): + layer = _preset_template_layer(presets_dir / preset_id, template_name) + if layer is not None: + layers.append(layer) + if layer[1] == "replace": + return compose_from_base() + + extensions_dir = repo_root / ".specify" / "extensions" + for extension_id in _sorted_extension_ids(extensions_dir): + extension_dir = extensions_dir / extension_id + candidate = _conventional_template(extension_dir, template_name) + if candidate is not None: + layers.append((candidate, "replace")) + return compose_from_base() + + core = repo_root / ".specify" / "templates" / f"{template_name}.md" + if core.is_file(): + layers.append((core, "replace")) + return compose_from_base() + + if not layers: + return None + + raise TemplateResolutionError( + f"Template '{template_name}' has composing layers but no replace base" + ) + + def get_invoke_separator(repo_root: Path) -> str: integration_json = repo_root / ".specify" / "integration.json" if not integration_json.is_file(): diff --git a/scripts/python/create_new_feature.py b/scripts/python/create_new_feature.py index c46837d9d5..f36064afbb 100644 --- a/scripts/python/create_new_feature.py +++ b/scripts/python/create_new_feature.py @@ -7,16 +7,25 @@ import json import re import shlex -import shutil import sys from dataclasses import dataclass from pathlib import Path try: - from common import get_repo_root, persist_feature_json, resolve_template + from common import ( + TemplateResolutionError, + get_repo_root, + persist_feature_json, + resolve_template_content, + ) except ImportError: # pragma: no cover - direct execution from unusual cwd sys.path.insert(0, str(Path(__file__).resolve().parent)) - from common import get_repo_root, persist_feature_json, resolve_template + from common import ( + TemplateResolutionError, + get_repo_root, + persist_feature_json, + resolve_template_content, + ) def _json_line(payload: object) -> str: @@ -374,12 +383,22 @@ def main(argv: list[str] | None = None) -> int: ) return 1 + template_content = None + needs_spec = not spec_file.is_file() + if needs_spec: + try: + template_content = resolve_template_content( + "spec-template", repo_root + ) + except TemplateResolutionError as exc: + print(f"Error: {exc}", file=sys.stderr) + return 1 + feature_dir.mkdir(parents=True, exist_ok=True) - if not spec_file.is_file(): - template = resolve_template("spec-template", repo_root) - if template is not None and template.is_file(): - shutil.copy(template, spec_file) + if needs_spec: + if template_content is not None: + spec_file.write_bytes(template_content.encode("utf-8")) else: print( "Warning: Spec template not found; created empty spec file", diff --git a/scripts/python/resolve_template.py b/scripts/python/resolve_template.py new file mode 100644 index 0000000000..d2a7da89b2 --- /dev/null +++ b/scripts/python/resolve_template.py @@ -0,0 +1,63 @@ +#!/usr/bin/env python3 +"""Resolve composed template content from the project template stack.""" + +from __future__ import annotations + +import argparse +import json +import sys +from pathlib import Path + +try: + from common import ( + TemplateResolutionError, + get_repo_root, + resolve_template_content, + ) +except ImportError: # pragma: no cover - direct execution from unusual cwd + sys.path.insert(0, str(Path(__file__).resolve().parent)) + from common import ( + TemplateResolutionError, + get_repo_root, + resolve_template_content, + ) + + +def main(argv: list[str] | None = None) -> int: + parser = argparse.ArgumentParser() + parser.add_argument("template_name") + parser.add_argument("--json", action="store_true") + args = parser.parse_args(argv) + + repo_root = get_repo_root(Path(__file__)) + try: + content = resolve_template_content(args.template_name, repo_root) + except TemplateResolutionError as exc: + print(f"ERROR: {exc}", file=sys.stderr) + return 1 + if content is None: + print( + f"ERROR: Could not resolve required {args.template_name} from the " + f"template override stack for {repo_root}", + file=sys.stderr, + ) + return 1 + + if args.json: + print( + json.dumps( + { + "TEMPLATE_NAME": args.template_name, + "TEMPLATE_CONTENT": content, + }, + ensure_ascii=False, + separators=(",", ":"), + ) + ) + else: + sys.stdout.write(content) + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/scripts/python/setup_plan.py b/scripts/python/setup_plan.py index 7b8e77ce5a..d25fdd7829 100644 --- a/scripts/python/setup_plan.py +++ b/scripts/python/setup_plan.py @@ -4,15 +4,22 @@ from __future__ import annotations import json -import shutil import sys from pathlib import Path try: - from common import get_feature_paths, resolve_template + from common import ( + TemplateResolutionError, + get_feature_paths, + resolve_template_content, + ) except ImportError: # pragma: no cover - direct execution from unusual cwd sys.path.insert(0, str(Path(__file__).resolve().parent)) - from common import get_feature_paths, resolve_template + from common import ( + TemplateResolutionError, + get_feature_paths, + resolve_template_content, + ) def _json_line(payload: object) -> str: @@ -55,9 +62,13 @@ def main(argv: list[str] | None = None) -> int: file=status_stream, ) else: - template = resolve_template("plan-template", paths.repo_root) - if template is not None and template.is_file(): - shutil.copy(template, paths.impl_plan) + try: + template_content = resolve_template_content("plan-template", paths.repo_root) + except TemplateResolutionError as exc: + print(f"Error: {exc}", file=sys.stderr) + return 1 + if template_content is not None: + paths.impl_plan.write_bytes(template_content.encode("utf-8")) print(f"Copied plan template to {paths.impl_plan}", file=status_stream) else: print("Warning: Plan template not found", file=status_stream) diff --git a/scripts/python/setup_tasks.py b/scripts/python/setup_tasks.py index b3abb6dc1a..957a5634cc 100644 --- a/scripts/python/setup_tasks.py +++ b/scripts/python/setup_tasks.py @@ -10,17 +10,21 @@ try: from common import ( FeaturePaths, + TemplateResolutionError, format_speckit_command, get_feature_paths, resolve_template, + resolve_template_content, ) except ImportError: # pragma: no cover - direct execution from unusual cwd sys.path.insert(0, str(Path(__file__).resolve().parent)) from common import ( FeaturePaths, + TemplateResolutionError, format_speckit_command, get_feature_paths, resolve_template, + resolve_template_content, ) @@ -103,8 +107,14 @@ def main(argv: list[str] | None = None) -> int: docs = _available_docs(paths) - tasks_template = resolve_template("tasks-template", paths.repo_root) - if tasks_template is None or not tasks_template.is_file(): + try: + tasks_template_content = resolve_template_content( + "tasks-template", paths.repo_root + ) + except TemplateResolutionError as exc: + print(f"ERROR: {exc}", file=sys.stderr) + return 1 + if tasks_template_content is None: print( "ERROR: Could not resolve required tasks-template from the template " f"override stack for {paths.repo_root}", @@ -121,18 +131,21 @@ def main(argv: list[str] | None = None) -> int: return 1 if json_mode: + tasks_template = resolve_template("tasks-template", paths.repo_root) sys.stdout.write( _json_line( { "FEATURE_DIR": str(paths.feature_dir), "AVAILABLE_DOCS": docs, - "TASKS_TEMPLATE": str(tasks_template), + "TASKS_TEMPLATE": str(tasks_template) if tasks_template else "", + "TASKS_TEMPLATE_CONTENT": tasks_template_content, } ) ) else: + tasks_template = resolve_template("tasks-template", paths.repo_root) print(f"FEATURE_DIR: {paths.feature_dir}") - print(f"TASKS_TEMPLATE: {tasks_template}") + print(f"TASKS_TEMPLATE: {tasks_template or 'not found'}") print("AVAILABLE_DOCS:") _check_file(paths.research, "research.md") _check_file(paths.data_model, "data-model.md") diff --git a/src/specify_cli/extensions/__init__.py b/src/specify_cli/extensions/__init__.py index 6d78354809..28529f845b 100644 --- a/src/specify_cli/extensions/__init__.py +++ b/src/specify_cli/extensions/__init__.py @@ -641,10 +641,39 @@ def _load(self) -> dict: if not isinstance(data.get("extensions"), dict): data["extensions"] = {} return data - except (json.JSONDecodeError, FileNotFoundError): - # Corrupted or missing registry, start fresh + except (json.JSONDecodeError, OSError, UnicodeDecodeError): + # Corrupted, unreadable, or missing registry, start fresh. Callers + # that must fail closed (resolution paths) consult is_corrupt() + # instead of relying on this recovery. return {"schema_version": self.SCHEMA_VERSION, "extensions": {}} + def is_corrupt(self) -> bool: + """Report whether an existing registry file is present but unreadable. + + ``_load`` deliberately recovers from a corrupt registry by normalizing + it to an empty mapping so install/enable/disable flows keep working. + Resolution paths, however, must fail closed: a corrupt registry that + normalizes to ``{}`` would otherwise cause every on-disk extension + directory to be admitted as an unregistered, enabled extension. This + probe lets those callers distinguish "no registry" (safe) from + "registry exists but is invalid" (unsafe) without changing recovery + behavior. An absent registry returns ``False``; a directory, broken + file, non-mapping root, or non-mapping ``extensions`` value returns + ``True``. + """ + if not self.registry_path.exists(): + return False + try: + with open(self.registry_path, "r", encoding="utf-8") as f: + data = json.load(f) + except (json.JSONDecodeError, OSError, UnicodeDecodeError): + return True + if not isinstance(data, dict): + return True + if "extensions" in data and not isinstance(data["extensions"], dict): + return True + return False + def _save(self): """Save registry to disk.""" self.extensions_dir.mkdir(parents=True, exist_ok=True) diff --git a/src/specify_cli/presets/__init__.py b/src/specify_cli/presets/__init__.py index cc5308f3fc..3ff3fd378c 100644 --- a/src/specify_cli/presets/__init__.py +++ b/src/specify_cli/presets/__init__.py @@ -56,6 +56,7 @@ _CONSTITUTION_PROVENANCE_FILE = ".constitution-template.json" +_CONSTITUTION_SYNC_PRESET_ID = "constitution-sync" def _content_sha256(content: bytes) -> str: @@ -3550,13 +3551,10 @@ def install_from_directory( stacklevel=2, ) - # Seed/re-seed memory/constitution.md from a preset-provided - # constitution-template. The constitution is the only template that is - # materialized to a live file rather than resolved on demand, so a - # preset that ships one (e.g. strategy: replace with a ratified - # constitution) must be propagated here. Guard against clobbering an - # already-authored constitution by only replacing a file whose recorded - # hash (or exact legacy core-template content) proves it was generated. + # Materialize constitution-template changes only for projects that opt + # into the constitution-sync preset. The core /constitution command + # resolves this template on demand; constitution-sync preserves the + # previous install-time behavior for teams that want reviewed snapshots. self._seed_constitution_from_preset(manifest, dest_dir) return manifest @@ -3564,14 +3562,13 @@ def install_from_directory( def _seed_constitution_from_preset( self, manifest: PresetManifest, preset_dir: Path ) -> None: - """Seed memory/constitution.md from a preset constitution-template. + """Seed memory/constitution.md when constitution-sync opts into snapshots. - Only runs when the preset declares a ``type: template`` entry named - ``constitution-template`` or provides one at a convention path, and the - live memory file is either missing or is an unchanged generated file. - Authored constitutions are never overwritten. + Installing constitution-sync itself materializes the currently resolved + stack. Later preset installs only reconcile when they provide a + ``constitution-template``. Authored constitutions are never overwritten. """ - provides_constitution = any( + provides_constitution = manifest.id == _CONSTITUTION_SYNC_PRESET_ID or any( t.get("type") == "template" and t.get("name") == "constitution-template" for t in manifest.templates ) or any( @@ -3592,7 +3589,7 @@ def _seed_constitution_from_preset( def reconcile_constitution( self, failure_context: str, *, create_if_missing: bool = False ) -> None: - """Reconcile generated constitution content without failing a persisted change.""" + """Reconcile an opted-in generated constitution without failing a change.""" try: self._reconcile_constitution(create_if_missing=create_if_missing) except (OSError, UnicodeDecodeError, PresetValidationError, ValueError) as exc: @@ -3604,7 +3601,11 @@ def reconcile_constitution( ) def _reconcile_constitution(self, *, create_if_missing: bool = False) -> None: - """Materialize the winning constitution layer when the live file is generated.""" + """Materialize the winning layer when constitution-sync is enabled.""" + sync_metadata = self.registry.get(_CONSTITUTION_SYNC_PRESET_ID) + if sync_metadata is None or not sync_metadata.get("enabled", True): + return + memory_constitution = ( self.project_root / ".specify" / "memory" / "constitution.md" ) @@ -4910,6 +4911,18 @@ def _get_manifest(self, pack_dir: Path) -> Optional["PresetManifest"]: self._manifest_cache[key] = None return self._manifest_cache[key] + @staticmethod + def _is_safe_registry_id(value: object) -> bool: + return isinstance(value, str) and re.fullmatch(r"[a-z0-9-]+", value) is not None + + def _get_all_presets_by_priority(self) -> List[tuple[str, dict]]: + registry = PresetRegistry(self.presets_dir) + return [ + (pack_id, metadata) + for pack_id, metadata in registry.list_by_priority() + if self._is_safe_registry_id(pack_id) + ] + def _manifest_declared_template( self, pack_dir: Path, template_name: str, template_type: str ) -> tuple[dict | None, Path | None]: @@ -4957,6 +4970,16 @@ def _get_all_extensions_by_priority(self) -> list[tuple[int, str, dict | None]]: return [] registry = ExtensionRegistry(self.extensions_dir) + # Fail closed on a corrupt registry. ExtensionRegistry._load() recovers + # by normalizing an unreadable registry to an empty mapping, which would + # otherwise cause the directory scan below to admit every on-disk + # directory as an unregistered, enabled extension — a fail-open path + # that could supply constitution content from an invalid registry state. + if registry.is_corrupt(): + raise PresetValidationError( + f"Invalid extension registry {registry.registry_path}: " + "refusing to enumerate extensions" + ) # Use keys() to track ALL extensions (including corrupted entries) without deep copy # This prevents corrupted entries from being picked up as "unregistered" dirs registered_extension_ids = registry.keys() @@ -4968,6 +4991,8 @@ def _get_all_extensions_by_priority(self) -> list[tuple[int, str, dict | None]]: # Only include enabled extensions in the result for ext_id, metadata in all_registered: + if not self._is_safe_registry_id(ext_id): + continue # Skip disabled extensions if not metadata.get("enabled", True): continue @@ -4976,7 +5001,7 @@ def _get_all_extensions_by_priority(self) -> list[tuple[int, str, dict | None]]: # Add unregistered directories with implicit priority=10 for ext_dir in self.extensions_dir.iterdir(): - if not ext_dir.is_dir() or ext_dir.name.startswith("."): + if not ext_dir.is_dir() or not self._is_safe_registry_id(ext_dir.name): continue if ext_dir.name not in registered_extension_ids: all_extensions.append((10, ext_dir.name, None)) @@ -5042,8 +5067,7 @@ def resolve( # Priority 2: Installed presets (sorted by priority — lower number wins) if not skip_presets and self.presets_dir.exists(): - registry = PresetRegistry(self.presets_dir) - for pack_id, _metadata in registry.list_by_priority(): + for pack_id, _metadata in self._get_all_presets_by_priority(): pack_dir = self.presets_dir / pack_id # The preset manifest is authoritative: if it declares this # template with an explicit ``file:``, resolve to that path — @@ -5236,13 +5260,11 @@ def resolve_with_source( return {"path": resolved_str, "source": "project override"} if str(self.presets_dir) in resolved_str and self.presets_dir.exists(): - registry = PresetRegistry(self.presets_dir) - for pack_id, _metadata in registry.list_by_priority(): + for pack_id, metadata in self._get_all_presets_by_priority(): pack_dir = self.presets_dir / pack_id try: resolved.relative_to(pack_dir) - meta = registry.get(pack_id) - version = meta.get("version", "?") if meta else "?" + version = metadata.get("version", "?") return { "path": resolved_str, "source": f"{pack_id} v{version}", @@ -5328,8 +5350,7 @@ def _find_in_subdirs(base_dir: Path) -> Optional[Path]: # Priority 2: Installed presets (sorted by priority — lower number = higher precedence) if self.presets_dir.exists(): - registry = PresetRegistry(self.presets_dir) - for pack_id, metadata in registry.list_by_priority(): + for pack_id, metadata in self._get_all_presets_by_priority(): pack_dir = self.presets_dir / pack_id # Read strategy and manifest file path from preset manifest strategy = "replace" diff --git a/src/specify_cli/presets/_commands.py b/src/specify_cli/presets/_commands.py index e601152766..145dc6e9df 100644 --- a/src/specify_cli/presets/_commands.py +++ b/src/specify_cli/presets/_commands.py @@ -9,6 +9,7 @@ from __future__ import annotations import os +import re from pathlib import Path import typer @@ -352,9 +353,26 @@ def preset_resolve( from .. import _require_specify_project from . import PresetResolver + is_command = "." in template_name + valid_name = ( + re.fullmatch(r"[a-z0-9-]+(?:\.[a-z0-9-]+)+", template_name) + if is_command + else re.fullmatch(r"[a-z0-9-]+", template_name) + ) + if valid_name is None: + typer.echo( + f"Error: invalid template name '{template_name}'; " + "use lowercase letters, digits, and hyphens, with non-empty " + "dot-separated segments for commands", + err=True, + ) + raise typer.Exit(1) + project_root = _require_specify_project() resolver = PresetResolver(project_root) - layers = resolver.collect_all_layers(template_name) + template_type = "command" if is_command else "template" + + layers = resolver.collect_all_layers(template_name, template_type) safe_template_name = _escape_markup(str(template_name)) if layers: @@ -377,7 +395,7 @@ def preset_resolve( if has_composition: # Verify composition is actually possible try: - composed = resolver.resolve_content(template_name) + composed = resolver.resolve_content(template_name, template_type) except Exception as exc: composed = None console.print( @@ -416,7 +434,7 @@ def preset_resolve( ) else: # No layers found — fall back to resolve_with_source for non-composition cases - result = resolver.resolve_with_source(template_name) + result = resolver.resolve_with_source(template_name, template_type) if result: console.print( f" [bold]{safe_template_name}[/bold]: " diff --git a/templates/commands/checklist.md b/templates/commands/checklist.md index 6a5c6d8745..b5bccfdc06 100644 --- a/templates/commands/checklist.md +++ b/templates/commands/checklist.md @@ -1,9 +1,9 @@ --- description: Generate a custom checklist for the current feature based on user requirements. scripts: - sh: scripts/bash/check-prerequisites.sh --json - ps: scripts/powershell/check-prerequisites.ps1 -Json - py: scripts/python/check_prerequisites.py --json + sh: scripts/bash/check-prerequisites.sh --json --template checklist-template + ps: scripts/powershell/check-prerequisites.ps1 -Json -Template checklist-template + py: scripts/python/check_prerequisites.py --json --template checklist-template --- ## Checklist Purpose: "Unit Tests for English" @@ -72,7 +72,7 @@ You **MUST** consider the user input before proceeding (if not empty). ## Execution Steps -1. **Setup**: Run `{SCRIPT}` from repo root and parse JSON for FEATURE_DIR and AVAILABLE_DOCS list. +1. **Setup**: Run `{SCRIPT}` from repo root and parse JSON for FEATURE_DIR, AVAILABLE_DOCS list, and TEMPLATE_CONTENT. - All file paths must be absolute. - For single quotes in args like "I'm Groot", use escape syntax: e.g 'I'\''m Groot' (or double-quote if possible: "I'm Groot"). @@ -127,7 +127,7 @@ You **MUST** consider the user input before proceeding (if not empty). - Use progressive disclosure: add follow-on retrieval only if gaps detected - If source docs are large, generate interim summary items instead of embedding raw text -6. **Generate checklist** - Create "Unit Tests for Requirements": +6. **Generate checklist** - Use TEMPLATE_CONTENT as the structural template and create "Unit Tests for Requirements": - Create `FEATURE_DIR/checklists/` directory if it doesn't exist - Generate unique checklist filename: - Use short, descriptive name based on domain (e.g., `ux.md`, `api.md`, `security.md`) diff --git a/templates/commands/constitution.md b/templates/commands/constitution.md index c631e8e84c..7b2f3684fb 100644 --- a/templates/commands/constitution.md +++ b/templates/commands/constitution.md @@ -4,6 +4,10 @@ handoffs: - label: Build Specification agent: speckit.specify prompt: Implement the feature specification based on the updated constitution. I want to build... +scripts: + sh: scripts/bash/resolve-template.sh constitution-template --json + ps: scripts/powershell/resolve-template.ps1 constitution-template -Json + py: scripts/python/resolve_template.py constitution-template --json --- ## User Input @@ -70,13 +74,22 @@ and commands read the constitution at runtime and are not modified here. ## Outline -You are updating the project constitution at `.specify/memory/constitution.md`. This file is a TEMPLATE containing placeholder tokens in square brackets (e.g. `[PROJECT_NAME]`, `[PRINCIPLE_1_NAME]`). Your job is to (a) collect/derive concrete values and (b) fill the template precisely. - -**Note**: If `.specify/memory/constitution.md` does not exist yet, it should have been initialized from `.specify/templates/constitution-template.md` during project setup. If it's missing, copy the template first. +You are updating the project constitution at `.specify/memory/constitution.md`. The active +constitution scaffold is resolved at command time from `constitution-template` through the Spec Kit +preset/template resolution stack. Follow this execution flow: -1. Load the existing constitution at `.specify/memory/constitution.md`. +1. Run `{SCRIPT}` from the repository root and parse `TEMPLATE_CONTENT` as the active template. + - The shared resolver applies project overrides, composing preset layers, and extension layers + before the core template fallback. It MUST succeed before continuing. + - If it fails, stop and report the resolution error; do not continue with only one contributing + template layer. + - If `.specify/memory/constitution.md` exists, load it as the source of current project-specific + values and amendments. Preserve information that is still applicable when applying the newly + resolved scaffold. + - If it does not exist, use the resolved template as the initial document. + - Do not write back to any versioned template layer. - Identify every placeholder token of the form `[ALL_CAPS_IDENTIFIER]`. **IMPORTANT**: The user might require less or more principles than the ones used in the template. If a number is specified, respect that - follow the general template. You will update the doc accordingly. @@ -90,7 +103,7 @@ Follow this execution flow: - PATCH: Clarifications, wording, typo fixes, non-semantic refinements. - If version bump type ambiguous, propose reasoning before finalizing. -3. Draft the updated constitution content: +3. Draft the updated constitution content using the resolved template as the required structure: - Replace every placeholder with concrete text (no bracketed tokens left except intentionally retained template slots that the project has chosen not to define yet—explicitly justify any left). - Preserve heading hierarchy and comments can be removed once replaced unless they still add clarifying guidance. - Ensure each Principle section: succinct name line, paragraph (or bullet list) capturing non‑negotiable rules, explicit rationale if not obvious. @@ -128,7 +141,7 @@ If the user supplies partial updates (e.g., only one principle revision), still If critical info missing (e.g., ratification date truly unknown), insert `TODO(): explanation` and include in the Sync Impact Report under deferred items. -Do not create a new template; always operate on the existing `.specify/memory/constitution.md` file. +Write only `.specify/memory/constitution.md`; do not create or modify template source files. ## Post-Execution Checks diff --git a/templates/commands/tasks.md b/templates/commands/tasks.md index 00d73354e3..64146a35aa 100644 --- a/templates/commands/tasks.md +++ b/templates/commands/tasks.md @@ -60,7 +60,7 @@ You **MUST** consider the user input before proceeding (if not empty). ## Outline -1. **Setup**: Run `{SCRIPT}` from repo root and parse FEATURE_DIR, TASKS_TEMPLATE, and AVAILABLE_DOCS list. `FEATURE_DIR` and `TASKS_TEMPLATE` must be absolute paths when provided. `AVAILABLE_DOCS` is a list of document names/relative paths available under `FEATURE_DIR` (for example `research.md` or `contracts/`). For single quotes in args like "I'm Groot", use escape syntax: e.g 'I'\''m Groot' (or double-quote if possible: "I'm Groot"). +1. **Setup**: Run `{SCRIPT}` from repo root and parse FEATURE_DIR, TASKS_TEMPLATE_CONTENT, TASKS_TEMPLATE, and AVAILABLE_DOCS list. `FEATURE_DIR` and `TASKS_TEMPLATE` must be absolute paths when provided. `AVAILABLE_DOCS` is a list of document names/relative paths available under `FEATURE_DIR` (for example `research.md` or `contracts/`). For single quotes in args like "I'm Groot", use escape syntax: e.g 'I'\''m Groot' (or double-quote if possible: "I'm Groot"). 2. **Load design documents**: Read from FEATURE_DIR: - **Required**: plan.md (tech stack, libraries, structure), spec.md (user stories with priorities) @@ -79,7 +79,7 @@ You **MUST** consider the user input before proceeding (if not empty). - Create parallel execution examples per user story - Validate task completeness (each user story has all needed tasks, independently testable) -4. **Generate tasks.md**: Read the tasks template from TASKS_TEMPLATE (from the JSON output above) and use it as structure. If TASKS_TEMPLATE is empty, fall back to `.specify/templates/tasks-template.md`. Fill with: +4. **Generate tasks.md**: Use TASKS_TEMPLATE_CONTENT (from the JSON output above) as the structure. For compatibility with older setup scripts that omit TASKS_TEMPLATE_CONTENT, read TASKS_TEMPLATE instead. Fill with: - Correct feature name from plan.md - Phase 1: Setup tasks (project initialization) - Phase 2: Foundational tasks (blocking prerequisites for all user stories) diff --git a/tests/integrations/test_integration_base_markdown.py b/tests/integrations/test_integration_base_markdown.py index aa906c440d..1ce88f90b2 100644 --- a/tests/integrations/test_integration_base_markdown.py +++ b/tests/integrations/test_integration_base_markdown.py @@ -241,11 +241,11 @@ def _expected_files(self, script_variant: str) -> list[str]: if script_variant == "sh": for name in ["check-prerequisites.sh", "common.sh", "create-new-feature.sh", - "setup-plan.sh", "setup-tasks.sh"]: + "resolve-template.sh", "setup-plan.sh", "setup-tasks.sh"]: files.append(f".specify/scripts/bash/{name}") else: for name in ["check-prerequisites.ps1", "common.ps1", "create-new-feature.ps1", - "setup-plan.ps1", "setup-tasks.ps1"]: + "resolve-template.ps1", "setup-plan.ps1", "setup-tasks.ps1"]: files.append(f".specify/scripts/powershell/{name}") for name in ["checklist-template.md", diff --git a/tests/integrations/test_integration_base_skills.py b/tests/integrations/test_integration_base_skills.py index 25551e1dc7..3d7e2de387 100644 --- a/tests/integrations/test_integration_base_skills.py +++ b/tests/integrations/test_integration_base_skills.py @@ -408,6 +408,7 @@ def _expected_files(self, script_variant: str) -> list[str]: ".specify/scripts/bash/check-prerequisites.sh", ".specify/scripts/bash/common.sh", ".specify/scripts/bash/create-new-feature.sh", + ".specify/scripts/bash/resolve-template.sh", ".specify/scripts/bash/setup-plan.sh", ".specify/scripts/bash/setup-tasks.sh", ] @@ -416,6 +417,7 @@ def _expected_files(self, script_variant: str) -> list[str]: ".specify/scripts/powershell/check-prerequisites.ps1", ".specify/scripts/powershell/common.ps1", ".specify/scripts/powershell/create-new-feature.ps1", + ".specify/scripts/powershell/resolve-template.ps1", ".specify/scripts/powershell/setup-plan.ps1", ".specify/scripts/powershell/setup-tasks.ps1", ] diff --git a/tests/integrations/test_integration_base_toml.py b/tests/integrations/test_integration_base_toml.py index 8a7344e4b2..4e4645a72b 100644 --- a/tests/integrations/test_integration_base_toml.py +++ b/tests/integrations/test_integration_base_toml.py @@ -494,6 +494,7 @@ def _expected_files(self, script_variant: str) -> list[str]: "check-prerequisites.sh", "common.sh", "create-new-feature.sh", + "resolve-template.sh", "setup-plan.sh", "setup-tasks.sh", ]: @@ -503,6 +504,7 @@ def _expected_files(self, script_variant: str) -> list[str]: "check-prerequisites.ps1", "common.ps1", "create-new-feature.ps1", + "resolve-template.ps1", "setup-plan.ps1", "setup-tasks.ps1", ]: diff --git a/tests/integrations/test_integration_base_yaml.py b/tests/integrations/test_integration_base_yaml.py index f3e39b24f8..572132dfb5 100644 --- a/tests/integrations/test_integration_base_yaml.py +++ b/tests/integrations/test_integration_base_yaml.py @@ -408,6 +408,7 @@ def _expected_files(self, script_variant: str) -> list[str]: "check-prerequisites.sh", "common.sh", "create-new-feature.sh", + "resolve-template.sh", "setup-plan.sh", "setup-tasks.sh", ]: @@ -417,6 +418,7 @@ def _expected_files(self, script_variant: str) -> list[str]: "check-prerequisites.ps1", "common.ps1", "create-new-feature.ps1", + "resolve-template.ps1", "setup-plan.ps1", "setup-tasks.ps1", ]: diff --git a/tests/integrations/test_integration_cline.py b/tests/integrations/test_integration_cline.py index f1abdedc8a..52ffa09104 100644 --- a/tests/integrations/test_integration_cline.py +++ b/tests/integrations/test_integration_cline.py @@ -191,6 +191,7 @@ def _expected_files(self, script_variant: str) -> list[str]: "check-prerequisites.sh", "common.sh", "create-new-feature.sh", + "resolve-template.sh", "setup-plan.sh", "setup-tasks.sh", ]: @@ -200,6 +201,7 @@ def _expected_files(self, script_variant: str) -> list[str]: "check-prerequisites.ps1", "common.ps1", "create-new-feature.ps1", + "resolve-template.ps1", "setup-plan.ps1", "setup-tasks.ps1", ]: diff --git a/tests/integrations/test_integration_copilot.py b/tests/integrations/test_integration_copilot.py index 7a680b7dd4..cee099c545 100644 --- a/tests/integrations/test_integration_copilot.py +++ b/tests/integrations/test_integration_copilot.py @@ -277,6 +277,7 @@ def test_complete_file_inventory_sh(self, tmp_path): ".specify/scripts/bash/check-prerequisites.sh", ".specify/scripts/bash/common.sh", ".specify/scripts/bash/create-new-feature.sh", + ".specify/scripts/bash/resolve-template.sh", ".specify/scripts/bash/setup-plan.sh", ".specify/scripts/bash/setup-tasks.sh", ".specify/templates/checklist-template.md", @@ -340,6 +341,7 @@ def test_complete_file_inventory_ps(self, tmp_path): ".specify/scripts/powershell/check-prerequisites.ps1", ".specify/scripts/powershell/common.ps1", ".specify/scripts/powershell/create-new-feature.ps1", + ".specify/scripts/powershell/resolve-template.ps1", ".specify/scripts/powershell/setup-plan.ps1", ".specify/scripts/powershell/setup-tasks.ps1", ".specify/templates/checklist-template.md", @@ -851,6 +853,7 @@ def test_complete_file_inventory_skills_sh(self, tmp_path): ".specify/scripts/bash/check-prerequisites.sh", ".specify/scripts/bash/common.sh", ".specify/scripts/bash/create-new-feature.sh", + ".specify/scripts/bash/resolve-template.sh", ".specify/scripts/bash/setup-plan.sh", ".specify/scripts/bash/setup-tasks.sh", # Templates diff --git a/tests/integrations/test_integration_generic.py b/tests/integrations/test_integration_generic.py index 202f7ab3dd..7fc12dd884 100644 --- a/tests/integrations/test_integration_generic.py +++ b/tests/integrations/test_integration_generic.py @@ -347,6 +347,7 @@ def test_complete_file_inventory_sh(self, tmp_path): ".specify/scripts/bash/check-prerequisites.sh", ".specify/scripts/bash/common.sh", ".specify/scripts/bash/create-new-feature.sh", + ".specify/scripts/bash/resolve-template.sh", ".specify/scripts/bash/setup-plan.sh", ".specify/scripts/bash/setup-tasks.sh", ".specify/templates/checklist-template.md", @@ -404,6 +405,7 @@ def test_complete_file_inventory_ps(self, tmp_path): ".specify/scripts/powershell/check-prerequisites.ps1", ".specify/scripts/powershell/common.ps1", ".specify/scripts/powershell/create-new-feature.ps1", + ".specify/scripts/powershell/resolve-template.ps1", ".specify/scripts/powershell/setup-plan.ps1", ".specify/scripts/powershell/setup-tasks.ps1", ".specify/templates/checklist-template.md", diff --git a/tests/parity_helpers.py b/tests/parity_helpers.py index 9289471eaf..27627dab5b 100644 --- a/tests/parity_helpers.py +++ b/tests/parity_helpers.py @@ -109,6 +109,67 @@ def write_feature_json( ) +def install_composition_stack( + repo: Path, template_name: str, core_content: str +) -> str: + """Install wrap/prepend/append presets over a core template.""" + templates = repo / ".specify" / "templates" + templates.mkdir(parents=True, exist_ok=True) + (templates / f"{template_name}.md").write_text(core_content, encoding="utf-8") + + layers = [ + ("wrap-pack", 1, "wrap", "## Wrapper\n{CORE_TEMPLATE}\n## End\n"), + ("prepend-pack", 2, "prepend", "# Prepended\n"), + ("append-pack", 3, "append", "# Appended\n"), + ] + registry: dict[str, object] = {"presets": {}} + registry_presets = registry["presets"] + assert isinstance(registry_presets, dict) + + for preset_id, priority, strategy, content in layers: + preset_dir = repo / ".specify" / "presets" / preset_id + template_dir = preset_dir / "templates" + template_dir.mkdir(parents=True) + (template_dir / f"{template_name}.md").write_text(content, encoding="utf-8") + (preset_dir / "preset.yml").write_text( + "provides:\n" + " templates:\n" + " - type: template\n" + f" name: {template_name}\n" + f" file: templates/{template_name}.md\n" + f" strategy: {strategy}\n", + encoding="utf-8", + ) + registry_presets[preset_id] = { + "enabled": True, + "priority": priority, + } + + (repo / ".specify" / "presets" / ".registry").write_text( + json.dumps(registry, separators=(",", ":")) + "\n", + encoding="utf-8", + ) + + appended = "# Appended\n" + prepended = "# Prepended\n" + wrapper = "## Wrapper\n{CORE_TEMPLATE}\n## End\n" + composed = f"{core_content}\n\n{appended}" + composed = f"{prepended}\n\n{composed}" + return wrapper.replace("{CORE_TEMPLATE}", composed) + + +def break_wrap_layer(repo: Path, template_name: str) -> None: + """Replace the installed wrap layer with one missing its placeholder.""" + ( + repo + / ".specify" + / "presets" + / "wrap-pack" + / "templates" + / f"{template_name}.md" + ).write_text("# Broken wrapper\n", encoding="utf-8") + + def normalize_repo_paths(text: str, repo: Path) -> str: """Replace the repo path with a placeholder so two-repo runs compare equal.""" repo_paths = sorted({str(repo), str(repo.resolve())}, key=len, reverse=True) diff --git a/tests/test_check_prerequisites_python_parity.py b/tests/test_check_prerequisites_python_parity.py index cdc02b915d..9ddd4dd9a5 100644 --- a/tests/test_check_prerequisites_python_parity.py +++ b/tests/test_check_prerequisites_python_parity.py @@ -12,6 +12,7 @@ import pytest from tests.conftest import requires_bash +from tests.parity_helpers import install_composition_stack PROJECT_ROOT = Path(__file__).resolve().parent.parent COMMON_SH = PROJECT_ROOT / "scripts" / "bash" / "common.sh" @@ -136,6 +137,87 @@ def _normalize_help_text(text: str) -> str: return "\n".join("" if not line.strip() else line for line in normalized.split("\n")) +@requires_bash +@pytest.mark.parametrize("missing", [False, True], ids=["composed", "missing"]) +def test_all_variants_resolve_requested_template( + prereq_repo: Path, missing: bool +) -> None: + _write_feature_json(prereq_repo) + feature = prereq_repo / "specs" / "001-my-feature" + feature.mkdir(parents=True) + (feature / "plan.md").write_text("# Plan\n", encoding="utf-8") + template_name = "missing-template" if missing else "checklist-template" + expected = install_composition_stack( + prereq_repo, "checklist-template", "# Checklist\n" + ) + + results = [ + _run( + _bash_cmd(prereq_repo, "--json", "--template", template_name), + prereq_repo, + ), + _run( + _py_cmd(prereq_repo, "--json", "--template", template_name), + prereq_repo, + ), + ] + if HAS_PWSH or _WINDOWS_POWERSHELL: + results.append( + _run( + _ps_cmd(prereq_repo, "-Json", "-Template", template_name), + prereq_repo, + ) + ) + + expected_status = 1 if missing else 0 + assert all(result.returncode == expected_status for result in results) + if missing: + assert all(result.stdout == "" for result in results) + else: + assert all( + _json_stdout(result)["TEMPLATE_CONTENT"] == expected + for result in results + ) + + +@requires_bash +@pytest.mark.parametrize("missing", [False, True], ids=["composed", "missing"]) +def test_all_variants_validate_requested_template_in_text_mode( + prereq_repo: Path, missing: bool +) -> None: + _write_feature_json(prereq_repo) + feature = prereq_repo / "specs" / "001-my-feature" + feature.mkdir(parents=True) + (feature / "plan.md").write_text("# Plan\n", encoding="utf-8") + template_name = "missing-template" if missing else "checklist-template" + install_composition_stack( + prereq_repo, "checklist-template", "# Checklist\n" + ) + + results = [ + _run( + _bash_cmd(prereq_repo, "--template", template_name), + prereq_repo, + ), + _run( + _py_cmd(prereq_repo, "--template", template_name), + prereq_repo, + ), + ] + if HAS_PWSH or _WINDOWS_POWERSHELL: + results.append( + _run( + _ps_cmd(prereq_repo, "-Template", template_name), + prereq_repo, + ) + ) + + expected_status = 1 if missing else 0 + assert all(result.returncode == expected_status for result in results) + if missing: + assert all(result.stdout == "" for result in results) + + @requires_bash @pytest.mark.parametrize( "args", diff --git a/tests/test_command_template_py_scripts.py b/tests/test_command_template_py_scripts.py index a634f1f2f0..07ef62c590 100644 --- a/tests/test_command_template_py_scripts.py +++ b/tests/test_command_template_py_scripts.py @@ -79,7 +79,7 @@ def test_template_renders_python_invocation(name: str): result = IntegrationBase.process_template(content, "agent", "py") assert "{SCRIPT}" not in result assert re.search( - r"python3 \.specify/scripts/python/\w+\.py(?: --[\w-]+)*", result + r"python3 \.specify/scripts/python/\w+\.py(?: [\w-]+)*", result ), f"{name} did not render a Python invocation" diff --git a/tests/test_create_new_feature_python_parity.py b/tests/test_create_new_feature_python_parity.py index 7c2c0e5622..41122b1f5f 100644 --- a/tests/test_create_new_feature_python_parity.py +++ b/tests/test_create_new_feature_python_parity.py @@ -13,6 +13,8 @@ from tests.parity_helpers import ( HAS_POWERSHELL, bash_cmd, + break_wrap_layer, + install_composition_stack, install_scripts, json_stdout, make_repo, @@ -382,12 +384,108 @@ def test_python_full_run_matches_bash(repo_pair: tuple[Path, Path]) -> None: branch = json_stdout(py)["BRANCH_NAME"] for repo in repo_pair: spec = repo / "specs" / branch / "spec.md" - assert spec.read_text(encoding="utf-8") == TEMPLATE_BODY + assert spec.read_bytes() == TEMPLATE_BODY.encode("utf-8") assert (repo_b / ".specify" / "feature.json").read_bytes() == ( repo_a / ".specify" / "feature.json" ).read_bytes() +@requires_bash +def test_all_variants_materialize_composed_spec_template(tmp_path: Path) -> None: + repos = [ + _setup_repo(tmp_path, "bash"), + _setup_repo(tmp_path, "powershell"), + _setup_repo(tmp_path, "python"), + ] + expected = "" + for current in repos: + expected = install_composition_stack( + current, "spec-template", TEMPLATE_BODY + ) + + bash = run( + bash_cmd( + repos[0], + SCRIPT, + "--json", + "--number", + "1", + "--short-name", + "composed", + "x", + ), + repos[0], + ) + py = run( + py_cmd( + repos[2], + SCRIPT, + "--json", + "--number", + "1", + "--short-name", + "composed", + "x", + ), + repos[2], + ) + results = [bash, py] + checked_repos = [repos[0], repos[2]] + if HAS_POWERSHELL: + results.insert( + 1, + run( + ps_cmd( + repos[1], + SCRIPT, + "-Json", + "-Number", + "1", + "-ShortName", + "composed", + "x", + ), + repos[1], + ), + ) + checked_repos.insert(1, repos[1]) + + assert all(result.returncode == 0 for result in results) + for current in checked_repos: + assert ( + current / "specs" / "001-composed" / "spec.md" + ).read_text(encoding="utf-8") == expected + + +@requires_bash +def test_all_variants_fail_for_broken_spec_composition(tmp_path: Path) -> None: + repos = [ + _setup_repo(tmp_path, "bash"), + _setup_repo(tmp_path, "powershell"), + _setup_repo(tmp_path, "python"), + ] + for current in repos: + install_composition_stack(current, "spec-template", TEMPLATE_BODY) + break_wrap_layer(current, "spec-template") + + bash = run(bash_cmd(repos[0], SCRIPT, "--json", "x"), repos[0]) + py = run(py_cmd(repos[2], SCRIPT, "--json", "x"), repos[2]) + results = [(bash, repos[0]), (py, repos[2])] + if HAS_POWERSHELL: + results.append( + ( + run(ps_cmd(repos[1], SCRIPT, "-Json", "x"), repos[1]), + repos[1], + ) + ) + + assert all(result.returncode != 0 for result, _ in results) + assert all( + not (current / "specs" / "001-x").exists() + for _, current in results + ) + + @requires_bash def test_python_missing_template_warning_matches_bash( repo_pair: tuple[Path, Path], diff --git a/tests/test_presets.py b/tests/test_presets.py index 243d13ab55..a2d0f0cdfd 100644 --- a/tests/test_presets.py +++ b/tests/test_presets.py @@ -1063,6 +1063,40 @@ def test_resolve_nonexistent(self, project_dir): result = resolver.resolve("nonexistent-template") assert result is None + def test_resolver_ignores_traversing_registry_ids(self, project_dir): + """Registry IDs cannot escape preset or extension install roots.""" + for registry_dir, registry_key, outside_name in ( + ("presets", "presets", "outside-preset"), + ("extensions", "extensions", "outside-extension"), + ): + outside = project_dir.parent / outside_name + (outside / "templates").mkdir(parents=True) + (outside / "templates" / "spec-template.md").write_text( + f"# Sensitive {registry_key}\n", + encoding="utf-8", + ) + installed = project_dir / ".specify" / registry_dir + installed.mkdir(parents=True, exist_ok=True) + (installed / ".registry").write_text( + json.dumps( + { + registry_key: { + f"../../../{outside_name}": { + "enabled": True, + "priority": 1, + } + } + } + ), + encoding="utf-8", + ) + + content = PresetResolver(project_dir).resolve_content("spec-template") + + assert content is not None + assert "Core Spec Template" in content + assert "Sensitive" not in content + def test_resolve_higher_priority_pack_wins(self, project_dir, temp_dir, valid_pack_data): """Test that a pack with lower priority number wins over higher number.""" manager = PresetManager(project_dir) @@ -1375,6 +1409,45 @@ def test_resolve_disabled_extension_not_picked_up_as_unregistered(self, project_ result = resolver.resolve("unique-disabled-template") assert result is None, "Disabled extension should not be picked up as unregistered" + @pytest.mark.parametrize( + "registry_bytes", + [b"{ not valid json", b'{"extensions": []}', b"[]"], + ids=["invalid_json", "non_mapping_extensions", "non_mapping_root"], + ) + def test_resolve_fails_closed_on_corrupt_extension_registry( + self, project_dir, registry_bytes + ): + """A corrupt extension registry must fail closed rather than let the + directory scan admit every on-disk extension as enabled.""" + extensions_dir = project_dir / ".specify" / "extensions" + ext_templates_dir = extensions_dir / "sneaky-ext" / "templates" + ext_templates_dir.mkdir(parents=True) + (ext_templates_dir / "custom-template.md").write_text( + "# Should not be served\n" + ) + (extensions_dir / ".registry").write_bytes(registry_bytes) + + resolver = PresetResolver(project_dir) + with pytest.raises(PresetValidationError, match="Invalid extension registry"): + resolver._get_all_extensions_by_priority() + with pytest.raises(PresetValidationError, match="Invalid extension registry"): + resolver.resolve("custom-template") + + def test_resolve_fails_closed_when_registry_is_directory(self, project_dir): + """A directory at the registry path must fail closed, not be treated as + an absent registry that enables every on-disk extension.""" + extensions_dir = project_dir / ".specify" / "extensions" + ext_templates_dir = extensions_dir / "sneaky-ext" / "templates" + ext_templates_dir.mkdir(parents=True) + (ext_templates_dir / "custom-template.md").write_text( + "# Should not be served\n" + ) + (extensions_dir / ".registry").mkdir() + + resolver = PresetResolver(project_dir) + with pytest.raises(PresetValidationError, match="Invalid extension registry"): + resolver.resolve("custom-template") + def test_resolve_pack_over_extension(self, project_dir, pack_dir, temp_dir, valid_pack_data): """Test that pack templates take priority over extension templates.""" # Create extension with templates @@ -3382,6 +3455,9 @@ def test_url_cache_expired(self, project_dir): SELF_TEST_PRESET_DIR = Path(__file__).parent.parent / "presets" / "self-test" +CONSTITUTION_SYNC_PRESET_DIR = ( + Path(__file__).parent.parent / "presets" / "constitution-sync" +) SELF_TEST_WRAP_WARNING = ( r"Cannot compose command 'speckit\.wrap-test': no base layer\. " r"Stale command files may remain\." @@ -3408,6 +3484,11 @@ def install_self_test_preset(manager: PresetManager, speckit_version: str = "0.1 return manager.install_from_directory(SELF_TEST_PRESET_DIR, speckit_version) +def install_constitution_sync_preset(manager: PresetManager) -> PresetManifest: + """Enable guarded install-time constitution materialization.""" + return manager.install_from_directory(CONSTITUTION_SYNC_PRESET_DIR, "0.15.0") + + def _make_convention_constitution_preset(temp_dir: Path) -> Path: """Create a preset whose constitution is found by convention, not its manifest.""" preset_dir = temp_dir / "convention-constitution" @@ -3540,6 +3621,7 @@ def test_self_test_removal_restores_core(self, project_dir): (templates_dir / f"{name}.md").write_text(f"# Core {name}\n") manager = PresetManager(project_dir) + install_constitution_sync_preset(manager) install_self_test_preset(manager) manager.remove("self-test") @@ -3558,6 +3640,7 @@ def test_self_test_removal_preserves_edited_constitution(self, project_dir): (templates_dir / "constitution-template.md").write_text("# Core Constitution\n") manager = PresetManager(project_dir) + install_constitution_sync_preset(manager) install_self_test_preset(manager) memory = project_dir / ".specify" / "memory" / "constitution.md" edited = memory.read_text() + "\n## Authored amendment\n" @@ -3641,19 +3724,16 @@ def test_self_test_no_commands_without_agent_dirs(self, project_dir): metadata = manager.registry.get("self-test") assert metadata["registered_commands"] == {} - def test_self_test_seeds_constitution_when_memory_absent(self, project_dir): - """Installing a preset seeds memory/constitution.md from its template.""" + def test_self_test_does_not_seed_constitution_without_sync(self, project_dir): + """Installing a preset does not materialize its constitution by default.""" manager = PresetManager(project_dir) install_self_test_preset(manager) memory = project_dir / ".specify" / "memory" / "constitution.md" - assert memory.exists(), "constitution.md was not seeded from the preset" - assert "preset:self-test" in memory.read_text(), ( - "constitution.md was not seeded from the self-test preset template" - ) + assert not memory.exists() - def test_self_test_reseeds_exact_core_constitution(self, project_dir): - """An unchanged core constitution is re-seeded from the preset template.""" + def test_self_test_preserves_generated_constitution_without_sync(self, project_dir): + """Preset install and removal preserve generated content without the opt-in.""" resolver = PresetResolver(project_dir) bundled_core = resolver._find_bundled_core( "constitution-template", "template", ".md" @@ -3666,10 +3746,19 @@ def test_self_test_reseeds_exact_core_constitution(self, project_dir): manager = PresetManager(project_dir) install_self_test_preset(manager) + manager.remove("self-test") - content = memory.read_text() - assert "preset:self-test" in content, "placeholder constitution was not re-seeded" - assert "[PROJECT_NAME]" not in content + assert memory.read_bytes() == core + + def test_self_test_seeds_constitution_with_sync(self, project_dir): + """constitution-sync preserves the previous install-time seeding behavior.""" + manager = PresetManager(project_dir) + install_constitution_sync_preset(manager) + install_self_test_preset(manager) + + memory = project_dir / ".specify" / "memory" / "constitution.md" + assert "preset:self-test" in memory.read_text() + assert "[PROJECT_NAME]" not in memory.read_text() @pytest.mark.parametrize( "provenance_content", @@ -3697,6 +3786,7 @@ def test_self_test_preserves_core_content_with_existing_invalid_provenance( original = memory.read_bytes() manager = PresetManager(project_dir) + install_constitution_sync_preset(manager) install_self_test_preset(manager) assert memory.read_bytes() == original @@ -3713,6 +3803,7 @@ def test_self_test_preserves_mutable_project_core_copy(self, project_dir): memory.write_text(authored) manager = PresetManager(project_dir) + install_constitution_sync_preset(manager) install_self_test_preset(manager) assert memory.read_text() == authored @@ -3759,7 +3850,9 @@ def test_core_prefixed_preset_does_not_establish_generated_provenance( ) ) - PresetManager(project_dir).install_from_directory(preset_dir, "0.1.5") + manager = PresetManager(project_dir) + install_constitution_sync_preset(manager) + manager.install_from_directory(preset_dir, "0.1.5") assert memory.read_text() == authored assert not (memory.parent / ".constitution-template.json").exists() @@ -3774,6 +3867,7 @@ def test_self_test_preserves_authored_constitution_with_placeholder( memory.write_text(authored) manager = PresetManager(project_dir) + install_constitution_sync_preset(manager) install_self_test_preset(manager) assert memory.read_text() == authored @@ -3786,6 +3880,7 @@ def test_self_test_preserves_authored_constitution(self, project_dir): memory.write_text(authored) manager = PresetManager(project_dir) + install_constitution_sync_preset(manager) install_self_test_preset(manager) assert memory.read_text() == authored, "authored constitution was overwritten" @@ -3843,6 +3938,7 @@ def test_constitution_seed_composes_wrap_strategy(self, project_dir, temp_dir): ) manager = PresetManager(project_dir) + install_constitution_sync_preset(manager) manager.install_from_directory(preset_dir, "0.1.5") memory = project_dir / ".specify" / "memory" / "constitution.md" @@ -3856,6 +3952,7 @@ def test_constitution_follows_priority_when_winning_preset_removed( ): """An unchanged generated constitution follows priority and fallback layers.""" manager = PresetManager(project_dir) + install_constitution_sync_preset(manager) install_self_test_preset(manager) preset_dir = temp_dir / "higher-priority" @@ -3903,6 +4000,7 @@ def test_convention_constitution_removal_restores_remaining_layer( ): """Removing a convention layer rematerializes the remaining resolver layer.""" manager = PresetManager(project_dir) + install_constitution_sync_preset(manager) install_self_test_preset(manager) manager.install_from_directory( _make_convention_constitution_preset(temp_dir), "0.1.5", priority=1 @@ -3924,6 +4022,7 @@ def test_convention_constitution_removal_preserves_edited_content( templates_dir = project_dir / ".specify" / "templates" (templates_dir / "constitution-template.md").write_text("# Core Constitution\n") manager = PresetManager(project_dir) + install_constitution_sync_preset(manager) manager.install_from_directory( _make_convention_constitution_preset(temp_dir), "0.1.5" ) @@ -3941,6 +4040,7 @@ def test_custom_constitution_removal_recovers_with_invalid_manifest( ): """Provenance triggers fallback when a custom-path manifest is invalid.""" manager = PresetManager(project_dir) + install_constitution_sync_preset(manager) install_self_test_preset(manager) preset_dir = temp_dir / "custom-constitution" @@ -4001,9 +4101,9 @@ def test_constitution_seed_rejects_symlinked_memory_directory( manager = PresetManager(project_dir) with pytest.warns(UserWarning, match="symlinked"): - install_self_test_preset(manager) + install_constitution_sync_preset(manager) - assert manager.registry.is_installed("self-test") + assert manager.registry.is_installed("constitution-sync") assert not (outside / "constitution.md").exists() def test_constitution_seed_rejects_dangling_destination_symlink( @@ -4020,9 +4120,9 @@ def test_constitution_seed_rejects_dangling_destination_symlink( manager = PresetManager(project_dir) with pytest.warns(UserWarning, match="symlinked"): - install_self_test_preset(manager) + install_constitution_sync_preset(manager) - assert manager.registry.is_installed("self-test") + assert manager.registry.is_installed("constitution-sync") assert not outside.exists() def test_constitution_materialization_error_is_nonfatal( @@ -4061,6 +4161,7 @@ def test_constitution_materialization_error_is_nonfatal( ) manager = PresetManager(project_dir) + install_constitution_sync_preset(manager) with pytest.warns(UserWarning, match="Failed to seed constitution"): manifest = manager.install_from_directory(preset_dir, "0.1.5") @@ -9557,6 +9658,7 @@ def test_set_priority_reconciles_generated_constitution( from specify_cli import app manager = PresetManager(project_dir) + install_constitution_sync_preset(manager) install_self_test_preset(manager) manager.install_from_directory( _make_convention_constitution_preset(temp_dir), "0.1.5", priority=20 @@ -9802,6 +9904,7 @@ def test_enable_disable_reconciles_generated_constitution( from specify_cli import app manager = PresetManager(project_dir) + install_constitution_sync_preset(manager) install_self_test_preset(manager) manager.install_from_directory( _make_convention_constitution_preset(temp_dir), "0.1.5", priority=1 @@ -10022,6 +10125,29 @@ def test_constitution_commands_guard_against_non_governance_work(command_path): assert "do not invoke it" in normalized_content or "without invoking it" in normalized_content +def test_core_constitution_command_resolves_template_at_runtime(): + """The core command must consume the composed scaffold on every invocation.""" + content = CORE_CONSTITUTION_COMMAND.read_text() + + assert "resolve-template.sh constitution-template --json" in content + assert "resolve-template.ps1 constitution-template -Json" in content + assert "resolve_template.py constitution-template --json" in content + assert "parse `TEMPLATE_CONTENT` as the active template" in content + assert "do not continue with only one contributing" in content + assert "Do not write back to any versioned template layer" in content + + +def test_core_checklist_command_resolves_template_at_runtime(): + """The checklist command must consume the composed scaffold.""" + content = (CORE_CONSTITUTION_COMMAND.parent / "checklist.md").read_text( + encoding="utf-8" + ) + + assert "--template checklist-template" in content + assert "TEMPLATE_CONTENT" in content + assert "Use TEMPLATE_CONTENT as the structural template" in content + + class TestLeanPreset: """Tests for the lean preset that ships with the repo.""" @@ -12239,10 +12365,10 @@ def fake_open(url, timeout=None, extra_headers=None): class TestEnsureConstitutionResolverAware: """`ensure_constitution_from_template` must resolve through PresetResolver. - The constitution is the only template materialized to a live file rather - than resolved on demand. These tests pin the regression from issue #3272: - a preset-provided ``constitution-template`` must seed memory, while the - core template is used when no preset overrides it. + Init materializes the live constitution once, while later /constitution + runs resolve on demand. These tests pin the regression from issue #3272: + a preset-provided ``constitution-template`` must win during the init seed, + while the core template is used when no preset overrides it. """ def _core_constitution(self, project_dir): @@ -12303,10 +12429,8 @@ def test_seeds_from_preset_when_installed(self, project_dir): manager = PresetManager(project_dir) install_self_test_preset(manager) - # Remove the memory file seeded during install to test ensure() in - # isolation; it must re-seed from the preset, not the core template. memory = project_dir / ".specify" / "memory" / "constitution.md" - memory.unlink() + assert not memory.exists() ensure_constitution_from_template(project_dir) @@ -12349,9 +12473,8 @@ def test_composes_wrap_strategy_when_ensuring(self, project_dir, temp_dir): manager = PresetManager(project_dir) manager.install_from_directory(self._wrap_constitution_preset(temp_dir), "0.1.5") - # Ensure we validate ensure() behavior directly. memory = project_dir / ".specify" / "memory" / "constitution.md" - memory.unlink() + assert not memory.exists() ensure_constitution_from_template(project_dir) content = memory.read_text() @@ -12720,11 +12843,41 @@ def test_unbalanced_markup_does_not_crash_list_or_info(self, temp_dir, project_d assert result.exit_code == 0, (args, result.output, result.exception) assert "Broken [/red] tag" in strip_ansi(result.output) - def test_resolve_escapes_template_name(self, project_dir): - """``preset resolve`` echoes its argument; an unbalanced tag must not crash.""" + def test_resolve_rejects_invalid_template_name(self, project_dir): + """``preset resolve`` rejects names before joining them into paths.""" result = self._invoke(project_dir, ["preset", "resolve", "no[/red]such"]) + assert result.exit_code == 1, (result.output, result.exception) + assert "invalid template name" in strip_ansi(result.output) + + def test_resolve_rejects_path_traversal(self, project_dir): + """The resolver rejects traversal before joining names into paths.""" + result = self._invoke( + project_dir, + ["preset", "resolve", "../../../README"], + ) + + assert result.exit_code == 1 + assert "invalid template name" in strip_ansi(result.output) + + def test_resolve_accepts_dotted_command_name(self, project_dir): + """Documented dotted command identifiers use command resolution.""" + result = self._invoke( + project_dir, + ["preset", "resolve", "speckit.constitution"], + ) + assert result.exit_code == 0, (result.output, result.exception) - assert "no[/red]such" in strip_ansi(result.output) + assert "constitution.md" in strip_ansi(result.output) + + def test_resolve_rejects_empty_command_segments(self, project_dir): + """Dotted command identifiers cannot contain empty path-like segments.""" + result = self._invoke( + project_dir, + ["preset", "resolve", "speckit..constitution"], + ) + + assert result.exit_code == 1 + assert "invalid template name" in strip_ansi(result.output) def test_resolve_escapes_layer_path_and_source(self, project_dir): """The top-layer path/source lines must render markup literally. @@ -12818,14 +12971,13 @@ def test_resolve_renders_composition_strategy_labels(self, temp_dir, project_dir assert "[base]" in output, output assert "[append]" in output, output - class TestConstitutionSyncPreset: - """The bundled opt-in ``constitution-sync`` preset re-adds propagation. + """The bundled opt-in ``constitution-sync`` preset re-adds materialization. Follow-up to #3790: core ``/constitution`` no longer propagates guidance - into templates. This preset restores that behavior for teams that treat - materialized templates as reviewed artifacts, delivered as a ``wrap`` of - the core command so it stays forward-compatible with core changes. + into templates. Issue #3950 also gates install-time constitution seeding on + this preset. Its command override remains a ``wrap`` of core so it stays + forward-compatible with core changes. """ PRESET_DIR = Path(__file__).parent.parent / "presets" / "constitution-sync" diff --git a/tests/test_resolve_template_python_parity.py b/tests/test_resolve_template_python_parity.py new file mode 100644 index 0000000000..9af5554b44 --- /dev/null +++ b/tests/test_resolve_template_python_parity.py @@ -0,0 +1,753 @@ +"""Parity tests for composed runtime template resolution.""" + +from __future__ import annotations + +import json +import os +from pathlib import Path + +import pytest + +from tests.conftest import requires_bash +from tests.parity_helpers import ( + HAS_POWERSHELL, + bash_cmd, + clean_env, + install_composition_stack, + install_scripts, + json_stdout, + make_repo, + ps_cmd, + py_cmd, + run, +) + +SCRIPT = "resolve-template" +TEMPLATE = "constitution-template" + + +def _setup_repo(tmp_path: Path) -> tuple[Path, str]: + repo = make_repo(tmp_path) + install_scripts(repo, SCRIPT) + expected = install_composition_stack(repo, TEMPLATE, "# Core\n") + return repo, expected + + +@requires_bash +def test_all_variants_emit_composed_template_content(tmp_path: Path) -> None: + repo, expected = _setup_repo(tmp_path) + results = [ + run(bash_cmd(repo, SCRIPT, TEMPLATE, "--json"), repo), + run(py_cmd(repo, SCRIPT, TEMPLATE, "--json"), repo), + ] + if HAS_POWERSHELL: + results.append(run(ps_cmd(repo, SCRIPT, TEMPLATE, "-Json"), repo)) + + assert all(result.returncode == 0 for result in results) + assert all(result.stderr == "" for result in results) + assert all( + json_stdout(result) + == {"TEMPLATE_NAME": TEMPLATE, "TEMPLATE_CONTENT": expected} + for result in results + ) + + +@requires_bash +@pytest.mark.parametrize( + "without_registry,core_content", + [ + (True, "# Core\n"), + (False, "# Café ✓\n"), + ], + ids=["directory_fallback", "unicode"], +) +def test_all_variants_preserve_composition_parity( + tmp_path: Path, without_registry: bool, core_content: str +) -> None: + repo = make_repo(tmp_path) + install_scripts(repo, SCRIPT) + expected = install_composition_stack(repo, TEMPLATE, core_content) + if without_registry: + (repo / ".specify" / "presets" / ".registry").unlink() + expected = ( + "# Prepended\n\n\n" + "## Wrapper\n" + f"{core_content}\n" + "## End\n\n\n" + "# Appended\n" + ) + + results = [ + run(bash_cmd(repo, SCRIPT, TEMPLATE, "--json"), repo), + run(py_cmd(repo, SCRIPT, TEMPLATE, "--json"), repo), + ] + if HAS_POWERSHELL: + results.append(run(ps_cmd(repo, SCRIPT, TEMPLATE, "-Json"), repo)) + + assert all(result.returncode == 0 for result in results) + assert all( + json_stdout(result)["TEMPLATE_CONTENT"] == expected + for result in results + ) + + +@requires_bash +def test_all_variants_read_utf8_registry_under_ascii_locale( + tmp_path: Path, +) -> None: + """Registry/manifest reads must force UTF-8, not the process locale. + + With UTF-8 mode disabled and a C locale, the interpreter's default text + encoding is ASCII. Non-ASCII *metadata* in the registry or a manifest must + still resolve, because the resolvers open those files as UTF-8 explicitly. + Template content stays ASCII so the pure-Python variant can emit it on the + ASCII stdout this configuration forces. + """ + repo = make_repo(tmp_path) + install_scripts(repo, SCRIPT) + expected = install_composition_stack(repo, TEMPLATE, "# Core\n") + + # Inject non-ASCII metadata into the preset registry and a manifest so a + # locale-dependent decode would raise instead of resolving cleanly. + registry = repo / ".specify" / "presets" / ".registry" + registry_data = json.loads(registry.read_text(encoding="utf-8")) + registry_data["presets"]["wrap-pack"]["description"] = "Café ✓ wrapper" + registry.write_text( + json.dumps(registry_data, separators=(",", ":")) + "\n", + encoding="utf-8", + ) + manifest = repo / ".specify" / "presets" / "wrap-pack" / "preset.yml" + manifest.write_text( + manifest.read_text(encoding="utf-8") + ' description: "Café ✓"\n', + encoding="utf-8", + ) + + env = clean_env() + # Force the interpreter's default text encoding to ASCII so an unqualified + # open() would fail on the non-ASCII metadata above. + env["PYTHONUTF8"] = "0" + env["PYTHONCOERCECLOCALE"] = "0" + env["LC_ALL"] = "C" + env["LANG"] = "C" + + results = [ + run(bash_cmd(repo, SCRIPT, TEMPLATE, "--json"), repo, env), + run(py_cmd(repo, SCRIPT, TEMPLATE, "--json"), repo, env), + ] + if HAS_POWERSHELL: + results.append(run(ps_cmd(repo, SCRIPT, TEMPLATE, "-Json"), repo, env)) + + assert all(result.returncode == 0 for result in results) + assert all( + json_stdout(result)["TEMPLATE_CONTENT"] == expected + for result in results + ) + + +@requires_bash +@pytest.mark.parametrize( + "template_name", + ["missing-template", "../../../outside"], + ids=["missing", "path_traversal"], +) +def test_all_variants_reject_unresolvable_template( + tmp_path: Path, template_name: str +) -> None: + repo = make_repo(tmp_path) + install_scripts(repo, SCRIPT) + (repo / "outside.md").write_text("sensitive content\n", encoding="utf-8") + + results = [ + run(bash_cmd(repo, SCRIPT, template_name, "--json"), repo), + run(py_cmd(repo, SCRIPT, template_name, "--json"), repo), + ] + if HAS_POWERSHELL: + results.append(run(ps_cmd(repo, SCRIPT, template_name, "-Json"), repo)) + + assert all(result.returncode == 1 for result in results) + assert all(result.stdout == "" for result in results) + assert all("sensitive content" not in result.stderr for result in results) + + +@requires_bash +def test_all_variants_ignore_traversing_preset_registry_ids(tmp_path: Path) -> None: + repo = make_repo(tmp_path) + install_scripts(repo, SCRIPT) + outside = repo.parent / "outside" + outside.mkdir() + (outside / f"{TEMPLATE}.md").write_text("sensitive content\n", encoding="utf-8") + presets = repo / ".specify" / "presets" + presets.mkdir(parents=True) + (presets / ".registry").write_text( + '{"presets":{"../../../outside":{"enabled":true,"priority":1}}}\n', + encoding="utf-8", + ) + + results = [ + run(bash_cmd(repo, SCRIPT, TEMPLATE, "--json"), repo), + run(py_cmd(repo, SCRIPT, TEMPLATE, "--json"), repo), + ] + if HAS_POWERSHELL: + results.append(run(ps_cmd(repo, SCRIPT, TEMPLATE, "-Json"), repo)) + + assert all(result.returncode == 1 for result in results) + assert all("sensitive content" not in result.stdout for result in results) + + +@requires_bash +def test_all_variants_support_root_level_preset_convention(tmp_path: Path) -> None: + repo = make_repo(tmp_path) + install_scripts(repo, SCRIPT) + preset = repo / ".specify" / "presets" / "root-pack" + preset.mkdir(parents=True) + (preset / f"{TEMPLATE}.md").write_text("# Root convention\n", encoding="utf-8") + (repo / ".specify" / "presets" / ".registry").write_text( + '{"presets":{"root-pack":{"enabled":true,"priority":1}}}\n', + encoding="utf-8", + ) + + results = [ + run(bash_cmd(repo, SCRIPT, TEMPLATE, "--json"), repo), + run(py_cmd(repo, SCRIPT, TEMPLATE, "--json"), repo), + ] + if HAS_POWERSHELL: + results.append(run(ps_cmd(repo, SCRIPT, TEMPLATE, "-Json"), repo)) + + assert all(result.returncode == 0 for result in results) + assert all( + json_stdout(result)["TEMPLATE_CONTENT"] == "# Root convention\n" + for result in results + ) + + +@requires_bash +def test_all_variants_honor_extension_registry_state_and_priority( + tmp_path: Path, +) -> None: + repo = make_repo(tmp_path) + install_scripts(repo, SCRIPT) + extensions = repo / ".specify" / "extensions" + for extension_id, content in ( + ("disabled-ext", "# Disabled\n"), + ("low-priority", "# Low priority\n"), + ("high-priority", "# High priority\n"), + ): + template_dir = extensions / extension_id / "templates" + template_dir.mkdir(parents=True) + (template_dir / f"{TEMPLATE}.md").write_text(content, encoding="utf-8") + (extensions / ".registry").write_text( + '{"extensions":{' + '"disabled-ext":{"enabled":null,"priority":1},' + '"low-priority":{"enabled":true,"priority":20},' + '"high-priority":{"enabled":true,"priority":5}' + "}}\n", + encoding="utf-8", + ) + + results = [ + run(bash_cmd(repo, SCRIPT, TEMPLATE, "--json"), repo), + run(py_cmd(repo, SCRIPT, TEMPLATE, "--json"), repo), + ] + if HAS_POWERSHELL: + results.append(run(ps_cmd(repo, SCRIPT, TEMPLATE, "-Json"), repo)) + + assert all(result.returncode == 0 for result in results) + assert all( + json_stdout(result)["TEMPLATE_CONTENT"] == "# High priority\n" + for result in results + ) + + +@requires_bash +def test_all_variants_support_root_level_extension_convention( + tmp_path: Path, +) -> None: + repo = make_repo(tmp_path) + install_scripts(repo, SCRIPT) + extension = repo / ".specify" / "extensions" / "root-extension" + extension.mkdir(parents=True) + (extension / f"{TEMPLATE}.md").write_text( + "# Root extension\n", + encoding="utf-8", + ) + (repo / ".specify" / "extensions" / ".registry").write_text( + '{"extensions":{"root-extension":{"enabled":true,"priority":1}}}\n', + encoding="utf-8", + ) + + results = [ + run(bash_cmd(repo, SCRIPT, TEMPLATE, "--json"), repo), + run(py_cmd(repo, SCRIPT, TEMPLATE, "--json"), repo), + ] + if HAS_POWERSHELL: + results.append(run(ps_cmd(repo, SCRIPT, TEMPLATE, "-Json"), repo)) + + assert all(result.returncode == 0 for result in results) + assert all( + json_stdout(result)["TEMPLATE_CONTENT"] == "# Root extension\n" + for result in results + ) + + +@requires_bash +def test_all_variants_treat_extension_registry_ids_case_sensitively( + tmp_path: Path, +) -> None: + repo = make_repo(tmp_path) + install_scripts(repo, SCRIPT) + extension = repo / ".specify" / "extensions" / "foo" / "templates" + extension.mkdir(parents=True) + (extension / f"{TEMPLATE}.md").write_text( + "# Lowercase extension\n", + encoding="utf-8", + ) + (repo / ".specify" / "extensions" / ".registry").write_text( + '{"extensions":{"FOO":{"enabled":true,"priority":1}}}\n', + encoding="utf-8", + ) + + results = [ + run(bash_cmd(repo, SCRIPT, TEMPLATE, "--json"), repo), + run(py_cmd(repo, SCRIPT, TEMPLATE, "--json"), repo), + ] + if HAS_POWERSHELL: + results.append(run(ps_cmd(repo, SCRIPT, TEMPLATE, "-Json"), repo)) + + assert all(result.returncode == 0 for result in results) + assert all( + json_stdout(result)["TEMPLATE_CONTENT"] == "# Lowercase extension\n" + for result in results + ) + + +@requires_bash +@pytest.mark.parametrize( + "registry_content", + ["{ not valid json", '{"extensions":[]}\n', "[]\n"], + ids=["invalid_json", "non_mapping_extensions", "non_mapping_root"], +) +def test_all_variants_fail_for_malformed_extension_registry( + tmp_path: Path, registry_content: str +) -> None: + """A corrupt extension registry must fail closed, not silently enable + every on-disk extension directory as unregistered.""" + repo = make_repo(tmp_path) + install_scripts(repo, SCRIPT) + extensions = repo / ".specify" / "extensions" + template_dir = extensions / "sneaky-ext" / "templates" + template_dir.mkdir(parents=True) + (template_dir / f"{TEMPLATE}.md").write_text( + "# Should not be served\n", encoding="utf-8" + ) + (extensions / ".registry").write_text(registry_content, encoding="utf-8") + + results = [ + run(bash_cmd(repo, SCRIPT, TEMPLATE, "--json"), repo), + run(py_cmd(repo, SCRIPT, TEMPLATE, "--json"), repo), + ] + if HAS_POWERSHELL: + results.append(run(ps_cmd(repo, SCRIPT, TEMPLATE, "-Json"), repo)) + + assert all(result.returncode != 0 for result in results) + assert all(result.stdout == "" for result in results) + assert all( + "Should not be served" not in result.stdout for result in results + ) + + +@requires_bash +def test_all_variants_fail_when_registry_is_a_directory( + tmp_path: Path, +) -> None: + """A directory at the extension registry path must fail closed, not be + treated as an absent registry that enables every on-disk extension.""" + repo = make_repo(tmp_path) + install_scripts(repo, SCRIPT) + extensions = repo / ".specify" / "extensions" + template_dir = extensions / "sneaky-ext" / "templates" + template_dir.mkdir(parents=True) + (template_dir / f"{TEMPLATE}.md").write_text( + "# Should not be served\n", encoding="utf-8" + ) + # Create ``.registry`` as a directory rather than a regular file. + (extensions / ".registry").mkdir() + + results = [ + run(bash_cmd(repo, SCRIPT, TEMPLATE, "--json"), repo), + run(py_cmd(repo, SCRIPT, TEMPLATE, "--json"), repo), + ] + if HAS_POWERSHELL: + results.append(run(ps_cmd(repo, SCRIPT, TEMPLATE, "-Json"), repo)) + + assert all(result.returncode != 0 for result in results) + assert all(result.stdout == "" for result in results) + + +@requires_bash +def test_all_variants_fail_when_registry_is_broken_symlink( + tmp_path: Path, +) -> None: + """A broken symlink at the extension registry path must fail closed across + Bash, Python, and PowerShell resolvers rather than being treated as absent.""" + repo = make_repo(tmp_path) + install_scripts(repo, SCRIPT) + extensions = repo / ".specify" / "extensions" + template_dir = extensions / "sneaky-ext" / "templates" + template_dir.mkdir(parents=True) + (template_dir / f"{TEMPLATE}.md").write_text( + "# Should not be served\n", encoding="utf-8" + ) + (extensions / ".registry").symlink_to(extensions / "does-not-exist") + + results = [ + run(bash_cmd(repo, SCRIPT, TEMPLATE, "--json"), repo), + run(py_cmd(repo, SCRIPT, TEMPLATE, "--json"), repo), + ] + if HAS_POWERSHELL: + results.append(run(ps_cmd(repo, SCRIPT, TEMPLATE, "-Json"), repo)) + + assert all(result.returncode != 0 for result in results) + assert all(result.stdout == "" for result in results) + + +@requires_bash +@pytest.mark.parametrize("base_kind", ["override", "preset"]) +def test_all_variants_ignore_malformed_layers_below_replace_base( + tmp_path: Path, + base_kind: str, +) -> None: + repo = make_repo(tmp_path) + install_scripts(repo, SCRIPT) + expected = "# Winning base\r\nBody\r\n" + presets = repo / ".specify" / "presets" + + if base_kind == "override": + override = repo / ".specify" / "templates" / "overrides" + override.mkdir(parents=True) + (override / f"{TEMPLATE}.md").write_bytes(expected.encode("utf-8")) + registry = {"presets": {"broken-pack": {"enabled": True, "priority": 1}}} + else: + winning = presets / "winning-pack" / "templates" + winning.mkdir(parents=True) + (winning / f"{TEMPLATE}.md").write_bytes(expected.encode("utf-8")) + registry = { + "presets": { + "winning-pack": {"enabled": True, "priority": 1}, + "broken-pack": {"enabled": True, "priority": 2}, + } + } + + broken = presets / "broken-pack" + broken.mkdir(parents=True) + (broken / "preset.yml").write_text("provides: [\n", encoding="utf-8") + (presets / ".registry").write_text( + json.dumps(registry, separators=(",", ":")) + "\n", + encoding="utf-8", + ) + + results = [ + run(bash_cmd(repo, SCRIPT, TEMPLATE, "--json"), repo), + run(py_cmd(repo, SCRIPT, TEMPLATE, "--json"), repo), + ] + if HAS_POWERSHELL: + results.append(run(ps_cmd(repo, SCRIPT, TEMPLATE, "-Json"), repo)) + + assert all(result.returncode == 0 for result in results) + assert all( + json_stdout(result)["TEMPLATE_CONTENT"] == expected + for result in results + ) + + +@requires_bash +@pytest.mark.parametrize( + ("entries", "expected"), + [ + ( + [ + ("disabled-pack", {"enabled": False, "priority": 0}), + ("numeric-pack", {"enabled": True, "priority": 2}), + ("string-pack", {"enabled": True, "priority": "1"}), + ], + "# string-pack\n", + ), + ( + [ + ("z-pack", {"enabled": True}), + ("a-pack", {"enabled": True}), + ], + "# a-pack\n", + ), + ( + [ + ("float-pack", {"enabled": True, "priority": 5.9}), + ("six-pack", {"enabled": True, "priority": 6}), + ], + "# float-pack\n", + ), + ( + [ + ("a-huge-pack", {"enabled": True, "priority": 2147483648}), + ("z-default-pack", {"enabled": True, "priority": "invalid"}), + ], + "# z-default-pack\n", + ), + ( + [ + ("decimal-string-pack", {"enabled": True, "priority": "5.9"}), + ("exponent-string-pack", {"enabled": True, "priority": "1e3"}), + ("hex-string-pack", {"enabled": True, "priority": "0x10"}), + ("six-pack", {"enabled": True, "priority": 6}), + ], + "# six-pack\n", + ), + ], + ids=[ + "mixed_priorities", + "equal_priority_id_tiebreaker", + "float_priority", + "large_integer_priority", + "non_integer_numeric_strings", + ], +) +def test_all_variants_normalize_and_tiebreak_preset_priorities( + tmp_path: Path, + entries: list[tuple[str, dict[str, object]]], + expected: str, +) -> None: + repo = make_repo(tmp_path) + install_scripts(repo, SCRIPT) + presets = repo / ".specify" / "presets" + registry: dict[str, object] = {"presets": {}} + registry_presets = registry["presets"] + assert isinstance(registry_presets, dict) + for preset_id, metadata in entries: + template_dir = presets / preset_id / "templates" + template_dir.mkdir(parents=True) + (template_dir / f"{TEMPLATE}.md").write_text( + f"# {preset_id}\n", + encoding="utf-8", + ) + registry_presets[preset_id] = metadata + (presets / ".registry").write_text( + json.dumps(registry, separators=(",", ":")) + "\n", + encoding="utf-8", + ) + + results = [ + run(bash_cmd(repo, SCRIPT, TEMPLATE, "--json"), repo), + run(py_cmd(repo, SCRIPT, TEMPLATE, "--json"), repo), + ] + if HAS_POWERSHELL: + results.append(run(ps_cmd(repo, SCRIPT, TEMPLATE, "-Json"), repo)) + + assert all(result.returncode == 0 for result in results) + assert all( + json_stdout(result)["TEMPLATE_CONTENT"] == expected + for result in results + ) + + +@requires_bash +def test_all_variants_fail_when_wrap_placeholder_is_missing( + tmp_path: Path, +) -> None: + repo = make_repo(tmp_path) + install_scripts(repo, SCRIPT) + templates = repo / ".specify" / "templates" + templates.mkdir(parents=True) + (templates / f"{TEMPLATE}.md").write_text("# Core\n", encoding="utf-8") + preset = repo / ".specify" / "presets" / "wrap-pack" + (preset / "templates").mkdir(parents=True) + (preset / "templates" / f"{TEMPLATE}.md").write_text( + "# Broken wrapper\n", encoding="utf-8" + ) + (preset / "preset.yml").write_text( + "provides:\n" + " templates:\n" + " - type: template\n" + f" name: {TEMPLATE}\n" + f" file: templates/{TEMPLATE}.md\n" + " strategy: wrap\n", + encoding="utf-8", + ) + (repo / ".specify" / "presets" / ".registry").write_text( + '{"presets":{"wrap-pack":{"enabled":true,"priority":1}}}\n', + encoding="utf-8", + ) + + results = [ + run(bash_cmd(repo, SCRIPT, TEMPLATE, "--json"), repo), + run(py_cmd(repo, SCRIPT, TEMPLATE, "--json"), repo), + ] + if HAS_POWERSHELL: + results.append(run(ps_cmd(repo, SCRIPT, TEMPLATE, "-Json"), repo)) + + assert all(result.returncode != 0 for result in results) + assert all(result.stdout == "" for result in results) + + +@requires_bash +def test_all_variants_fail_when_yaml_parser_is_unavailable( + tmp_path: Path, +) -> None: + repo, _ = _setup_repo(tmp_path) + blocker = tmp_path / "blocker" + blocker.mkdir() + (blocker / "yaml.py").write_text( + "raise ImportError('simulated missing PyYAML')\n", + encoding="utf-8", + ) + env = clean_env() + env["PYTHONPATH"] = str(blocker) + + results = [ + run(bash_cmd(repo, SCRIPT, TEMPLATE, "--json"), repo, env), + run(py_cmd(repo, SCRIPT, TEMPLATE, "--json"), repo, env), + ] + if HAS_POWERSHELL: + results.append( + run(ps_cmd(repo, SCRIPT, TEMPLATE, "-Json"), repo, env) + ) + + assert all(result.returncode != 0 for result in results) + assert all(result.stdout == "" for result in results) + + +@requires_bash +def test_bash_fails_when_override_read_fails(tmp_path: Path) -> None: + repo = make_repo(tmp_path) + install_scripts(repo, SCRIPT) + override = repo / ".specify" / "templates" / "overrides" + override.mkdir(parents=True) + (override / f"{TEMPLATE}.md").write_text("# Override\n", encoding="utf-8") + shim_dir = tmp_path / "bin" + shim_dir.mkdir() + cat_shim = shim_dir / "cat" + cat_shim.write_text( + "#!/bin/sh\n" + "case \"$1\" in\n" + " */.specify/templates/overrides/*) exit 1 ;;\n" + "esac\n" + "exec /bin/cat \"$@\"\n", + encoding="utf-8", + ) + cat_shim.chmod(0o755) + env = clean_env() + env["PATH"] = f"{shim_dir}{os.pathsep}{env.get('PATH', '')}" + + result = run(bash_cmd(repo, SCRIPT, TEMPLATE, "--json"), repo, env) + + assert result.returncode != 0 + assert result.stdout == "" + + +@requires_bash +@pytest.mark.parametrize( + "manifest_content", + [ + "provides: [\n", + "", + "provides:\n templates:\n - null\n", + "provides:\n templates: {}\n", + "preset:\n id: wrap-pack\n", + "provides:\n templates: []\n", + f"""provides: + templates: + - type: template + name: {TEMPLATE} + file: null + strategy: wrap +""", + f"""provides: + templates: + - type: template + name: {TEMPLATE} + file: templates/{TEMPLATE}.md + strategy: 123 +""", + f"""provides: + templates: + - type: template + name: {TEMPLATE} + file: templates/{TEMPLATE}.md + strategy: wrap + - type: template + name: unrelated-template + file: null + strategy: append +""", + f"""provides: + templates: + - name: {TEMPLATE} + file: templates/{TEMPLATE}.md + strategy: wrap + - type: template + name: unrelated-template + file: templates/other.md +""", + f"""provides: + templates: + - type: template + name: {TEMPLATE} + file: templates/{TEMPLATE}.md + strategy: wrap + - type: template + name: unrelated-template +""", + f"""provides: + templates: + - type: template + name: {TEMPLATE} + file: templates/{TEMPLATE}.md + strategy: wrap + - type: bogus + name: unrelated-template + file: templates/other.md +""", + f"""provides: + templates: + - type: template + name: {TEMPLATE} + file: templates/{TEMPLATE}.md + strategy: wrap + - type: template + name: unrelated-template + file: templates/other.md + strategy: merge +""", + ], + ids=[ + "invalid_yaml", + "empty_document", + "non_mapping_template_entry", + "non_list_templates", + "missing_provides", + "empty_templates", + "non_string_file", + "non_string_strategy", + "malformed_entry_after_match", + "entry_missing_type", + "entry_missing_file", + "unsupported_type", + "unsupported_strategy", + ], +) +def test_all_variants_fail_for_malformed_preset_manifest( + tmp_path: Path, + manifest_content: str, +) -> None: + repo, _ = _setup_repo(tmp_path) + ( + repo / ".specify" / "presets" / "wrap-pack" / "preset.yml" + ).write_text(manifest_content, encoding="utf-8") + + results = [ + run(bash_cmd(repo, SCRIPT, TEMPLATE, "--json"), repo), + run(py_cmd(repo, SCRIPT, TEMPLATE, "--json"), repo), + ] + if HAS_POWERSHELL: + results.append(run(ps_cmd(repo, SCRIPT, TEMPLATE, "-Json"), repo)) + + assert all(result.returncode != 0 for result in results) + assert all(result.stdout == "" for result in results) diff --git a/tests/test_setup_plan_python_parity.py b/tests/test_setup_plan_python_parity.py index 9d9a67b620..e8372125a3 100644 --- a/tests/test_setup_plan_python_parity.py +++ b/tests/test_setup_plan_python_parity.py @@ -11,7 +11,9 @@ HAS_POWERSHELL, POWERSHELL_EXE, bash_cmd, + break_wrap_layer, clean_env, + install_composition_stack, install_scripts, json_stdout, make_repo, @@ -60,7 +62,57 @@ def test_python_fresh_copy_matches_bash(tmp_path: Path) -> None: ) for repo in (repo_a, repo_b): plan = repo / "specs" / "001-my-feature" / "plan.md" - assert plan.read_text(encoding="utf-8") == TEMPLATE_BODY + assert plan.read_bytes() == TEMPLATE_BODY.encode("utf-8") + + +@requires_bash +def test_all_variants_materialize_composed_plan_template(tmp_path: Path) -> None: + repos = [ + _setup_repo(tmp_path, "bash"), + _setup_repo(tmp_path, "powershell"), + _setup_repo(tmp_path, "python"), + ] + expected = "" + for current in repos: + expected = install_composition_stack( + current, "plan-template", TEMPLATE_BODY + ) + + results = [ + run(bash_cmd(repos[0], SCRIPT, "--json"), repos[0]), + run(py_cmd(repos[2], SCRIPT, "--json"), repos[2]), + ] + checked_repos = [repos[0], repos[2]] + if HAS_POWERSHELL: + results.insert(1, run(ps_cmd(repos[1], SCRIPT, "-Json"), repos[1])) + checked_repos.insert(1, repos[1]) + + assert all(result.returncode == 0 for result in results) + for current in checked_repos: + assert ( + current / "specs" / "001-my-feature" / "plan.md" + ).read_text(encoding="utf-8") == expected + + +@requires_bash +def test_all_variants_fail_for_broken_plan_composition(tmp_path: Path) -> None: + repos = [ + _setup_repo(tmp_path, "bash"), + _setup_repo(tmp_path, "powershell"), + _setup_repo(tmp_path, "python"), + ] + for current in repos: + install_composition_stack(current, "plan-template", TEMPLATE_BODY) + break_wrap_layer(current, "plan-template") + + results = [ + run(bash_cmd(repos[0], SCRIPT, "--json"), repos[0]), + run(py_cmd(repos[2], SCRIPT, "--json"), repos[2]), + ] + if HAS_POWERSHELL: + results.append(run(ps_cmd(repos[1], SCRIPT, "-Json"), repos[1])) + + assert all(result.returncode != 0 for result in results) @requires_bash @@ -119,13 +171,19 @@ def test_python_missing_template_matches_bash(tmp_path: Path) -> None: @requires_bash @pytest.mark.parametrize( - "registry", + ("registry", "expected"), [ - '{"presets": {"alpha": {"priority": "high"}, "beta": {"priority": 1}}}', - '{"presets": {"alpha": {"priority": 2}, "beta": {"priority": 1}, "gamma": {"priority": null}}}', - "[]", - '{"presets":[]}', - '{"presets":null}', + ( + '{"presets": {"alpha": {"priority": "high"}, "beta": {"priority": 1}}}', + "# beta plan\n", + ), + ( + '{"presets": {"alpha": {"priority": 2}, "beta": {"priority": 1}, "gamma": {"priority": null}}}', + "# beta plan\n", + ), + ("[]", "# alpha plan\n"), + ('{"presets":[]}', "# alpha plan\n"), + ('{"presets":null}', "# alpha plan\n"), ], ids=[ "mixed_priorities", @@ -135,10 +193,10 @@ def test_python_missing_template_matches_bash(tmp_path: Path) -> None: "null_presets", ], ) -def test_all_variants_broken_registry_falls_back_to_dir_scan( - tmp_path: Path, registry: str +def test_all_variants_normalize_or_fallback_for_registry( + tmp_path: Path, registry: str, expected: str ) -> None: - """Malformed registries fall back to the alphabetical directory scan.""" + """Priorities normalize canonically; malformed shapes fall back to directories.""" repos = [ _setup_repo(tmp_path, "bash", template=False), _setup_repo(tmp_path, "powershell", template=False), @@ -183,7 +241,7 @@ def test_all_variants_broken_registry_falls_back_to_dir_scan( ) == 1 for _, repo in results: plan = repo / "specs" / "001-my-feature" / "plan.md" - assert plan.read_text(encoding="utf-8") == "# alpha plan\n" + assert plan.read_text(encoding="utf-8") == expected @pytest.mark.skipif(not HAS_POWERSHELL, reason="no PowerShell available") diff --git a/tests/test_setup_tasks.py b/tests/test_setup_tasks.py index 26d1c798eb..a3f02b63a2 100644 --- a/tests/test_setup_tasks.py +++ b/tests/test_setup_tasks.py @@ -719,7 +719,7 @@ def test_setup_tasks_ps_core_template_resolved(tasks_repo: Path) -> None: [exe, "-NoProfile", "-File", str(script), "-Json"], cwd=tasks_repo, capture_output=True, - text=True, + encoding="utf-8", check=False, env=_clean_env(), ) diff --git a/tests/test_setup_tasks_python_parity.py b/tests/test_setup_tasks_python_parity.py index 29d0e2b5aa..1529005380 100644 --- a/tests/test_setup_tasks_python_parity.py +++ b/tests/test_setup_tasks_python_parity.py @@ -10,7 +10,9 @@ from tests.parity_helpers import ( HAS_POWERSHELL, bash_cmd, + break_wrap_layer, clean_env, + install_composition_stack, install_scripts, json_stdout, make_repo, @@ -87,6 +89,42 @@ def test_python_override_template_wins_matches_bash(repo: Path) -> None: assert json_stdout(py)["TASKS_TEMPLATE"].endswith("overrides/tasks-template.md") +@requires_bash +def test_all_variants_return_composed_tasks_template(repo: Path) -> None: + expected = install_composition_stack( + repo, "tasks-template", "# Tasks Template\n" + ) + + results = [ + run(bash_cmd(repo, SCRIPT, "--json"), repo), + run(py_cmd(repo, SCRIPT, "--json"), repo), + ] + if HAS_POWERSHELL: + results.append(run(ps_cmd(repo, SCRIPT, "-Json"), repo)) + + assert all(result.returncode == 0 for result in results) + assert all( + json_stdout(result)["TASKS_TEMPLATE_CONTENT"] == expected + for result in results + ) + + +@requires_bash +def test_all_variants_fail_for_broken_tasks_composition(repo: Path) -> None: + install_composition_stack(repo, "tasks-template", "# Tasks Template\n") + break_wrap_layer(repo, "tasks-template") + + results = [ + run(bash_cmd(repo, SCRIPT, "--json"), repo), + run(py_cmd(repo, SCRIPT, "--json"), repo), + ] + if HAS_POWERSHELL: + results.append(run(ps_cmd(repo, SCRIPT, "-Json"), repo)) + + assert all(result.returncode != 0 for result in results) + assert all(result.stdout == "" for result in results) + + @requires_bash @pytest.mark.parametrize( "missing",