Skip to content

fix(extensions): preserve keep-config leftover on reinstall (data loss)#3380

Open
jawwad-ali wants to merge 4 commits into
github:mainfrom
jawwad-ali:fix/extension-reinstall-preserves-config
Open

fix(extensions): preserve keep-config leftover on reinstall (data loss)#3380
jawwad-ali wants to merge 4 commits into
github:mainfrom
jawwad-ali:fix/extension-reinstall-preserves-config

Conversation

@jawwad-ali

Copy link
Copy Markdown
Contributor

Description

extension remove --keep-config deliberately preserves the top-level *-config.yml / *-config.local.yml in the extension dir and drops the registry entry, so the user can reinstall and keep their configuration. But a subsequent install_from_directory silently destroys it:

if dest_dir.exists():
    shutil.rmtree(dest_dir)   # wipes the preserved config
...
if did_remove:                # only runs on the --force path
    ...restore from .backup...

After a keep-config remove the registry reports not installed, so the reinstall is not a --force removal → did_remove is False → the restore block never runs → the config is gone. Reproduced on main @ 92b7cf7: a user config (api_key: MY-CUSTOMIZED-VALUE) survives remove(keep_config=True) but is wiped by the next reinstall — the file doesn't even exist afterward. This is data loss that directly contradicts the feature's promise.

Fix

Before the rmtree, capture any top-level *-config.yml / *-config.local.yml from the existing dest_dir (in-memory), and write them back after the fresh copytree — using the same *-config filter and symlink guard the --force restore path already applies. The --force .backup restore still takes precedence for that path; this only covers the in-place keep-config leftover.

Testing

New test_reinstall_preserves_keep_config_leftover: install → write custom config → remove(keep_config=True) → reinstall (no force) → assert the custom config survives. Fails before (config file gone after reinstall — verified by source-stash), passes after. Full tests/test_extensions.py: 327 passed, 3 skipped (no regressions; keep_config=False/.backup path unaffected). uvx ruff check clean.

AI Disclosure

  • I did use AI assistance (describe below)

Found and fixed with Claude Code (Claude Fable 5) under my direction. AI traced the keep-config leftover through the reinstall path and found the restore only fires on --force; I reproduced the data loss, verified fail-before/pass-after and the full suite, and reviewed the diff.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes data loss when reinstalling an extension after specify extension remove --keep-config by preserving top-level extension config files across the install directory wipe-and-recopy behavior in ExtensionManager.install_from_directory.

Changes:

  • Capture existing top-level *-config.yml / *-config.local.yml files before removing the destination directory, then restore them after copytree when reinstalling from a directory.
  • Add a regression test ensuring a keep-config leftover survives a reinstall that is not a --force overwrite.
Show a summary per file
File Description
tests/test_extensions.py Adds regression coverage for reinstall-after-keep-config preservation.
src/specify_cli/extensions/init.py Preserves *-config*.yml files across reinstall when the extension directory exists but registry reports not installed.

Review details

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

  • Files reviewed: 2/2 changed files
  • Comments generated: 1
  • Review effort level: Low

Comment thread src/specify_cli/extensions/__init__.py Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review details

  • Files reviewed: 2/2 changed files
  • Comments generated: 1
  • Review effort level: Low

Comment thread src/specify_cli/extensions/__init__.py Outdated
@mnriem

mnriem commented Jul 9, 2026

Copy link
Copy Markdown
Collaborator

Please address Copilot feedback

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review details

  • Files reviewed: 2/2 changed files
  • Comments generated: 1
  • Review effort level: Low

Comment thread src/specify_cli/extensions/__init__.py
@mnriem

mnriem commented Jul 10, 2026

Copy link
Copy Markdown
Collaborator

Please address Copilot feedback and resolve test & lint errors

jawwad-ali and others added 4 commits July 10, 2026 21:34
remove(keep_config=True) intentionally leaves the top-level *-config.yml /
*-config.local.yml in place and drops the registry entry. A subsequent
install_from_directory is not a --force removal (did_remove is False), so
the unconditional 'if dest_dir.exists(): shutil.rmtree(dest_dir)' wiped
the preserved config and the backup-restore path never ran — silently
destroying user configuration the feature promised to keep. Capture the
top-level *-config.yml/*-config.local.yml from an existing dest_dir before
the rmtree and write them back after the fresh copytree, mirroring the
*-config filter the --force restore path already uses.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Per review: config files can hold secrets, so recreating them with
write_bytes() (default perms) could widen access. Capture the original
st_mode alongside the bytes and re-apply it (permission bits) after
restore, matching the --force path's shutil.copy2 mode preservation.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… copy2)

Per review: the restore comment claimed it mirrors shutil.copy2 (mode +
mtime) but only restored permission bits. Capture and re-apply the
original atime/mtime via os.utime alongside the mode, so the keep-config
leftover restore truly matches copy2's timestamp preservation.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…mode/time restore

Two hardening fixes to the reinstall keep-config restore loop:

1. Never write *through* a non-regular file at the config path. If dest_cfg
   is a symlink (copied with symlinks=True, or swapped in by a racing process
   between copytree and the restore), write_bytes() would follow it and
   clobber an arbitrary target outside the extension dir. Drop any symlink
   (unlink, never follow) or stray directory (rmtree) first, so write_bytes()
   always creates a fresh regular file — and skip the entry if the path can't
   be made safe rather than clobber.

2. Restore mode and mtime independently. The single try/except meant a chmod
   failure skipped os.utime (and vice versa); either can succeed on its own,
   so each gets its own guard.

Tests: a symlink-at-path case (proves the write doesn't follow the link to an
external victim; skipped where symlinks need privilege, runs on CI) and a
directory-at-path case (cross-platform: fails/crashes before the guard, passes
after). Behavior for the normal regular-file case is unchanged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@jawwad-ali jawwad-ali force-pushed the fix/extension-reinstall-preserves-config branch from 8bcd0a7 to 83ef225 Compare July 10, 2026 16:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants