Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 16 additions & 5 deletions acceptance/bin/normalize_uc_payload.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,11 +9,16 @@
yes, Azure no), so a committed golden cannot show them. The CLI already classifies them as
backend defaults, so they never affect the plan; dropping them here is output-only.

Managed properties surface in three shapes, all handled by pruning matching keys:
Managed properties surface in these shapes, all handled by pruning matching keys:
- a `properties` map (remote_state.properties, new_state.value.properties, ...);
an emptied `properties` object is then dropped so {} vs absent doesn't diverge.
- a `changes` entry keyed `properties` or `properties['unity.catalog.managed...']`;
an emptied `changes` object is likewise dropped.
- a plan change entry that is purely a managed-default change (keyed `properties` with a
managed-only `remote`, or `properties['unity.catalog.managed...']`). It is dropped
wherever it appears — including at the top level, where `jq '.plan[].changes'` extracts
it without its enclosing `changes` key. An emptied `changes` object is likewise dropped.

Only the managed-default change is dropped, not every `backend_default` skip: a plain
backend default like a volume's `storage_location` is deterministic and stays in the golden.

Any field names passed as arguments are additionally deleted wherever they appear. These
are the volatile server-set fields (created_at, metastore_id, schema_id, ...) each test
Expand Down Expand Up @@ -66,6 +71,10 @@ def prune(node, drop_fields):
{'properties': {'k': 'v'}}
>>> prune({"changes": {"properties": {"action": "skip", "remote": {"unity.catalog.managed.a.defaults.b": "1"}}, "name": {"action": "update"}}}, set())
{'changes': {'name': {'action': 'update'}}}
>>> prune({"comment": {"action": "update"}, "properties": {"action": "skip", "reason": "backend_default", "remote": {"unity.catalog.managed.a.defaults.b": "1"}}}, set())
{'comment': {'action': 'update'}}
>>> prune({"storage_location": {"action": "skip", "reason": "backend_default"}}, set())
{'storage_location': {'action': 'skip', 'reason': 'backend_default'}}
>>> prune([{"metastore_id": "m", "full_name": "c.s"}], {"metastore_id"})
[{'full_name': 'c.s'}]
"""
Expand All @@ -78,8 +87,10 @@ def prune(node, drop_fields):
for key, value in node.items():
if key in drop_fields or managed_re.search(key):
continue
if key == "changes" and isinstance(value, dict):
value = {k: v for k, v in value.items() if not is_managed_change(k, v)}
# Drop a managed-default change wherever it sits: nested under `changes`, or at the
# top level after `jq '.plan[].changes'` has stripped the enclosing `changes` key.
if is_managed_change(key, value):
continue
pruned = prune(value, drop_fields)
# Drop a properties/changes object emptied by managed-key removal so goldens
# don't diverge on {} (cloud that injects) vs absent (cloud that doesn't).
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ title "Set the comment out of band, the way Catalog Explorer does"
MSYS_NO_PATHCONV=1 $CLI api patch "/api/2.1/unity-catalog/catalogs/$CATALOG" --json '{"comment":"set outside the bundle"}' > /dev/null

title "The remote comment is drift, so the plan updates the catalog"
trace $CLI bundle plan --output json | jq '.plan[].changes'
trace $CLI bundle plan --output json | jq '.plan[].changes' | normalize_uc_payload.py

title "Redeploy clears the comment"
trace $CLI bundle deploy
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ title "Set the comment out of band, the way Catalog Explorer does"
MSYS_NO_PATHCONV=1 $CLI api patch "/api/2.1/unity-catalog/schemas/$SCHEMA" --json '{"comment":"set outside the bundle"}' > /dev/null

title "The remote comment is drift, so the plan updates the schema"
trace $CLI bundle plan --output json | jq '.plan[].changes'
trace $CLI bundle plan --output json | jq '.plan[].changes' | normalize_uc_payload.py

title "Redeploy clears the comment"
trace $CLI bundle deploy
Expand Down
Loading