feat: add AgentTemplate preparation and immutable revisions - #2434
Merged
Conversation
Signed-off-by: Eitan Yarmush <eitan.yarmush@solo.io>
Contributor
There was a problem hiding this comment.
Pull request overview
Adds an AgentTemplate “preparation” pipeline that compiles Harness+AgentTemplate attachments into deterministic runtime bundles, provisions immutable Substrate ActorTemplate revisions, and persists/maintains revision tracking in Postgres (including cleanup of unreferenced revisions). The PR also updates runtime contracts for Substrate v0.0.11 (gRPC on port 80, readiness-only endpoint on 8081) and aligns both Go/Python ADKs with credential placeholder expansion.
Changes:
- Compile
v1alpha3Harness+AgentTemplateinto a backend-neutral prepared bundle and derive an immutable revision hash. - Provision immutable Substrate
ActorTemplaterevisions and track desired/latest-successful revisions in Postgres via new migrations + sqlc queries. - Add credential placeholder expansion when materializing config from env vars, plus readiness endpoint changes (port 8081) across Go and Python runtimes.
Reviewed changes
Copilot reviewed 26 out of 30 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| python/packages/kagent-adk/tests/unittests/test_config_materialize.py | Adds unit coverage for credential placeholder expansion during config materialization. |
| python/packages/kagent-adk/src/kagent/adk/cli.py | Updates substrate comment to reflect env-injected config (not secret-backed mounts). |
| python/packages/kagent-adk/src/kagent/adk/_config_materialize.py | Expands __KAGENT_ENV[...]__ placeholders in KAGENT_CONFIG_JSON before writing config files. |
| python/packages/kagent-adk/src/kagent/adk/_a2a.py | Adds a dedicated readiness listener on port 8081 via app lifespan. |
| go/go.sum | Updates dependency checksums (notably Substrate v0.0.11). |
| go/go.mod | Updates Substrate replacement to v0.0.11 and adjusts OTel grpc instrumentation dependency. |
| go/core/pkg/sandboxbackend/substrate/prepared_template.go | Introduces prepared immutable ActorTemplate creation for a given revision/bundle. |
| go/core/pkg/sandboxbackend/substrate/prepared_template_test.go | Adds tests ensuring prepared templates preserve SecretKeyRefs and avoid creating config Secrets. |
| go/core/pkg/sandboxbackend/substrate/openclaw.go | Adapts actor status IP reporting to the new worker assignment fields. |
| go/core/pkg/sandboxbackend/substrate/lifecycle_shared.go | Adjusts env var sanitization to correctly preserve literal values (including empty). |
| go/core/pkg/migrations/core/000008_prepared_revisions.up.sql | Adds prepared_revision and agent_template_attachment tables + indexes for revision tracking. |
| go/core/pkg/migrations/core/000008_prepared_revisions.down.sql | Drops prepared revision tracking tables. |
| go/core/pkg/app/app.go | Wires in the new AgentTemplateController when the substrate lifecycle is enabled and DB supports it. |
| go/core/internal/service/system/service.go | Updates Substrate actor fields to match the newer API (worker assignment, latest snapshot). |
| go/core/internal/preparation/bundle.go | Adds bundle model + stable revision hashing for prepared runtime inputs. |
| go/core/internal/preparation/bundle_test.go | Adds unit test asserting revision changes with runtime input changes. |
| go/core/internal/database/queries/prepared_revisions.sql | Adds sqlc queries for upserting attachments/revisions, marking success, retirement, and GC. |
| go/core/internal/database/prepared_revisions.go | Implements PreparedRevisionStore on Postgres client using generated queries. |
| go/core/internal/database/gen/querier.go | Extends sqlc Querier interface to include prepared revision operations. |
| go/core/internal/database/gen/prepared_revisions.sql.go | Adds generated sqlc code for prepared revision queries. |
| go/core/internal/database/gen/models.go | Adds generated model structs for new tables. |
| go/core/internal/controller/translator/agent/template.go | Refactors prompt source resolution and expands template context to include AgentTemplate name. |
| go/core/internal/controller/translator/agent/agenttemplate_compiler.go | Adds compilation of AgentTemplate+Harness into prepared bundles, including credential placeholders and egress destinations. |
| go/core/internal/controller/translator/agent/agenttemplate_compiler_test.go | Adds test ensuring prepared config/snapshot does not include secret values and includes placeholders/env refs. |
| go/core/internal/controller/translator/agent/adk_api_translator.go | Extends translator interface to expose CompileAgentTemplate. |
| go/core/internal/controller/agenttemplate_controller.go | Adds controller to manage preparation lifecycle, status, DB persistence, and revision GC. |
| go/core/internal/controller/agenttemplate_controller_test.go | Adds controller test validating latest-success semantics until backing template is ready. |
| go/adk/pkg/config/config_materialize.go | Adds placeholder expansion for KAGENT_CONFIG_JSON when writing config files. |
| go/adk/pkg/config/config_materialize_test.go | Adds unit test for Go placeholder expansion behavior. |
| go/adk/pkg/a2a/server/server.go | Adds a dedicated readiness HTTP server on port 8081 and shuts it down gracefully. |
Files not reviewed (3)
- go/core/internal/database/gen/models.go: Generated file
- go/core/internal/database/gen/prepared_revisions.sql.go: Generated file
- go/core/internal/database/gen/querier.go: Generated file
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Comment on lines
+211
to
+223
| async def handle(reader: asyncio.StreamReader, writer: asyncio.StreamWriter): | ||
| request = await reader.readline() | ||
| status = b"200 OK" if request.startswith(b"GET /readyz ") else b"404 Not Found" | ||
| writer.write(b"HTTP/1.1 " + status + b"\r\nContent-Length: 2\r\nConnection: close\r\n\r\nOK") | ||
| await writer.drain() | ||
| writer.close() | ||
| await writer.wait_closed() | ||
|
|
||
| @asynccontextmanager | ||
| async def lifespan(app: FastAPI): | ||
| server = await asyncio.start_server(handle, host="::", port=8081) | ||
| try: | ||
| yield |
Comment on lines
+34
to
+35
| if env_key == "KAGENT_CONFIG_JSON" and "__KAGENT_ENV[" in value: | ||
| value = json.dumps(_expand_config_env(json.loads(value)), separators=(",", ":")) |
Comment on lines
+194
to
+198
| go func() { | ||
| if err := s.readyServer.ListenAndServe(); err != nil && err != http.ErrServerClosed { | ||
| s.listenErr <- err | ||
| } | ||
| }() |
Comment on lines
+111
to
+115
| for _, revision := range revisions { | ||
| if err := r.Lifecycle.DeletePreparedTemplate(ctx, revision.BackingResource); err != nil { | ||
| return err | ||
| } | ||
| if err := r.Store.DeleteUnreferencedPreparedRevision(ctx, revision.Revision); err != nil { |
Signed-off-by: Eitan Yarmush <eitan.yarmush@solo.io>
Signed-off-by: Eitan Yarmush <eitan.yarmush@solo.io>
Signed-off-by: Eitan Yarmush <eitan.yarmush@solo.io>
Signed-off-by: Eitan Yarmush <eitan.yarmush@solo.io>
Signed-off-by: Eitan Yarmush <eitan.yarmush@solo.io>
Signed-off-by: Eitan Yarmush <eitan.yarmush@solo.io>
Signed-off-by: Eitan Yarmush <eitan.yarmush@solo.io>
Signed-off-by: Eitan Yarmush <eitan.yarmush@solo.io>
Signed-off-by: Eitan Yarmush <eitan.yarmush@solo.io>
Signed-off-by: Eitan Yarmush <eitan.yarmush@solo.io>
Signed-off-by: Eitan Yarmush <eitan.yarmush@solo.io>
Signed-off-by: Eitan Yarmush <eitan.yarmush@solo.io>
Signed-off-by: Eitan Yarmush <eitan.yarmush@solo.io>
Signed-off-by: Eitan Yarmush <eitan.yarmush@solo.io>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Testing