Release/1.16.0

.github/ACTIONS-REFERENCE.md

@@ -43,6 +43,7 @@ GitHub provides two mechanisms for storing configuration values:
 | CDK_FILE_UPLOAD_CORS_ORIGINS | Variable | No | None | Platform | Additional CORS origins for the file upload S3 bucket only (appended to global CORS origins) |
 | CDK_FILE_UPLOAD_MAX_SIZE_MB | Variable | No | `10` | Platform | Maximum file upload size in megabytes |
 | CDK_FINE_TUNING_CORS_ORIGINS | Variable | No | None | SageMaker Fine-Tuning | Additional CORS origins for the fine-tuning S3 bucket only (appended to global CORS origins) |
+| CDK_FINE_TUNING_ENABLED | Variable | No | `true` | App API | Mounts the `/fine-tuning` and `/admin/fine-tuning` routers (sets the container's `FINE_TUNING_ENABLED`). Default ON; set to `false` as a kill switch. Storage and IAM are provisioned either way, so switching off never orphans datasets or trained models. |
 | CDK_FINE_TUNING_DEFAULT_QUOTA_HOURS | Variable | No | `0` | App API | Default monthly GPU-hour quota for all authenticated users. `0` = whitelist-only (admin must grant each user). Positive value (e.g. `5`) = open access with that default budget. |
 | CDK_FRONTEND_BUCKET_NAME | Variable | No | None | Frontend | S3 bucket name for frontend assets (defaults to generated name with account ID) |
 | CDK_FRONTEND_CORS_ORIGINS | Variable | No | None | Frontend | Additional CORS origins for the frontend SSM export only (appended to global CORS origins) |

.github/docs/deploy/step-03-github-config.md

@@ -109,6 +109,28 @@ The per-origin cert vars below are **optional overrides** — set one only if yo
 | `CDK_ARTIFACTS_EXTRA_FRAME_ANCESTORS` | — | Comma-separated extra origins (beyond `https://{CDK_DOMAIN_NAME}`) allowed to embed artifact iframes via CSP `frame-ancestors` — applied to both the CloudFront response-headers policy and the render Lambda. Set to `http://localhost:4200` to point a local SPA at this deployment. **Leave unset in production**: every listed origin can frame your users' artifacts (still render-token gated, but a real loosening on a shared environment). |
 | `CDK_MCP_SANDBOX_EXTRA_FRAME_ANCESTORS` | — | Comma-separated extra origins (beyond `https://{CDK_DOMAIN_NAME}`) allowed to embed the MCP Apps sandbox proxy via CSP `frame-ancestors`. Set to `http://localhost:4200` to point a local SPA at this deployment. **Leave unset in production.** |
 | `CDK_FINE_TUNING_CORS_ORIGINS` | — | Comma-separated extra CORS origins for the SageMaker fine-tuning data bucket, beyond `https://{CDK_DOMAIN_NAME}`. Optional — fine-tuning itself is always provisioned. |
+| `CDK_FINE_TUNING_ENABLED` | `true` | Mounts the fine-tuning routers. Default ON — leave unset unless you want the kill switch. Note this is a *runtime* flag on the app-api container; the identically-named CDK stack gate was removed in the single-stack migration. |
+| `CDK_FINE_TUNING_DEFAULT_QUOTA_HOURS` | `0` | Monthly GPU-hour quota auto-granted to any authenticated user. `0` = whitelist-only (an admin grants each user). A positive value (e.g. `10`) = open access with that budget. |
+
+### Managed Knowledge Bases
+
+Every variable below is **optional**. Leave them all unset for the shipped state: the managed knowledge-base backend is deployed but **dormant** — no knowledge base is created managed, no migration runs, and the daily reconciler reports what it *would* delete without deleting anything.
+
+The three flags are independent opt-ins that each default to **off**. An unset GitHub Variable arrives at the deploy as an empty string, which is read as off — so forgetting one never silently arms it.
+
+| Variable Name | Default | Description |
+|---------------|---------|-------------|
+| `CDK_MANAGED_KB_NEW_DEFAULT` | `false` | Set to `true` so newly created knowledge bases are provisioned on the managed backend instead of the legacy one. Existing knowledge bases are untouched. |
+| `CDK_MANAGED_KB_MIGRATION_ENABLED` | `false` | Set to `true` to let the background migration worker run at all. While unset, the worker performs no work and its schedule stays disabled. |
+| `CDK_MANAGED_KB_RECONCILER_ARMED` | `false` | Set to `true` to let the daily reconciler **delete** orphaned knowledge bases. While unset the reconciler still runs and still logs every deletion it intends to make — review those logs before arming it. |
+| `CDK_MANAGED_KB_PER_OWNER_BYTES` | `104857600` (100 MB) | Per-owner stored-bytes cap for the standard role tier, **in bytes**. Deliberately below the 1 GB user-files precedent: at 30,000 users a 1 GB cap permits 30 TB. |
+| `CDK_MANAGED_KB_PER_OWNER_ELEVATED_BYTES` | `1073741824` (1 GB) | Per-owner cap for the elevated, admin-granted tier, **in bytes**. |
+| `CDK_MANAGED_KB_PER_KB_CEILING_BYTES` | `524288000` (500 MB) | Ceiling for any single knowledge base, **in bytes**, bounding one runaway corpus inside an owner's allowance. |
+| `CDK_MANAGED_KB_RETENTION_WINDOW_DAYS` | `30` | How long legacy vector data is kept after a knowledge base is promoted to the managed backend, **in days**, so a rollback stays possible. Do not set below `30`. |
+| `CDK_MANAGED_KB_STORAGE_ALARM_GB` | `500` | CloudWatch alarm threshold for **fleet-wide** managed knowledge base storage, **in GB**. The per-owner caps above bound one user; this is the only thing that bounds the whole account. |
+| `CDK_MANAGED_KB_DAILY_COST_ALARM_USD` | `100` | CloudWatch alarm threshold for the rolled-up daily Knowledge-Base cost, **in USD**. Set alongside the storage alarm — per-owner caps alone permit roughly two orders of magnitude more spend than expected usage. |
+
+> Accepted values for the three flags are `true`, `false`, `1`, `0`, or empty (empty means off). Anything else fails fast at deploy time with a message naming the variable.
 
 ---
 

.github/workflows/backend.yml

@@ -237,6 +237,89 @@ jobs:
       - name: Deploy kb-sync worker image
         run: bash scripts/build/deploy-image-lambda-one.sh kb-sync-worker
 
+  build-kb-migration:
+    name: Build kb-migration image
+    needs: test-backend
+    # Native ARM64 runner — all four kb-migration Lambdas are arm64 (see the
+    # managed-kb CDK construct), matching the kb-sync pattern.
+    runs-on: ubuntu-24.04-arm
+    environment: ${{ (github.ref == 'refs/heads/main' && 'production') || 'development' }}
+
+    permissions:
+      id-token: write
+      contents: read
+
+    env:
+      CDK_AWS_REGION: ${{ vars.AWS_REGION }}
+      CDK_AWS_ACCOUNT: ${{ vars.CDK_AWS_ACCOUNT }}
+      CDK_PROJECT_PREFIX: ${{ vars.CDK_PROJECT_PREFIX }}
+      AWS_REGION: ${{ vars.AWS_REGION }}
+      AWS_ACCOUNT_ID: ${{ vars.CDK_AWS_ACCOUNT }}
+      AWS_ROLE_ARN: ${{ secrets.AWS_ROLE_ARN }}
+      AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
+      AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
+
+    outputs:
+      image_tag: ${{ steps.build.outputs.image_tag }}
+
+    steps:
+      - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
+      - uses: ./.github/actions/build-and-push-image
+        id: build
+        with:
+          image-name: kb-migration
+          aws-region: ${{ vars.AWS_REGION || 'us-west-2' }}
+          aws-role-arn: ${{ secrets.AWS_ROLE_ARN }}
+          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
+          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
+
+  deploy-kb-migration-code:
+    name: Deploy kb-migration Lambda images
+    # ONE image, FOUR functions: dispatcher, worker, reconciler and ingestion
+    # consumer share the kb-migration image and differ only in
+    # ImageConfig.Command (CDK-owned), so a single job points all four at the
+    # freshly-built tag.
+    #
+    # This is the job that replaces the bootstrap stub
+    # (infrastructure/bootstrap-assets/kb-migration/) with the real handlers.
+    # Until it has run once, an enrolled knowledge base sits in `shadow` while
+    # the dispatcher ticks into a no-op — safe, because the work keys are sparse
+    # and the first real tick picks up everything that accumulated.
+    needs: [build-kb-migration, test-backend]
+    runs-on: ubuntu-24.04
+    environment: ${{ (github.ref == 'refs/heads/main' && 'production') || 'development' }}
+
+    permissions:
+      id-token: write
+      contents: read
+
+    env:
+      CDK_AWS_REGION: ${{ vars.AWS_REGION }}
+      CDK_AWS_ACCOUNT: ${{ vars.CDK_AWS_ACCOUNT }}
+      CDK_PROJECT_PREFIX: ${{ vars.CDK_PROJECT_PREFIX }}
+      AWS_REGION: ${{ vars.AWS_REGION }}
+      AWS_ACCOUNT_ID: ${{ vars.CDK_AWS_ACCOUNT }}
+      AWS_ROLE_ARN: ${{ secrets.AWS_ROLE_ARN }}
+      AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
+      AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
+
+    steps:
+      - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
+      - uses: ./.github/actions/configure-aws-credentials
+        with:
+          aws-region: ${{ vars.AWS_REGION || 'us-west-2' }}
+          aws-role-arn: ${{ secrets.AWS_ROLE_ARN }}
+          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
+          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
+      - name: Deploy kb-migration dispatcher image
+        run: bash scripts/build/deploy-image-lambda-one.sh kb-migration-dispatcher
+      - name: Deploy kb-migration worker image
+        run: bash scripts/build/deploy-image-lambda-one.sh kb-migration-worker
+      - name: Deploy kb-migration reconciler image
+        run: bash scripts/build/deploy-image-lambda-one.sh kb-migration-reconciler
+      - name: Deploy kb-migration ingestion consumer image
+        run: bash scripts/build/deploy-image-lambda-one.sh kb-migration-ingestion-consumer
+
   build-scheduled-runs:
     name: Build scheduled-runs image
     needs: test-backend

.github/workflows/platform.yml

@@ -57,6 +57,11 @@ jobs:
       CDK_PROJECT_PREFIX: ${{ vars.CDK_PROJECT_PREFIX }}
       CDK_DOMAIN_NAME: ${{ vars.CDK_DOMAIN_NAME }}
       CDK_CORS_ORIGINS: ${{ vars.CDK_CORS_ORIGINS }}
+      # The Environment tag value. Not cosmetic: it is the filter the managed-KB
+      # reconciler and scripts/teardown/managed-kb.sh match knowledge bases on, so
+      # an unset value here made a dev deploy tag its knowledge bases 'prod' while
+      # teardown looked for 'dev' — matching nothing and reporting success.
+      CDK_TAG_ENVIRONMENT: ${{ vars.CDK_TAG_ENVIRONMENT }}
       CDK_VPC_CIDR: ${{ vars.CDK_VPC_CIDR }}
       CDK_ALB_SUBDOMAIN: ${{ vars.CDK_ALB_SUBDOMAIN }}
       CDK_CERTIFICATE_ARN: ${{ vars.CDK_CERTIFICATE_ARN }}
@@ -128,6 +133,25 @@ jobs:
       # and see the ones their role grants. Which skills a cohort gets is a
       # role's `grantedSkills`, managed in the admin roles UI.
       CDK_SKILLS_ENABLED: ${{ vars.CDK_SKILLS_ENABLED }}
+      # SageMaker fine-tuning. The tables, bucket, SageMaker role and IAM
+      # grants deploy unconditionally, but these two decide whether the feature
+      # is reachable and how it is rationed, and neither was forwarded before —
+      # so every deployed environment served 404s from `/fine-tuning/*` while
+      # the repo variable `CDK_FINE_TUNING_ENABLED` read "true".
+      #
+      # ENABLED is default ON with a kill switch: unset resolves to empty
+      # string, which config.ts treats as the default (on). Set it to "false"
+      # to dark-stop the routes; storage is untouched either way, so no dataset
+      # or trained model is orphaned.
+      #
+      # DEFAULT_QUOTA_HOURS picks the access model: `0` (the default) is
+      # whitelist-only, where an admin grants each user explicitly; a positive
+      # value auto-grants that monthly GPU-hour budget to any signed-in user.
+      CDK_FINE_TUNING_ENABLED: ${{ vars.CDK_FINE_TUNING_ENABLED }}
+      CDK_FINE_TUNING_DEFAULT_QUOTA_HOURS: ${{ vars.CDK_FINE_TUNING_DEFAULT_QUOTA_HOURS }}
+      # Extra CORS origins for the fine-tuning data bucket, beyond the global
+      # CDK_CORS_ORIGINS above. Also never forwarded until now.
+      CDK_FINE_TUNING_CORS_ORIGINS: ${{ vars.CDK_FINE_TUNING_CORS_ORIGINS }}
       # Agent Designer /agents surface. Default OFF (opt-in) until the Phase-4
       # Designer UI ships — a headless API helps no forker. Set the
       # `CDK_AGENTS_API_ENABLED` variable to "true" in an environment (e.g. dev)
@@ -146,6 +170,43 @@ jobs:
       # cdk.context.json stay inert.
       CDK_MCP_TOKEN_ENRICHMENT_ENABLED: ${{ vars.CDK_MCP_TOKEN_ENRICHMENT_ENABLED }}
       CDK_MCP_TOKEN_ENRICHMENT_CLAIMS: ${{ vars.CDK_MCP_TOKEN_ENRICHMENT_CLAIMS }}
+      # Managed knowledge bases (.kiro/specs/managed-kb-migration). THREE
+      # INDEPENDENT OPT-IN flags, all defaulting to OFF — the inverse of the
+      # kill-switch flags above, and the difference matters here. An unset
+      # GitHub Actions variable renders as an EMPTY STRING, not as absent, so
+      # a `!== 'false'` reading of an unset variable would resolve to TRUE and
+      # arm the feature on every fork. config.ts reads these with
+      # parseBooleanEnv, which maps both unset and empty to undefined and falls
+      # through to `false` (Requirement 19.8). Leave all three unset to deploy
+      # the managed backend without starting a fleet migration.
+      #
+      #   CDK_MANAGED_KB_NEW_DEFAULT       new KBs are created managed
+      #   CDK_MANAGED_KB_MIGRATION_ENABLED the background migrator runs at all
+      #   CDK_MANAGED_KB_RECONCILER_ARMED  the daily reconciler DELETES orphans
+      #                                    rather than only reporting them
+      #
+      # reconcilerArmed is the inverted one: the Reconciler is deployed and
+      # running from day one but DISARMED, so its judgement can be reviewed
+      # against real data before it deletes anything (Requirements 14.7, 19.7).
+      CDK_MANAGED_KB_NEW_DEFAULT: ${{ vars.CDK_MANAGED_KB_NEW_DEFAULT }}
+      CDK_MANAGED_KB_MIGRATION_ENABLED: ${{ vars.CDK_MANAGED_KB_MIGRATION_ENABLED }}
+      CDK_MANAGED_KB_RECONCILER_ARMED: ${{ vars.CDK_MANAGED_KB_RECONCILER_ARMED }}
+      # Storage cost controls. Byte_Caps are in BYTES (Requirement 12.2),
+      # defaulting to 100 MB standard / 1 GB elevated / 500 MB per knowledge
+      # base; the retention window is in DAYS and must stay >= 30
+      # (Requirement 15.11). Leave unset to take those defaults — these exist
+      # so an environment can tune them without a code change.
+      CDK_MANAGED_KB_PER_OWNER_BYTES: ${{ vars.CDK_MANAGED_KB_PER_OWNER_BYTES }}
+      CDK_MANAGED_KB_PER_OWNER_ELEVATED_BYTES: ${{ vars.CDK_MANAGED_KB_PER_OWNER_ELEVATED_BYTES }}
+      CDK_MANAGED_KB_PER_KB_CEILING_BYTES: ${{ vars.CDK_MANAGED_KB_PER_KB_CEILING_BYTES }}
+      CDK_MANAGED_KB_RETENTION_WINDOW_DAYS: ${{ vars.CDK_MANAGED_KB_RETENTION_WINDOW_DAYS }}
+      # Fleet-level alarm thresholds (Requirement 12.13). The Byte_Caps above
+      # bound ONE owner; these two bound the whole account, which is the gap
+      # between ~$169/month expected and ~$15,000/month that per-owner caps
+      # alone permit. Storage is in GB (default 500), daily cost in USD
+      # (default 100). Leave unset to take those defaults.
+      CDK_MANAGED_KB_STORAGE_ALARM_GB: ${{ vars.CDK_MANAGED_KB_STORAGE_ALARM_GB }}
+      CDK_MANAGED_KB_DAILY_COST_ALARM_USD: ${{ vars.CDK_MANAGED_KB_DAILY_COST_ALARM_USD }}
       # Secrets
       AWS_ROLE_ARN: ${{ secrets.AWS_ROLE_ARN }}
       AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}

.kiro/specs/managed-kb-migration/.config.kiro

@@ -0,0 +1 @@
+{"specId": "612a1431-367c-427e-8fe0-08872d03a9b9", "workflowType": "design-first", "specType": "feature"}