diff --git a/research/tanzu-platform-ai.md b/research/tanzu-platform-ai.md new file mode 100644 index 0000000..2541eb3 --- /dev/null +++ b/research/tanzu-platform-ai.md @@ -0,0 +1,123 @@ +--- +title: "VMware Tanzu Platform — GenAI Tile and Agent Foundations Built on Cloud Foundry" +author: Ruben Koster (@rkoster) +date: 2026-08-24 +tags: [runtime-lifecycle, sandboxing-isolation, identity, inter-agent-comms, observability-governance, ecosystem-survey] +cf_areas: [buildpacks, capi, diego, uaa] +status: draft +sources: + - https://techdocs.broadcom.com/us/en/vmware-tanzu/platform/ai-services/10-0/ai/index.html + - https://techdocs.broadcom.com/us/en/vmware-tanzu/platform/ai-services/10-0/ai/explanation-journaling.html + - https://www.cloudfoundry.org/blog/from-idea-to-production-delivering-an-ai-ready-platform-as-a-service-with-vmware-tanzu-platform/ + - https://investors.broadcom.com/news-releases/news-release-details/broadcom-announces-tanzu-platform-agent-foundations-bringing + - https://blogs.vmware.com/tanzu/scalable-agentic-applications-with-model-context-protocol-mcp/ +--- + +## Summary + +VMware Tanzu Platform is Broadcom's commercial distribution built around Cloud Foundry +(packaged as Tanzu Platform for Cloud Foundry / Tanzu Application Service), layering +proprietary tiles and runtime functionality on top of the open-source project. It is +adapting to AI on two tracks. First, "GenAI on Tanzu Platform" adds LLM hosting as an +ordinary BOSH-managed tile/broker, changing nothing about the underlying app/service model. +Second, "Tanzu Platform agent foundations" (announced April 2026) introduces a "secure-by- +default agentic runtime" on VMware Cloud Foundation that extends buildpacks, secrets, and +networking with agent-specific isolation primitives — functionality added by Broadcom on +top of CF rather than part of the open-source project itself. + +## Key findings + +- **GenAI tile = standard CF tile-and-broker pattern**: "GenAI on Tanzu Platform" hosts + LLMs on Elastic Runtime VMs, lifecycle-managed via Tanzu Operations Manager/BOSH, and + exposes them through the Cloud Foundry Marketplace like any other service — no new + compute primitive. Selling points are privacy (models run inside the customer's own + infrastructure), no token/usage limits (bounded only by hardware), and CPU-only support + for PoC deployments (slower, but no GPU requirement). +- **Choice of inference engine is explicit and documented**: the tile supports both vLLM + and Ollama as backing engines, with a dedicated doc ("Choosing Between vLLM and Ollama") + guiding operators — vLLM for higher-throughput/production GPU serving, Ollama for + simpler/CPU-friendly setups. This operator-facing choice has no obvious CF-broker + equivalent today (most SaaS inference brokers hide the engine entirely). +- **Journaling (beta) as a governance/ML-ops feature**: setting `"store": true` on a + request (OpenAI API convention) logs the request/response pair into a per-instance + transactions journal, exportable in Bedrock or OpenAI formats, explicitly to support + cross-model comparison, evaluation, fine-tuning, and distillation workflows. This + normalizes transaction logging across backends that may not natively support it — a + concrete example of the platform absorbing an ML-ops concern instead of leaving log + capture to each app. +- **"Agent foundations" is a separate, proprietary runtime layer, not just another tile**: + announced April 2026 as a "secure-by-default agentic runtime" on VMware Cloud Foundation + (VCF), it introduces primitives not present in upstream CF: + - **Immutable supply chain**: agent containers are built via buildpacks (not raw + Dockerfiles), giving automatic patching and provenance — reusing the buildpack concept + as the trust boundary for agent code, rather than inventing a new packaging mechanism. + - **Structural secrets isolation**: agents are prevented from reading each other's + credentials at runtime, closing lateral-movement paths; combined with VMware vDefend, + this extends to infra services and external SaaS connections. This goes beyond CF's + current per-app credential scoping. + - **Zero-trust networking and sandboxing**: connectivity to internal systems/models is + deny-by-default; access is granted only via explicit service bindings; pre-defined + resource limits bound "runaway agentic loops" — a new isolation concern (unbounded + agent tool-call loops) that standard app sandboxing doesn't address. +- **Elastic + HA framing, but not clearly durable execution**: the runtime is described as + auto-scaling IaaS resources for both "short-lived and long-running agents," with "four + layers of high availability" and self-healing infrastructure. This is infrastructure-level + resilience (VM/instance-level self-healing), not documented as workflow-level checkpoint/ + replay of agent state — the same gap identified for Heroku (`heroku-ai-platform.md`); it's + unclear from public material whether an agent loop's in-progress state survives a + restart, or whether "self-heals" means only "the platform reschedules a fresh instance." +- **Centralized AI gateway for models, tools, and cost/safety governance**: "Model and + Tools Serving and Brokering" is described as a centralized gateway controlling tool and + model availability, usage, cost, and safety filters across both public models and private + models running on VCF — conceptually similar to Heroku's MCP Toolkit gateway, but framed + explicitly around enterprise governance (cost/safety filtering) rather than just + tool-interop. +- **MCP positioned as the standard integration layer, pre-curated by IT**: Tanzu's MCP + messaging emphasizes MCP as reusable "glue code" — build one MCP server per system of + record (e.g., JIRA) and every agent can consume it, instead of bespoke integration code + per agent. The quick-start experience gives developers a **pre-built agent** with governed + access to models, MCP servers, and marketplace services that IT has pre-curated — a + developer-self-service-within-guardrails model comparable to how CF's marketplace already + works for ordinary services. +- **An installer product ("AI Starter Kit") exists to lower the adoption barrier**: a + fully-automated installer bundling AI middleware and brokered AI services into a small + pre-configured foundation, aimed at platform teams piloting AI capability before + promoting it to developers — suggesting the full stack (GenAI tile + brokered services + + reference architecture) is still considered non-trivial to stand up from scratch. + +## CF relevance + +Because Tanzu Platform is built directly on Cloud Foundry, this note distinguishes what's +already achievable in upstream CF from what Broadcom has added as proprietary +functionality. The GenAI tile shows "LLM as a marketplace service" is achievable in +upstream CF today via an OSBAPI broker — it's a standard tile/broker pattern, not a new +primitive. "Agent foundations" is different: it's a proprietary runtime layer describing +capabilities upstream CF does not currently have — buildpack-built agent containers with +automatic patching, structural secrets isolation between agents, zero-trust +sandboxing/networking for agent loops, and a centralized AI gateway for tool/model +governance. Whether or how any of this maps to upstream CF primitives (Diego, Garden, UAA) +is unclear from the public material and would need further investigation. As with the +Heroku note, durable/resumable agent execution does not appear to be solved here either. + +## Open questions + +- Are "structural secrets isolation" and "zero-trust sandboxing for agent loops" + implemented as extensions to Garden/runc and Diego's networking policies, or as a + separate proprietary control plane layered alongside Diego? Public material is + press-release-level and doesn't specify. +- Does "four layers of high availability" for agents amount to durable, resumable agent + execution (state survives a restart) — or is it standard infra-level self-healing + (instance rescheduling) marketed toward agent workloads? Needs deeper docs once "agent + foundations" documentation matures beyond the announcement. +- Which parts of "agent foundations" (if any) are implemented purely in Broadcom-proprietary + components vs. building on existing upstream CF primitives (buildpacks, Diego, UAA)? +- Is the "centralized AI gateway" (model/tool serving, cost, safety filtering) conceptually + similar to an MCP-gateway-as-a-marketplace-service, and if so, how would that be scoped as + an ordinary CF service broker? +- Journaling captures raw prompts/responses per service instance for fine-tuning/evaluation + — what data-retention and multi-tenant isolation guarantees would an equivalent CF-native + feature need (per space/org opt-in, encryption at rest, retention limits)? +- How does "pre-built agent with pre-curated governed access to models/MCP servers/ + marketplace services" map onto CF's existing space/org RBAC and marketplace visibility + controls — is this achievable today with existing UAA scopes and service-plan visibility, + or does it need a new governance primitive?