Skip to content

feat(playwright-1): add reusable Playwright E2E foundation#5792

Merged
BillLeoutsakosvl346 merged 10 commits into
e2e/settings-playwrightfrom
e2e/01-playwright-foundation
Jul 21, 2026
Merged

feat(playwright-1): add reusable Playwright E2E foundation#5792
BillLeoutsakosvl346 merged 10 commits into
e2e/settings-playwrightfrom
e2e/01-playwright-foundation

Conversation

@BillLeoutsakosvl346

Copy link
Copy Markdown
Contributor

Summary

  • add a hermetic Playwright/Chromium foundation with Node 22 workers, guarded CLI projects, per-run pgvector databases, real migrations, production Next.js, and realtime
  • exercise real Better Auth signup/logout/login/storage-state flows while keeping same-origin Sim APIs real and redirecting server-side Stripe calls to a strict loopback fake
  • add fail-closed environment, hostname, database, process, readiness, diagnostics, cleanup, and CI artifact boundaries reusable by future E2E suites
  • add an informational settings E2E CI job parallel to existing Vitest/build jobs

Production-facing deltas

  • centralize Stripe SDK options so both Stripe clients share the guarded E2E transport override; normal deployments remain unchanged when the override is absent
  • make /api/health dynamic and include an E2E run identity only when configured, preventing readiness from accepting another local process
  • add REALTIME_HOST with the existing 0.0.0.0 default; E2E binds realtime to 127.0.0.1

Safety notes

  • E2E child environments start from an allowlist, shadow local .env* keys, use a run-scoped HOME, and disable ambient cloud credential discovery
  • database creation/deletion accepts only numeric-loopback sim_e2e_* targets
  • cleanup uses DROP DATABASE ... WITH (FORCE), requiring PostgreSQL 13+; CI/local guidance pins pgvector/PostgreSQL 17
  • raw Playwright, unsafe project/shard/config/worker overrides, real Stripe transport, and unguarded cleanup paths fail closed

Test plan

  • full production E2E foundation: 13/13 Playwright tests
  • repeat safety: 22/22 with --repeat-each=2
  • filtered navigation-only execution
  • missing-executable/process readiness/port/database/env/Stripe fake safety specs
  • orchestrator-only SIGTERM cleanup with no leaked database or E2E processes
  • bun run type-check in apps/sim
  • bun run type-check in apps/realtime
  • repository bun run lint:check
  • focused auth, billing, health, and navigation Vitest suites
  • Bun 1.3.13 frozen-lockfile install

Scope

This PR targets e2e/settings-playwright, not staging. It is Step 1 of the settings E2E roadmap; persona factories, authorization matrices, and full settings coverage follow in later focused PRs.

Made with Cursor

Bill Leoutsakos added 4 commits July 20, 2026 15:44
Establish a hermetic full-stack browser harness so settings and future surfaces can exercise real authentication, APIs, and isolated database state safely.
Fail closed on stale processes, preserve cleanup during interruption, and keep credentials and personal fixture data out of uploaded diagnostics.
Make destructive database operations and service bindings strictly local while ensuring interrupted runs fully stop managed children before cleanup.
Close the remaining lifecycle, credential-isolation, and CLI escape hatches so future persona and settings suites can build on a fail-closed foundation.
@BillLeoutsakosvl346
BillLeoutsakosvl346 requested a review from a team as a code owner July 21, 2026 01:28
@vercel

vercel Bot commented Jul 21, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
docs Skipped Skipped Jul 21, 2026 2:37am

Request Review

@greptile-apps

greptile-apps Bot commented Jul 21, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR adds a reusable full-stack Playwright E2E foundation. The main changes are:

  • Run-scoped PostgreSQL databases with guarded creation and cleanup.
  • Production Next.js and realtime startup with run-aware readiness checks.
  • Better Auth smoke flows and a strict loopback Stripe fake.
  • Isolated child environments, diagnostics, and an informational CI job.
  • Shared Stripe transport configuration and configurable realtime binding.

Confidence Score: 4/5

Interrupted runs can leave descendant processes or a run database behind.

  • Managed cleanup signals only direct child PIDs.
  • A second signal can terminate the synchronous cleanup path early.
  • Database targeting, environment isolation, readiness identity, and Stripe transport otherwise use strong guards.

apps/sim/e2e/support/process.ts and apps/sim/e2e/scripts/run.ts

Important Files Changed

Filename Overview
apps/sim/e2e/scripts/run.ts Adds orchestration, signal handling, diagnostics, and cleanup for the complete E2E stack.
apps/sim/e2e/support/process.ts Adds managed-process completion and shutdown, but cleanup does not cover descendant processes.
apps/sim/e2e/support/database.ts Adds guarded run-database creation and forced deletion on numeric loopback hosts.
apps/sim/e2e/support/deployment-profile.ts Builds an isolated child environment with guarded database and Stripe settings.
apps/sim/lib/billing/stripe-client-config.ts Centralizes Stripe SDK configuration and restricts E2E overrides to guarded loopback use.
apps/sim/e2e/fakes/stripe/server.ts Implements a loopback Stripe fake with strict authentication, routes, and request tracking.
apps/sim/e2e/support/stack.ts Starts the production services and verifies readiness against the current run identity.
.github/workflows/test-build.yml Adds the informational settings E2E job and failure artifact upload.

Sequence Diagram

%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
    participant O as E2E Orchestrator
    participant DB as Run Database
    participant SF as Stripe Fake
    participant RT as Realtime
    participant APP as Next.js
    participant PW as Playwright
    O->>DB: "Create and migrate sim_e2e_* database"
    O->>SF: Start loopback fake
    O->>RT: Start and verify run ID
    O->>APP: Build, start, and verify run ID
    O->>PW: Run guarded projects
    PW->>APP: Exercise auth and settings flows
    APP->>SF: Send guarded Stripe requests
    O->>RT: Stop process
    O->>APP: Stop process
    O->>SF: Stop and save request log
    O->>DB: Force-drop run database
Loading
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
    participant O as E2E Orchestrator
    participant DB as Run Database
    participant SF as Stripe Fake
    participant RT as Realtime
    participant APP as Next.js
    participant PW as Playwright
    O->>DB: "Create and migrate sim_e2e_* database"
    O->>SF: Start loopback fake
    O->>RT: Start and verify run ID
    O->>APP: Build, start, and verify run ID
    O->>PW: Run guarded projects
    PW->>APP: Exercise auth and settings flows
    APP->>SF: Send guarded Stripe requests
    O->>RT: Stop process
    O->>APP: Stop process
    O->>SF: Stop and save request log
    O->>DB: Force-drop run database
Loading

Comments Outside Diff (1)

  1. apps/sim/e2e/support/process.ts, line 227-228 (link)

    P1 Descendant Processes Survive Cleanup

    When a signal interrupts next build, Playwright, or another command that spawned workers, this code records and signals only the direct child PID. Its descendants can be reparented and continue running, leaving E2E processes or occupied ports on a reusable CI runner.

Reviews (1): Last reviewed commit: "test(e2e): finalize harness safety bound..." | Re-trigger Greptile

Comment on lines +174 to +175
process.once('SIGINT', handleSignal)
process.once('SIGTERM', handleSignal)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Second Signal Aborts Cleanup

A second SIGINT or SIGTERM during the synchronous database and process cleanup uses the OS default action and can terminate both the orchestrator and its foreground cleanup subprocess. A double Ctrl-C or CI termination escalation during the five-second database retry can therefore leave the run-scoped database or child processes behind.

Comment thread apps/sim/e2e/support/hosts.ts
Comment thread apps/sim/e2e/scripts/run.ts Outdated
Use a detached cleanup supervisor so repeated signals cannot interrupt process-group shutdown or forced removal of the guarded run database.
Comment thread apps/sim/e2e/scripts/signal-cleanup.ts Outdated
Comment thread apps/sim/e2e/support/database.ts
Prevent empty process-group targets and verify forced-drop retries against final database state so interrupted runs cannot self-signal or report false cleanup failures.
Comment thread apps/sim/e2e/support/hosts.ts
Comment thread apps/sim/e2e/support/process.ts
Reject dual-stack host mappings that cannot reach IPv4-only services and tolerate child exit races during process-group signal fallback.
Accept safe dual-stack loopback resolution while pinning Chromium and Node traffic to 127.0.0.1 so CI reaches IPv4-only services reliably.
Comment thread apps/sim/e2e/support/process.ts
Mark successful readiness before the managed process completion branch can reject, preventing later normal shutdown from surfacing an abandoned promise failure.

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 47661af. Configure here.

Comment thread apps/sim/e2e/scripts/run.ts
Keep Node-based Playwright requests on the same IPv4 path as Chromium and use direct loopback for orchestrator probes under dual-stack CI DNS.
@BillLeoutsakosvl346
BillLeoutsakosvl346 merged commit bdd0a48 into e2e/settings-playwright Jul 21, 2026
19 checks passed
@icecrasher321 icecrasher321 changed the title Add reusable Playwright E2E foundation feat(playwright-1): add reusable Playwright E2E foundation Jul 21, 2026
@waleedlatif1
waleedlatif1 deleted the e2e/01-playwright-foundation branch July 21, 2026 19:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant