Skip to content

feat(async-config-poll): add BLOCK_BOOT_ON_WATCH to block boot on the config watch#148

Merged
Aditya-eddy merged 1 commit into
mainfrom
async-boot-blocking-watch
Jul 22, 2026
Merged

feat(async-config-poll): add BLOCK_BOOT_ON_WATCH to block boot on the config watch#148
Aditya-eddy merged 1 commit into
mainfrom
async-boot-blocking-watch

Conversation

@Aditya-eddy

@Aditya-eddy Aditya-eddy commented Jul 21, 2026

Copy link
Copy Markdown
Member

What & why

Adds an env-gated BLOCK_BOOT_ON_WATCH mode to ConfigWatchService so the app can boot-block on the config-watch long-poll — the shape that exposes the async value-epoch fix in keploy.

Today the sample's config watch runs on a background daemon (boot never depends on it), so it can't reproduce the real-world boot deadlock a synchronously-blocking config client hits: at replay the old anchor-hold async engine parks a poll whose delivery is anchored past the reachable test window, and the app never becomes ready.

With BLOCK_BOOT_ON_WATCH=true, init() performs a synchronous GET /v1/buckets/app-config?watch=true and blocks boot until it returns:

  • Old (anchor-hold) engine: the poll is parked → the app never accepts connections → ingress tests fail with connection refused (the deadlock).
  • Value-epoch engine: the startup epoch is served immediately → boot proceeds → tests pass.

When the env is unset (the default / record path, and the existing periodic + httppoll scenarios), the watch keeps running on the background daemon exactly as before — no behavior change.

Pairs with

keploy/keploy#4379 (async value-epoch replay). That PR adds a boot_blocking cell to the async-config-poll e2e which records the app normally and replays it with BLOCK_BOOT_ON_WATCH=true, asserting the app boots and the ingress tests pass — i.e. the value-epoch engine serves the startup epoch instead of parking it. That CI cell needs this env support, so this PR should merge first.

🤖 Generated with Claude Code

@Aditya-eddy @claude
feat(async-config-poll): add BLOCK_BOOT_ON_WATCH to block boot on the…
701288f 
… config watch

Env-gated (replay-only) mode: init() does a synchronous GET
/v1/buckets/app-config?watch=true and blocks boot until it returns, the
shape a synchronously-blocking config client hits. Lets the keploy
async-config-poll e2e reproduce the boot deadlock — the old anchor-hold
async engine parks the poll (app never ready) while the value-epoch engine
serves the startup epoch immediately (app boots). When unset (default /
record path, and the periodic + httppoll scenarios) the watch keeps running
on the background daemon, unchanged.

Pairs with keploy/keploy#4379.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: Aditya Sharma <aditya282003@gmail.com>
Copilot AI review requested due to automatic review settings July 21, 2026 09:20
Copilot AI reviewed Jul 21, 2026

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Aditya-eddy added a commit to keploy/keploy that referenced this pull request Jul 21, 2026
@Aditya-eddy @claude
ci(async): add boot_blocking scenario to async-config-poll e2e
5220e0a 
Adds a third async-config-poll cell that records the config watch as a
background poll (as httppoll) but REPLAYS with the app boot-blocking
synchronously on the watch=true poll (BLOCK_BOOT_ON_WATCH). This exercises
the boot-deadlock shape: the value-epoch engine must serve the startup
epoch so the app boots and the ingress tests pass; the old anchor-hold
engine would park the poll and the app would never become ready.

- java-linux.sh: add the boot_blocking scenario (REPLAY_APP_ENV override so
  record and replay can drive the app differently), assert the app's
  boot-watch was served (not parked), and reuse the poll:true/no-pollDurationMs
  record-side check. Also corrects the stale matrix comment.
- java_linux.yml: add boot_blocking to the scenario matrix.

Requires keploy/samples-java#148 (BLOCK_BOOT_ON_WATCH app support) in the
checked-out sample.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: Aditya Sharma <aditya282003@gmail.com>
Aditya-eddy added a commit to keploy/keploy that referenced this pull request Jul 21, 2026
@Aditya-eddy @claude
ci(async): pin samples-java checkout to the BLOCK_BOOT_ON_WATCH branc…
911d758 
…h (temporary)

The boot_blocking async-config-poll cell needs the app's BLOCK_BOOT_ON_WATCH
support (keploy/samples-java#148), which isn't on samples-java's default
branch yet. Pin the async-config-poll job's samples-java checkout to that
branch so the cell is exercised now; repoint to the default branch once #148
merges.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: Aditya Sharma <aditya282003@gmail.com>
@Aditya-eddy
Aditya-eddy merged commit 5ef19b3 into main Jul 22, 2026
2 checks passed
Aditya-eddy added a commit to keploy/keploy that referenced this pull request Jul 22, 2026
@Aditya-eddy @claude
ci(async): checkout samples-java default branch (BLOCK_BOOT_ON_WATCH …
fa63357 
…merged)

keploy/samples-java#148 is merged, so the async-config-poll job no longer
needs the temporary branch pin — checkout the default branch, which now
carries BLOCK_BOOT_ON_WATCH for the boot_blocking scenario.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: Aditya Sharma <aditya282003@gmail.com>
Aditya-eddy added a commit to keploy/keploy that referenced this pull request Jul 22, 2026
@Aditya-eddy @claude
feat(async): value-epoch replay for httpPoll lanes (fix long-poll boo…
f3dadd0 
…t deadlock) (#4379)

* feat(async): per-lane throttleMs with 1s default

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: Aditya Sharma <aditya282003@gmail.com>

* feat(async): ResponseValueKey for record-time change detection

Signed-off-by: Aditya Sharma <aditya282003@gmail.com>

* feat(async): value-epoch selection in Decide (serve current epoch by completed)

Signed-off-by: Aditya Sharma <aditya282003@gmail.com>

* feat(async): throttle-bounded poll hold with wake-early-on-change

Decide now holds each poll up to lane.ThrottleDuration() via holdThrottle,
a bounded cond.Wait that wakes early on an AdvanceWindow/OnTestComplete
broadcast or ctx cancel -- pacing unchanged polls without a busy loop.
Also sets ThrottleMs:10 on the Task 3 tests' lanes so the suite stays
fast under the reintroduced default 1s hold.

Signed-off-by: Aditya Sharma <aditya282003@gmail.com>

* feat(async): record poll lanes as collapsed value epochs (drop pollDurationMs)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: Aditya Sharma <aditya282003@gmail.com>

* test(async): record->replay epoch contract (boot answered, change at anchor)

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Signed-off-by: Aditya Sharma <aditya282003@gmail.com>

* fix(async): gate replay throttle hold to poll lanes only

Engine.Decide called e.holdThrottle unconditionally for every lane with
loaded epochs, so a non-poll async lane (Type not ending in "Poll") paid
up to ThrottleDuration() (default 1s) before serving its current epoch,
even though only poll lanes need pacing against busy-looping an unchanged
long-poll answer. Gate the hold behind lane.IsPoll() so non-poll lanes
serve immediately.

Add TestDecideNonPollLaneSkipsThrottle proving a non-poll lane with
ThrottleMs: 5000 returns in well under 500ms. The three pre-existing
throttle tests (TestDecideHeldUpToThrottle, TestDecideWakesEarlyOnAdvance,
TestDecideReturnsOnCtxCancel) used Type: "fake", which was never actually a
poll lane; switch them to "fakePoll" (BaseType still resolves to "fake")
so they keep exercising real hold/wake-early/ctx-cancel behavior instead
of silently passing without ever entering the hold.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: Aditya Sharma <aditya282003@gmail.com>

* docs(async): fix stale LogReport comment for the value-epoch model

LogReport's doc comment still described an "armed-but-never-polled
(not-exercised)" state, which cannot occur under the value-epoch model:
every epoch is always re-selectable/serveable once effective, so nothing
arms and then goes unpolled. Update the comment to say what the verdict
actually reports (served/shape_flags) and note that not_exercised/held
are retained in the emitted log line only for output-format stability
(the periodic CI cell greps the line) and are always 0 now.

Comment-only: the emitted fields, their names, and Report()'s values are
unchanged, since CI greps "not_exercised": N in the log line.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: Aditya Sharma <aditya282003@gmail.com>

* test(proxy): give TestLoadAsyncMocksForwardsToEngine an explicit ThrottleMs

Its lane (Type: "fake") is non-poll, so IsPoll() gates it out of the
throttle hold entirely now; ThrottleMs: 10 documents that intent
explicitly instead of relying on the omitted-field default (1s) never
being reached. Test still asserts Decide serves "b", the newest
same-AnchorPos epoch.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: Aditya Sharma <aditya282003@gmail.com>

* ci(java): update async_config_poll httppoll assertions for the epoch model

Two httppoll-scenario assertions were written against the retired
open-duration model and no longer hold:

- Record-side: pollDurationMs no longer exists on AsyncMeta, so grep -aq
  'pollDurationMs:' always fails now. Replace with proof that the poll
  mock is recorded as a value epoch: 'poll: true' present AND
  pollDurationMs: absent.

- Replay-side: Engine.Decide no longer holds a poll's connection open
  waiting for its anchor (Report()/LogReport hard-code held=0 under the
  value-epoch model), so held >= 1 always fails now. Replace with the
  served >= 1 proof already established generically above (reusing the
  served shell variable already parsed from the "async egress verdict"
  log line) — proving the recorded poll value was served on replay. The
  periodic cell's checks are untouched.

Updated the surrounding doc comments to match. Verified with `bash -n`
(no syntax error); the Java e2e itself cannot run locally.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: Aditya Sharma <aditya282003@gmail.com>

* review(async): document all-lanes epoch selection; pin non-poll behavior

Address PR #4379 review:
- Major: Decide's current-epoch SELECTION applies to every async lane (poll +
  non-poll), not just poll lanes — documented explicitly, and non-poll
  re-selectable behavior pinned by TestNonPollServesReselectableCurrentEpoch so
  a future change can't silently revert to one-shot delivery.
- holdThrottle comment: guarantee is 'answered within at most d', not 'spaced d
  apart'; shared cond wakes all parked polls; paces all polls (not only unchanged).
- epoch_contract_test: use Type 'fakePoll' so the boot-answered assertion
  genuinely covers the poll holdThrottle path it narrates.
- ResponseValueKey: note the required-interface bump; verified no external
  AsyncParser implementer in keploy-enterprise or k8s-proxy; document that a
  header-only value signal is out of scope.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: Aditya Sharma <aditya282003@gmail.com>

* ci(async): add boot_blocking scenario to async-config-poll e2e

Adds a third async-config-poll cell that records the config watch as a
background poll (as httppoll) but REPLAYS with the app boot-blocking
synchronously on the watch=true poll (BLOCK_BOOT_ON_WATCH). This exercises
the boot-deadlock shape: the value-epoch engine must serve the startup
epoch so the app boots and the ingress tests pass; the old anchor-hold
engine would park the poll and the app would never become ready.

- java-linux.sh: add the boot_blocking scenario (REPLAY_APP_ENV override so
  record and replay can drive the app differently), assert the app's
  boot-watch was served (not parked), and reuse the poll:true/no-pollDurationMs
  record-side check. Also corrects the stale matrix comment.
- java_linux.yml: add boot_blocking to the scenario matrix.

Requires keploy/samples-java#148 (BLOCK_BOOT_ON_WATCH app support) in the
checked-out sample.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: Aditya Sharma <aditya282003@gmail.com>

* ci(async): pin samples-java checkout to the BLOCK_BOOT_ON_WATCH branch (temporary)

The boot_blocking async-config-poll cell needs the app's BLOCK_BOOT_ON_WATCH
support (keploy/samples-java#148), which isn't on samples-java's default
branch yet. Pin the async-config-poll job's samples-java checkout to that
branch so the cell is exercised now; repoint to the default branch once #148
merges.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: Aditya Sharma <aditya282003@gmail.com>

* ci(async): checkout samples-java default branch (BLOCK_BOOT_ON_WATCH merged)

keploy/samples-java#148 is merged, so the async-config-poll job no longer
needs the temporary branch pin — checkout the default branch, which now
carries BLOCK_BOOT_ON_WATCH for the boot_blocking scenario.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: Aditya Sharma <aditya282003@gmail.com>

---------

Signed-off-by: Aditya Sharma <aditya282003@gmail.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot Copilot left review comments
@gouravkrosx gouravkrosx gouravkrosx approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@Aditya-eddy @gouravkrosx