Skip to content

feat(live-runner): session-owned payment lifecycle for streamed sessions - #31

Closed
rickstaa wants to merge 7 commits into
ja/live-runnerfrom
rs/live-runner-session-payments
Closed

feat(live-runner): session-owned payment lifecycle for streamed sessions#31
rickstaa wants to merge 7 commits into
ja/live-runnerfrom
rs/live-runner-session-payments

Conversation

@rickstaa

@rickstaa rickstaa commented Jun 17, 2026

Copy link
Copy Markdown
Member

Streamed live-runner sessions must keep paying for as long as the session is open; this PR makes LiveRunnerSession own that lifecycle. Ready for review against ja/live-runner.

What this adds

Streamed live-runner sessions (trickle / websocket / SSE / plain HTTP against app_url) must keep paying for as long as the session is open. After the reservation 402, go-livepeer holds the session as a prepaid balance: a server-side ticker (-livePaymentInterval, 5s default) debits the balance and releases the session once it runs dry. Unlike call_runner (request/response, where the orchestrator pulls payment via a 402 inline), a held-open transport has no request to attach payment to, so the client pushes on a cadence below the server tick.

This makes LiveRunnerSession own that lifecycle so callers do not hand-manage a background task. On-chain sessions are self-funding by default: reserve_session() starts the payment loop automatically, and the loop stops itself once the orchestrator reports the session gone.

Commits

  • 3b60852 surface payment session and add run_session_payments (the interval payment loop).
  • 37ee769 make the session own start/stop of that loop.
  • 81ee7be handle the 482 skip-payment gate and anchor the interval to the server tick.
  • b817c09 review follow-up: session-scoped payments + liveness, TLS-consistent sender, stop_payments(), auto-started self-funding sessions, cadence from payment_interval_ms.
  • f0b9056 merge ja/live-runner (SSE support, feat: add SSE/chunked streaming to call_runner #25) with the stream path carrying the same payment semantics as the JSON path (see below).
  • cc1b21a review-readiness cleanups: 403 + stream-semantics tests, drop dead _live_runner_session_from_json, echo example on the new lifecycle, pytest dev dep.

Changes

  • Session-scoped payments and liveness (review pt. 1): the loop pays {control_url}/payment (from the reserve response, go-livepeer #3938) instead of the blind /payment credit endpoint. A 404 means the orchestrator released the session: the loop stops and surfaces it via session.released / await session.wait_released(); aclose() then skips the redundant stop call. 409 (fixed-price session) stops follow-up payments quietly. Orchestrators that predate control_url fall back to the generic endpoint.
  • TLS consistency (review pt. 2): _post_empty is hoisted into http.py as post_empty (TCPConnector(ssl=False) like every other helper, typed status errors via the shared error mapper) and LivePaymentSession.send_payment routes through it — one place to distinguish 404/409/403.
  • stop_payments() (review pt. 3): symmetric complement to start_payments(); cancels only the loop, session stays reserved, restartable for drain/hand-off flows.
  • Self-funding sessions: reserve_session(auto_pay=True) starts payments at reserve time (matching Scope's connect() and this SDK's register_runner() heartbeat pattern), so a bare session = await reserve_session(...) is no longer silently released ~5s later. auto_pay=False restores the manual lifecycle.
  • Cadence from the challenge: payment_interval now defaults to 60% of the orchestrator's advertised payment_interval_ms (5s tick → 3s cadence), falling back to 3s when absent. Reporting the interval is go-livepeer #4001 (open); the SDK side is forward/backward compatible either way. Explicit payment_interval still wins.
  • Merged ja/live-runner SSE support (feat: add SSE/chunked streaming to call_runner #25): the merge resolution deliberately extends the payment semantics to the stream=True path — fixed-price streams drop payment_session after the single upfront payment (same as the JSON path), and LiveRunnerCallStream now carries session_id (the challenge manifest id) and server_payment_interval. This is scoped plumbing for the follow-up below, pinned by a dedicated test.

Usage

# default: self-funding; payments stop automatically if the session is released
async with await reserve_session(app=APP, signer_url=signer) as session:
    # ask the app to set up its stream; the app responds with its trickle URLs
    echo = await post_json(f"{session.app_url}/echo", {"radius": 75})
    # publish/subscribe on the returned URLs, or hold websocket / SSE / HTTP
    # against session.app_url — the session stays funded throughout

# manual lifecycle
session = await reserve_session(app=APP, signer_url=signer, auto_pay=False)
session.start_payments()
try:
    ...
finally:
    await session.aclose()

(Trickle channels themselves are created runner-side via create_trickle_channels, authorized by the session token the orchestrator proxy injects; the client only ever sees the URLs the app hands back.)

Design notes vs lv2v and scope

  • Uses the transport-agnostic interval driver rather than lv2v's per-output-segment sender. The general live-runner path has no single output stream to meter, can create multiple app-defined trickle channels, and must also cover websocket (no segments). The orchestrator meters by time anyway (LivePaymentProcessor on livePaymentInterval), so interval push aligns more directly with the server than per-segment does.
  • Payment delivery is out of band, so one interval loop funds any held-open transport.
  • Silent-drop detection is now handled on the payment path itself (session-scoped 404) rather than requiring an events channel; an events-based fail-fast client remains a possible later layer on top of wait_released().

Out of scope / follow-ups

  • Long-lived single-shot funding (websocket + stream=True): a paid single-shot connection that outlives its prepaid balance is cancelled by the orchestrator once the balance drains. The 402 dance on the websocket upgrade and an auto payment loop for time-priced single-shot connections land in a follow-up PR. This PR only ensures LiveRunnerCallStream carries session_id / payment_session / server_payment_interval so the follow-up can attach run_session_payments via the synthesized session-scoped payment URL ({orch}/apps/{runner_id}/session/{manifest_id}/payment, go-livepeer #4000: single-shot session id == challenge manifest id). Persistent sessions (trickle / websocket / SSE against app_url) are already fully funded by this PR.
  • stop_runner_session via control_url: deferred per earlier review.

Tests

tests/test_live_runner_payments.py (25 cases): offchain no-op, immediate first payment, session-scoped vs generic endpoint routing, 404 → released, 409 fixed-price stop, 403 mismatch stop, transient/HTTP-5xx retry, skip-cycle gate, idempotency, stop_payments (incl. restart), aclose (incl. released skip), async context manager, payment_interval_ms challenge parsing, call_runner(stream=True) payment semantics (challenge session id, cadence, fixed-price drop), reserve_session auto-start / opt-out / derived and explicit intervals. Full suite: 34 passed.

Relates to ENG-130.

🤖 Generated with Claude Code

rickstaa and others added 2 commits June 15, 2026 20:40
reserve_session now returns the LivePaymentSession built during the reserve
payment challenge (previously discarded). Add run_session_payments(session),
a timer-driven loop that calls LivePaymentSession.send_payment on an interval
to keep a long-lived session funded — the orchestrator meters open sessions by
wall-clock time and releases them when the balance runs dry. No-op offchain.

Reusable across any held-open transport (trickle today, websockets next).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add a session-owned payment lifecycle to LiveRunnerSession so streamed
(trickle/websocket) sessions stay funded without the caller hand-managing
a background task. Mirrors the lv2v Lv2vJob.start_payment_sender/close
shape, but uses the transport-agnostic interval driver (run_session_payments)
since the general live-runner path has no single output stream to meter and
must also cover websocket.

- LiveRunnerSession (still frozen) gains start_payments(), aclose(), and
  async context manager support; _payment_task stored via object.__setattr__.
  start_payments is idempotent, a no-op offchain, and warns instead of raising
  when called without a running loop.
- run_session_payments now pays immediately before the first sleep (a cold
  start can leave a long gap after the reservation payment) and logs+retries
  per-cycle failures instead of dying.
- reserve_session threads payment_interval through to the session.

Callers can now do `async with await reserve_session(...) as session:` and get
automatic start/stop, or use start_payments()/aclose() manually.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@linear-code

linear-code Bot commented Jun 17, 2026

Copy link
Copy Markdown

ENG-130

@coderabbitai

coderabbitai Bot commented Jun 17, 2026

Copy link
Copy Markdown

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 3ba6159d-5b60-4775-8f37-449e7febe355

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch rs/live-runner-session-payments

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

…o server tick

Refinements from reviewing the go-livepeer orchestrator design (ai_http.go
ReserveLiveRunnerSession): after the reservation 402, the orchestrator holds
the session as a prepaid balance debited by a server-side ticker
(-livePaymentInterval, 5s default) and silently releases it when underfunded.
The client just keeps crediting out-of-band, which is what run_session_payments
already does. Two corrections:

- Treat HTTP 482 / SkipPaymentCycle as a healthy "balance current" gate (debug,
  keep looping) instead of logging it as a payment failure. The orchestrator
  uses it to prevent overpayment; only genuine errors warn.
- Document that payment_interval must stay at or below the orchestrator's
  livePaymentInterval (5s), which is why the 3s default carries margin.

Add a test asserting a skip cycle does not kill the loop.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@j0sh

j0sh commented Jun 25, 2026

Copy link
Copy Markdown
Collaborator

Thanks for this @rickstaa . I spent some time reviewing this, comparing to how Scope does it and filling in some gaps (in the runner implementation, not your code).

From Claude:

1. Session liveness in the payment loop

The loop currently has no notion of whether the session it's funding still exists, and only stops if aclose is called. There's no other signal from the orchestrator unless the client makes an outbound app request (and 404s there aren't necessarily intercepted by the runner client). The generic /payment endpoint isn't tied to a concrete live-runner session, so it just credits a balance and returns 200 even if a session does not exist.

See livepeer/go-livepeer@e545b98 for a workaround on the go-livepeer-side: it adds a payment endpoint scoped to the session that returns a 404 if the session doesn't exist.

The loop should point to that endpoint and, on a 404, stop sending payments (and surface it as "session released").

To more easily construct the payment URL on the client, a base control URL was added in livepeer/go-livepeer@81f24cf ... take the control_url with a /payment path appended to it. The session stop endpoint can get the same control_url-based treatment, but that needn't happen in this PR.

2. send_payment doesn't disable cert verification like the rest of the SDK

LivePaymentSession.send_payment POSTs via a default aiohttp.ClientSession (normal TLS verification), while every async JSON helper forces TCPConnector(ssl=False) and the sync lv2v sender uses an unverified context. For consistency it might be best to match the existing behavior. The SDK already has the right primitive: _post_empty in live_runner.py does exactly this (POST empty body, raise on >= 400) with TCPConnector(ssl=False). Suggest hoisting _post_empty into http.py (next to post_json, and to avoid the live_runner ← remote_signer import cycle) and routing the payment POST through it. This also gives a single place to distinguish the 404/403 status codes needed.

We can have the SDK verify TLS by default if TOFU is disabled (which it should be for the live runner) but that's a larger change outside the scope of this PR.

3. Symmetrical stop_payments() (optional)

Soft suggestion: it might be nice to have a small stop_payments() that cancels only the payments loop without stopping the session, as a symmetrical complement to start_payment. This might be handy for tests and any "drain / hand off funding" flow. Session tear-down via close already stops payments so this is purely optional.

rickstaa and others added 2 commits July 25, 2026 10:48
…g sessions

Address review feedback on the payment lifecycle:

- Pay via the session-scoped {control_url}/payment endpoint from the reserve
  response. A 404 there means the orchestrator released the session: the loop
  stops and marks the session released (session.released / wait_released());
  409 (fixed-price) stops follow-up payments; aclose() skips the stop call for
  released sessions. Falls back to the generic /payment endpoint when the
  orchestrator predates control_url.
- Hoist _post_empty into http.py as post_empty (ssl=False like every other
  helper, typed status errors) and route LivePaymentSession.send_payment
  through it.
- Add LiveRunnerSession.stop_payments() as the symmetric complement to
  start_payments().
- Make on-chain sessions self-funding: reserve_session() auto-starts the
  payment loop (auto_pay=True) so any transport against app_url stays paid
  for, and derives the cadence from the challenge's payment_interval_ms when
  the orchestrator reports it (60% of the server tick; 3s fallback).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CBkCesUhvPkNquGK48V84c
…ner-session-payments

# Conflicts:
#	src/livepeer_gateway/live_runner.py
@rickstaa

Copy link
Copy Markdown
Member Author

Update 2026-07-26: merged the latest ja/live-runner (89ac0f7, decimal USD per hour pricing) into this branch (6ad5e2f); one conflict in call_runner resolved keeping both the fixed-price payment_session handling and server_payment_interval. All 31 tests pass.

Validated end to end on-chain (Arbitrum mainnet) via the flux-klein example (livepeer/runner-app-examples#35): session reserved and funded through the remote signer, tickets signed at the new price scale, clean release. Once this lands, flux-klein and the streamdiffusion examples can re-pin from rs/live-runner-session-payments to ja/live-runner.

🤖 Generated with Claude Code

https://claude.ai/code/session_01VtVeDdep8i9UrNKbm9yr8G

rickstaa and others added 2 commits July 28, 2026 11:19
…ner-session-payments

Carries the SSE stream path (#25) over this branch's payment semantics:
fixed-price streams drop payment_session after the single upfront
payment, LiveRunnerCallStream gains session_id (the challenge manifest
id) and server_payment_interval, and the aiohttp import lost in the
textual merge is restored.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- cover the 403 mismatch stop and pin call_runner(stream=True) payment
  semantics (challenge session id, cadence, fixed-price drop)
- drop dead _live_runner_session_from_json, which built sessions that
  could never self-fund
- move examples/echo/client.py to the self-funding session lifecycle
  (async with + --signer-url)
- declare pytest in the dev extra

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@rickstaa

Copy link
Copy Markdown
Member Author

Superseded by #53. Thanks for the review here @j0sh — all three points are implemented there, and the shared loop it introduces covers single-shot calls and streams as well as reserved sessions:

  1. Session liveness — payments go to the session-scoped endpoint from Live Runner go-livepeer#3938; a 404 stops the loop and surfaces released. feat(live-runner): keep metered sessions funded (single-shot, streams, reserved sessions) #53 builds that URL from the challenge's orchestrator + manifest_id and the discovered runner_id, which yields the same string as {control_url}/payment but also works for single-shot, which never receives a control URL.
  2. Cert verification_post_empty hoisted into http.py as you suggested, with the payment POST routed through it; that's also what gives the loop typed status codes to branch on.
  3. stop_payments() — on LiveRunnerSession only, since a session outlives its call. A single-shot stream is its request, so aclose() covers that case.

Closing rather than rebasing: this branch predates fixed pricing (89ac0f7), so its payment-type selection would start a loop on fixed-price sessions and hammer 409s, and it predates SSE single-shot (#25). Between that and the loop being replaced by the shared one, there was more rework than diff left to keep.

@rickstaa rickstaa closed this Jul 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Encapsulate live-runner session payments in a stateful session/client object

2 participants