Skip to content

feat(live-runner): keep metered sessions funded (single-shot, streams, reserved sessions) - #53

Open
rickstaa wants to merge 12 commits into
ja/live-runnerfrom
rs/live-runner-payments
Open

feat(live-runner): keep metered sessions funded (single-shot, streams, reserved sessions)#53
rickstaa wants to merge 12 commits into
ja/live-runnerfrom
rs/live-runner-payments

Conversation

@rickstaa

@rickstaa rickstaa commented Jul 30, 2026

Copy link
Copy Markdown
Member

Supersedes #31 and #50, which each solved one half of this.

The problem

The orchestrator debits a metered session (hour/seconds, 720p) every -livePaymentInterval — 5s by default — checks the balance before each debit, and on the first tick it cannot cover releases the session and cancels whatever is in flight. The SDK only ever sent the upfront 402 challenge payment, which buys the signer's preroll: ~10s for seconds pricing, ~60s for 720p. So anything metered outliving the preroll died on a timer:

  • call_runner — the request cancelled mid-flight
  • call_runner(stream=True) — the stream cut
  • reserve_session — the session released; it discarded payment_session outright, so it could never have paid again

Fixed pricing was unaffected: the upfront payment settles the whole bill and the orchestrator starts no ticker.

The shape

The protocol is identical in all three cases — same challenge, same headers, same signer types, same ticker, same session-scoped top-up endpoint. Only the lifetime differs. So the loop lives on LivePaymentSession, which every path already holds, and each caller contributes only its lifecycle:

async def run_payments(self, *, payment_url=None) -> bool:
    """Keep a metered session funded until cancelled or the session ends."""
Holder Starts Stops
in-flight request before the await finally, when the response lands
LiveRunnerCallStream at construction aclose()
LiveRunnerSession at reserve stop_payments() / aclose()

Nothing in the loop knows which one called it.

Addressing the #31 review

Session liveness. Payments go to the session-scoped endpoint, which 404s once the session is gone, instead of the generic /payment that credits a balance and returns 200 whether or not the session exists. On 404 the loop stops and the owner surfaces released.

Reserved sessions take the URL from control_url in the reservation response, as you described. Single-shot calls can't — a challenge carries only payment_params, orchestrator, manifest_id — so they rebuild it from those parts. _session_control_url and _payment_endpoint keep the two derivations agreeing by construction rather than by coincidence.

Cert verification. _post_empty hoisted into http.py as post_empty and the payment POST routed through it, so payments disable cert verification like every other helper instead of using a default ClientSession. As you noted, that's also what gives the loop typed status codes to branch on — without it there is no 404 to detect.

stop_payments(). On LiveRunnerSession, where a session outlives its call and draining is meaningful. Not on the stream: a single-shot stream is its request, so "keep it open but stop paying" isn't a real state — aclose() covers it.

Notes for review

  • No auto_pay flag. For a plain call an opt-out is unusable — the caller is blocked on the await and cannot fund manually, so it would only mean "die at the preroll." Offchain is the no-payments path, and the loop cannot overspend: a payment covers elapsed time, and a call shorter than one interval never pays twice.
  • Terminal rejections are one rule, matching byoc.py: a non-retryable 4xx stops the loop (404 released, 409 fixed price, 403 mismatch), and 408/429 fall through to a retry. Stopping on an unfamiliar 4xx is deliberate — a silent loop minting tickets nobody will honour is worse than a loud stop.
  • payment_interval_ms is deliberately not consumed. It arrives with server: Report payment interval in payment challenges go-livepeer#4001, still open; until then a 3s constant runs against the 5s default tick. There's a TODO on PAYMENT_INTERVAL_S recording the swap.
  • A stream that is opened and never closed keeps paying. Reading to the end and forgetting aclose() self-corrects — the orchestrator releases its session when the request completes, and the next payment 404s — but abandoning an open stream is unbounded. Hence the async context manager.
  • _live_runner_session_from_json is deleted: no callers, and it built sessions that could never fund themselves.
  • post_empty discards the orchestrator's PaymentResult, which carries refreshed ticket params. Both senders already did that before this PR — the SDK refreshes reactively off the signer's 480. Tracked in Payment responses are discarded instead of refreshing ticket params #55, not changed here.
  • feat(live-runner): session-owned payment lifecycle for streamed sessions #31's payment-type selection predates fixed pricing (89ac0f7), so it would have started a loop on fixed-price sessions and hammered 409s. This branch gates on _runner_payment_type, so fixed never starts one.

Testing

tests/test_live_runner_payments.py — 19 cases against a fake orchestrator and signer, driving the real code paths, one group per mode:

  • single-shot metered — pays on cadence during a slow call, stops when the response lands, stops when the request fails, uses the session-scoped URL, falls back to the generic one when runner_id is unknown
  • streamed — funds until aclose(), marks itself released on a 404
  • reserved sessions — the full discovery → reserve → session path: funds itself at its control URL and stops on close, stop_payments() drains without releasing, a 404 marks it released, and a session whose orchestrator reports no control URL still funds itself at the generic endpoint
  • fixed and offchain — never start a loop, in all three modes
  • status handling — 404/409/403 each terminal after one attempt, 482 and 5xx retried
  • endpoint derivation — the reported and rebuilt payment URLs agree, both degrade to "", and ids are escaped so they cannot add path segments

Deliberately framework-free — plain functions and asserts — since the repo has no pytest dependency and no CI. It runs today with uv run python tests/test_live_runner_payments.py, and pytest discovers it unchanged whenever the framework lands.

Still outstanding: an on-chain run on Arbitrum through the remote signer. No bundled example can exercise this — examples/text runs an offchain orchestrator with no price_info, and ai_http.go:781 reserves the session directly without a challenge whenever the price is nil, so validation needs a priced runner on an on-chain node.

🤖 Generated with Claude Code

rickstaa and others added 3 commits July 30, 2026 12:40
Metered live-runner pricing debits a prepaid balance on the
orchestrator's -livePaymentInterval and drops the session on the first
tick it cannot cover, so every metered caller needs the same loop.
Put it on the payment session itself, where both the single-shot and
the reserved-session paths already hold one:

- post_empty moves into http.py (Josh's suggestion on #31) so the
  payment POST disables cert verification like the rest of the SDK and
  raises typed status errors; send_payment routes through it and takes
  a payment_url for the session-scoped endpoint, which 404s once the
  session is gone instead of blind-crediting a balance
- run_payments pays on an interval, returns True when the orchestrator
  reports the session released, and stops on the other terminal
  rejections (409 fixed price, 403 mismatch) rather than minting
  tickets no one will honor

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A metered single-shot call is billed for as long as it runs, but only
the upfront challenge payment was ever sent, so anything outliving the
signer's preroll (~10s for seconds pricing) was cancelled mid-flight by
the orchestrator. Hold the funding for exactly as long as the work:

- a plain call funds itself while the request is in flight and stops
  the moment the response lands
- a streamed call hands the funding to LiveRunnerCallStream, since the
  stream outlives the call; aclose() stops it, and a released session
  surfaces as stream.released
- payments go to the session-scoped endpoint, built from the challenge
  and the discovered runner id, so a dead session is reported instead
  of silently credited

Fixed pricing is settled upfront and never starts a loop. Offchain
calls have no payment session, so they never start one either.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…are held

reserve_session paid the reservation challenge and then dropped the
payment session on the floor, so a metered session was released by the
orchestrator seconds later no matter what the client did with it. The
session now owns its funding from the moment it is reserved:

- payments run against the session-scoped endpoint until the session is
  closed; a 404 means the orchestrator let it go, surfaced as
  session.released
- stop_payments() drops funding without releasing the session, for
  draining or handing funding elsewhere; aclose() and the async context
  manager do the same on the way out
- drop _live_runner_session_from_json, which had no callers and built
  sessions that could never fund themselves

Fixed-price and offchain reservations carry no payment session, so they
are unaffected.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Jul 30, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: ddcb3dee-e6b8-4145-bfa3-72ab032761f7

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Binding PAYMENT_INTERVAL_S as a default argument freezes it at import,
so overriding the module constant has no effect. Resolve it inside
run_payments instead, which keeps the cadence adjustable in one place.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@rickstaa
rickstaa force-pushed the rs/live-runner-payments branch from 8bc5f73 to 08a52e6 Compare July 30, 2026 11:14
livepeer/go-livepeer#4001 adds payment_interval_ms to payment
challenges. Until it lands the client guesses a cadence against the
orchestrator's default debit interval, so record what replaces the
guess.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
rickstaa and others added 4 commits July 30, 2026 13:34
The reservation response reports the session's control URL, which the
orchestrator builds as ServiceURI/apps/{runner}/session/{session}; its
/payment path is the endpoint the funding loop wants. Use it directly
for reserved sessions rather than the endpoint call_runner derives, so
the client follows the orchestrator's own routing instead of
reconstructing it. Single-shot calls keep deriving it, since a payment
challenge carries no control URL.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
CancelledError has inherited from BaseException since Python 3.8, and
this package requires 3.12, so the broadest handler in the loop cannot
swallow it. Catching it only to re-raise added nothing.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Say what drives the loop (the orchestrator debits on its own interval
and drops the session on the first tick it cannot cover), what
payment_url buys over the generic endpoint, and what the two return
values mean.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The old comment listed only the three known terminal codes, which read
as if they were the only ones that stop the loop. Lead with the rule
instead: a 4xx will not change on a retry, and 408 and 429 are the two
that ask to be retried anyway.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@rickstaa
rickstaa force-pushed the rs/live-runner-payments branch from bf00eea to b486fa4 Compare July 30, 2026 12:12
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@rickstaa
rickstaa marked this pull request as ready for review July 30, 2026 13:52
Behaviour:
- reserve_session no longer falls back to a payment endpoint derived
  from the challenge. An orchestrator reporting no control URL gets
  paid at its generic /payment, where before it would have been sent
  to a route it may not serve and answered with a 404 the loop reads
  as a released session.
- send_payment rejects a payment the signer returned without segCreds
  instead of sending an empty segment header, which fails the
  orchestrator's sig check and comes back 403 - again read as a dead
  session rather than a bad signer.

Shape:
- split the endpoint derivation into _session_control_url and
  _payment_endpoint, so one place knows a payment endpoint sits under
  a session's control URL and the reported and derived URLs agree by
  construction
- drop LiveRunnerCallResult.payment_url, which no longer has a
  consumer, and LiveRunnerSession derives its own from control_url
- give LiveRunnerCallStream the same _start_payments as the session
- drop run_payments' interval_s parameter; the cadence is the module
  constant until the orchestrator reports one
- keep the stream's new fields after the existing ones so its
  constructor call stays positional
- reuse selection.py's own _string_value for the control URL

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@rickstaa

Copy link
Copy Markdown
Member Author

@j0sh tests are a bit hacky we can also remove them and handle all tests in #56.

@rickstaa
rickstaa force-pushed the rs/live-runner-payments branch from c3ef421 to d448606 Compare July 30, 2026 17:37
The signer's opening payment bounds it, not the orchestrator's debit
interval: both sides bill by elapsed time at the same rate. Drops the
pointer to livepeer/go-livepeer#4001, closed for the same reason.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant