feat(live-runner): keep metered sessions funded (single-shot, streams, reserved sessions) - #53
Open
rickstaa wants to merge 12 commits into
Open
feat(live-runner): keep metered sessions funded (single-shot, streams, reserved sessions)#53rickstaa wants to merge 12 commits into
rickstaa wants to merge 12 commits into
Conversation
Metered live-runner pricing debits a prepaid balance on the orchestrator's -livePaymentInterval and drops the session on the first tick it cannot cover, so every metered caller needs the same loop. Put it on the payment session itself, where both the single-shot and the reserved-session paths already hold one: - post_empty moves into http.py (Josh's suggestion on #31) so the payment POST disables cert verification like the rest of the SDK and raises typed status errors; send_payment routes through it and takes a payment_url for the session-scoped endpoint, which 404s once the session is gone instead of blind-crediting a balance - run_payments pays on an interval, returns True when the orchestrator reports the session released, and stops on the other terminal rejections (409 fixed price, 403 mismatch) rather than minting tickets no one will honor Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A metered single-shot call is billed for as long as it runs, but only the upfront challenge payment was ever sent, so anything outliving the signer's preroll (~10s for seconds pricing) was cancelled mid-flight by the orchestrator. Hold the funding for exactly as long as the work: - a plain call funds itself while the request is in flight and stops the moment the response lands - a streamed call hands the funding to LiveRunnerCallStream, since the stream outlives the call; aclose() stops it, and a released session surfaces as stream.released - payments go to the session-scoped endpoint, built from the challenge and the discovered runner id, so a dead session is reported instead of silently credited Fixed pricing is settled upfront and never starts a loop. Offchain calls have no payment session, so they never start one either. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…are held reserve_session paid the reservation challenge and then dropped the payment session on the floor, so a metered session was released by the orchestrator seconds later no matter what the client did with it. The session now owns its funding from the moment it is reserved: - payments run against the session-scoped endpoint until the session is closed; a 404 means the orchestrator let it go, surfaced as session.released - stop_payments() drops funding without releasing the session, for draining or handing funding elsewhere; aclose() and the async context manager do the same on the way out - drop _live_runner_session_from_json, which had no callers and built sessions that could never fund themselves Fixed-price and offchain reservations carry no payment session, so they are unaffected. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
This was referenced Jul 30, 2026
Binding PAYMENT_INTERVAL_S as a default argument freezes it at import, so overriding the module constant has no effect. Resolve it inside run_payments instead, which keeps the cadence adjustable in one place. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
rickstaa
force-pushed
the
rs/live-runner-payments
branch
from
July 30, 2026 11:14
8bc5f73 to
08a52e6
Compare
livepeer/go-livepeer#4001 adds payment_interval_ms to payment challenges. Until it lands the client guesses a cadence against the orchestrator's default debit interval, so record what replaces the guess. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The reservation response reports the session's control URL, which the
orchestrator builds as ServiceURI/apps/{runner}/session/{session}; its
/payment path is the endpoint the funding loop wants. Use it directly
for reserved sessions rather than the endpoint call_runner derives, so
the client follows the orchestrator's own routing instead of
reconstructing it. Single-shot calls keep deriving it, since a payment
challenge carries no control URL.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
CancelledError has inherited from BaseException since Python 3.8, and this package requires 3.12, so the broadest handler in the loop cannot swallow it. Catching it only to re-raise added nothing. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Say what drives the loop (the orchestrator debits on its own interval and drops the session on the first tick it cannot cover), what payment_url buys over the generic endpoint, and what the two return values mean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The old comment listed only the three known terminal codes, which read as if they were the only ones that stop the loop. Lead with the rule instead: a 4xx will not change on a retry, and 408 and 429 are the two that ask to be retried anyway. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
rickstaa
force-pushed
the
rs/live-runner-payments
branch
from
July 30, 2026 12:12
bf00eea to
b486fa4
Compare
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
rickstaa
marked this pull request as ready for review
July 30, 2026 13:52
Behaviour: - reserve_session no longer falls back to a payment endpoint derived from the challenge. An orchestrator reporting no control URL gets paid at its generic /payment, where before it would have been sent to a route it may not serve and answered with a 404 the loop reads as a released session. - send_payment rejects a payment the signer returned without segCreds instead of sending an empty segment header, which fails the orchestrator's sig check and comes back 403 - again read as a dead session rather than a bad signer. Shape: - split the endpoint derivation into _session_control_url and _payment_endpoint, so one place knows a payment endpoint sits under a session's control URL and the reported and derived URLs agree by construction - drop LiveRunnerCallResult.payment_url, which no longer has a consumer, and LiveRunnerSession derives its own from control_url - give LiveRunnerCallStream the same _start_payments as the session - drop run_payments' interval_s parameter; the cadence is the module constant until the orchestrator reports one - keep the stream's new fields after the existing ones so its constructor call stays positional - reuse selection.py's own _string_value for the control URL Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Member
Author
rickstaa
force-pushed
the
rs/live-runner-payments
branch
from
July 30, 2026 17:37
c3ef421 to
d448606
Compare
The signer's opening payment bounds it, not the orchestrator's debit interval: both sides bill by elapsed time at the same rate. Drops the pointer to livepeer/go-livepeer#4001, closed for the same reason. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
rickstaa
force-pushed
the
rs/live-runner-payments
branch
from
July 30, 2026 17:40
d448606 to
6e9d15c
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Supersedes #31 and #50, which each solved one half of this.
The problem
The orchestrator debits a metered session (
hour/seconds,720p) every-livePaymentInterval— 5s by default — checks the balance before each debit, and on the first tick it cannot cover releases the session and cancels whatever is in flight. The SDK only ever sent the upfront 402 challenge payment, which buys the signer's preroll: ~10s for seconds pricing, ~60s for 720p. So anything metered outliving the preroll died on a timer:call_runner— the request cancelled mid-flightcall_runner(stream=True)— the stream cutreserve_session— the session released; it discardedpayment_sessionoutright, so it could never have paid againFixed pricing was unaffected: the upfront payment settles the whole bill and the orchestrator starts no ticker.
The shape
The protocol is identical in all three cases — same challenge, same headers, same signer types, same ticker, same session-scoped top-up endpoint. Only the lifetime differs. So the loop lives on
LivePaymentSession, which every path already holds, and each caller contributes only its lifecycle:finally, when the response landsLiveRunnerCallStreamaclose()LiveRunnerSessionstop_payments()/aclose()Nothing in the loop knows which one called it.
Addressing the #31 review
Session liveness. Payments go to the session-scoped endpoint, which 404s once the session is gone, instead of the generic
/paymentthat credits a balance and returns 200 whether or not the session exists. On 404 the loop stops and the owner surfacesreleased.Reserved sessions take the URL from
control_urlin the reservation response, as you described. Single-shot calls can't — a challenge carries onlypayment_params,orchestrator,manifest_id— so they rebuild it from those parts._session_control_urland_payment_endpointkeep the two derivations agreeing by construction rather than by coincidence.Cert verification.
_post_emptyhoisted intohttp.pyaspost_emptyand the payment POST routed through it, so payments disable cert verification like every other helper instead of using a defaultClientSession. As you noted, that's also what gives the loop typed status codes to branch on — without it there is no 404 to detect.stop_payments(). OnLiveRunnerSession, where a session outlives its call and draining is meaningful. Not on the stream: a single-shot stream is its request, so "keep it open but stop paying" isn't a real state —aclose()covers it.Notes for review
auto_payflag. For a plain call an opt-out is unusable — the caller is blocked on the await and cannot fund manually, so it would only mean "die at the preroll." Offchain is the no-payments path, and the loop cannot overspend: a payment covers elapsed time, and a call shorter than one interval never pays twice.byoc.py: a non-retryable4xxstops the loop (404 released, 409 fixed price, 403 mismatch), and 408/429 fall through to a retry. Stopping on an unfamiliar 4xx is deliberate — a silent loop minting tickets nobody will honour is worse than a loud stop.payment_interval_msis deliberately not consumed. It arrives with server: Report payment interval in payment challenges go-livepeer#4001, still open; until then a 3s constant runs against the 5s default tick. There's a TODO onPAYMENT_INTERVAL_Srecording the swap.aclose()self-corrects — the orchestrator releases its session when the request completes, and the next payment 404s — but abandoning an open stream is unbounded. Hence the async context manager._live_runner_session_from_jsonis deleted: no callers, and it built sessions that could never fund themselves.post_emptydiscards the orchestrator'sPaymentResult, which carries refreshed ticket params. Both senders already did that before this PR — the SDK refreshes reactively off the signer's 480. Tracked in Payment responses are discarded instead of refreshing ticket params #55, not changed here.89ac0f7), so it would have started a loop on fixed-price sessions and hammered 409s. This branch gates on_runner_payment_type, so fixed never starts one.Testing
tests/test_live_runner_payments.py— 19 cases against a fake orchestrator and signer, driving the real code paths, one group per mode:runner_idis unknownaclose(), marks itselfreleasedon a 404stop_payments()drains without releasing, a 404 marks it released, and a session whose orchestrator reports no control URL still funds itself at the generic endpoint"", and ids are escaped so they cannot add path segmentsDeliberately framework-free — plain functions and asserts — since the repo has no pytest dependency and no CI. It runs today with
uv run python tests/test_live_runner_payments.py, and pytest discovers it unchanged whenever the framework lands.Still outstanding: an on-chain run on Arbitrum through the remote signer. No bundled example can exercise this —
examples/textruns an offchain orchestrator with noprice_info, andai_http.go:781reserves the session directly without a challenge whenever the price is nil, so validation needs a priced runner on an on-chain node.🤖 Generated with Claude Code