Conversation
session-deps is a new repo that consolidates how we handle loading and doing static bundle builds of various common external dependencies across Session projects.
Carrying a libsodium fork is too much of a nuissance as updating it to the latest version is non-trivial. This drops the libsodium-internal fork in favour of using tweenacl's implementation *just* for the X25519 -> Ed25519 pubkey conversion, and using a stock libsodium for everything else. This also bumps the libsodium requirement up to 1.0.21: that version will be required for SHAKE support in future commits on this branch.
- Blake2b hashing:
- Add nicer hash::blake2b functions for simpler has computations where
you can just pass a bunch of spannables and get the hash over them,
rather than needing to do a bunch of manual C API update calls.
- Add a ""_b2b_pers for a compile-time validated blake2b
personalisation string
- Drop make_blake2b32_hasher().
- TODO: convert these to make full use of hash::blake2b(...), as
above.
- Change cleared_array to take a Char type instead of forcing unsigned
char.
- Add cleared_uchars (and cleared_bytes) for the old force-unsigned char
typedef.
- Make `to_span` work for any input convertible to a span
- Tighten up various functions taking fixed length values (like session
ids, pubkeys) to take compile-time-fixed spans instead of dynamic
extent spans. This allows these generic functions to not have to
worry about length checking.
- Add a generic random::fill(s) function that fills some spannable type
s with random bytes.
This switches to the PR branches for session-router and libquic to use session-deps. (This was required under ninja builds, in particular, to get the deduplication handling for gmp and nettle via new session-deps code to deal with that).
- Refactor manual libsodium blake2b hash calls to use simpler hash::blake2b functions instead. - Unify usage of array_uc32/33/64 and uc32/33/64: now we have just uc32/33/64 and cleared_uc32/33/64 (i.e. the "array_" prefix is gone). - Add a unsigned char array literal, which is quite useful for static hash keys.
- Including raw integer bytes would break the hashes on a non-little endian arches; this adds a helper than ensures we are always hashing the little-endian value. - Make the remaining manual hash use blake2b_pers. This didn't get autoconverted before because the `if` around part of the hash, but that if is actually completely unnecessary: if the value is empty, a blake2b hash update does nothing, and so it can just be always included (and when empty, it is still the right thing).
oxenc has buggy "constexpr" overloads that just break if invoked, and they get invoked here with a uint8_t value + unsigned char array. The issue is fixed in oxenc dev, but switching to a byte here works around it for current and older oxenc versions.
The recent commit to unify the types wasn't applied to the test suite.
hash::blake2b (and related) now take integers directly, writing the integer bytes (with byte swapping applied, if necessary), which further simplies the hashing API.
Make way for account keys to be in here as well.
If two devices rotate account keys at approximately the same time, both might end up with inconsistent "active" keys. This commit adds deterministic tie breaking (always prefering the later, with fallback to seed ordering).
Adds a "needs push" concept, along with more tracking fields to let us distinguish between various possible states.
`session::AdjustedClock` now carries an adjustable static offset and when `now()` is called it returns standard system clock timepoints with the adjustment applied. The networking code had a similar adjustment already, although it was per-network-object: this change replaces that, and makes it now global across all uses of the clock anywhere, instead of per-Network instance.
Response::status was a uint32_t whose domain changed by endpoint (0/1/2 for
most, plus 100/101 for add-payment) -- and for every response except add-payment
it just duplicated !errors.empty(). Remove it: Response is now { errors } with a
success() helper (C: response_header drops status; success == errors_count == 0),
and add-payment collapses to the same -- the backend already sends the human
reason (including already-redeemed / unknown-payment) in errors, so display is
unaffected. Deletes the AddProPaymentResponseStatus C++ enum, the C
SESSION_PRO_BACKEND_ADD_PRO_PAYMENT_RESPONSE_STATUS enum, and its magic-int
values. Parsers still read the wire "status" int internally to decide success;
they just no longer surface it. (A future machine-readable discriminator, if
needed, should be an opaque string slug on the wire like the Q13 statuses.)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…r (Delta #12) Reworks the response envelope to spec §5 / Delta #12. The wire `status` becomes a string ok/fail/error parsed into a CLOSED ResponseStatus enum -- an unrecognized value is a protocol error, never passed through (backend confirmed status is exhaustive, Q14). The `errors` array becomes a single `error` string (diagnostic/fallback, first-reason-only). A new open/additive `error_code` slug (opaque; unknown values pass through) carries the machine-readable outcome. Response is renamed ResponseBase (it's only ever a base) and is now { ResponseStatus status; optional<string> error_code; optional<string> error; } with an explicit operator bool(). The C response_header mirrors it (status enum + error_code/error string8, {NULL,0} == none). get-details account status key renamed status -> user_status (Q15). Deletes the obsolete integer status enum. A libsession-side parse failure yields a protocol error (error_code "invalid_response"). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Implements pro-wire-protocol.md Delta #13 / §1.1: all five Ed25519 signatures (the offline proof §2 + the four signed requests §3.1–3.4) now sign the message bytes directly instead of a BLAKE2b-256 digest. - Add session::pro::signed_message() (src/pro_message.hpp): builds the signed message per §1.1 — 16-byte domain prefix, fixed-width raw keys/tags (self-delimiting), integers as canonical decimal ASCII (std::to_chars), strings verbatim, a single NUL between adjacent variable-length fields. - Replace ProProof::hash() with ProProof::signed_message() and delete the C session_protocol_pro_proof_hash() (the 32-byte digest no longer exists in the protocol). - Drop the now-dead hash.hpp includes; BLAKE2b is gone from the Pro signing path. - Retype/rename the 16-byte domain-prefix constants (session_protocol.hpp) from _b2b_pers spans to std::string_view, *_PERS -> *_DOMAIN; drop the C session_protocol_pro_proof_hash declaration. - Update tests to sign via signed_message() and to compare the proof's signature (not a hash) across the encode/decode round-trip. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Client-facing enumeration of the user-visible purchasable payment platforms (provider-code slugs: google_play, app_store) so clients build their "purchase via …" store list at runtime instead of hardcoding two stores. Hidden mechanisms (rangeproof) are handled but never listed — they surface only for users who already hold them. C++ `visible_platforms()` returns std::span<const std::string_view>; C `session_pro_backend_visible_platforms(size_t*)` returns a static pointer-array derived from it. Both built from the existing SESSION_PRO_BACKEND_PAYMENT_PROVIDER_CODE_* constants (single source). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Parse the closed `plan` grammar (pro-wire-protocol.md §1 / Delta #14) in libsession rather than handing clients the raw wire code to reinterpret. get-details items now carry a parsed ProPlanPeriod{count, unit} (unit ∈ second/day/week/month/year/lifetime) instead of a raw string. - Add parse_plan_period() + ProPlanPeriod/ProPlanUnit (C++), mirrored by SESSION_PRO_BACKEND_PLAN_UNIT + plan_count/plan_unit on the C item. - The unit is preserved, never canonicalized ("12m" != "1y" as values). "lifetime" -> {0, lifetime}; invariant count == 0 iff unit == lifetime. - Closed grammar (a backend<->libsession contract, like the §5 status enum): an unrecognized code is a protocol error, not passed through raw. - Reserves a testing-only `s` (seconds) unit; single-unit only ("1y6m" invalid). Response-parse only -- no wire or signing change. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Use the idiomatic `typedef struct X { ... } X;` instead of the split
`typedef struct X X; struct X { ... };`. None of these structs are self-
or mutually-referential, so the forward typedef bought nothing but extra
lines.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The provider-code slugs were defined in the C header (static const char[]) and referenced from C++ — backwards for a library whose C++ side is the primary implementation. Move them to C++ constexpr std::string_view (session::pro_backend::PAYMENT_PROVIDER_*, the single source) and have the C `SESSION_PRO_BACKEND_PAYMENT_PROVIDER_CODE_*` symbols reference them as `extern const char* const`, matching how URL / PUBKEY / the endpoints already work. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Rework the C get-details/proof/revocations/refund response structs so each one owns the parsed C++ response object (via an opaque `void* internal_`) and its fields point into it, instead of deep-copying into fixed buffers backed by a hand-rolled bump arena. This mirrors how the request side already owns its C++ ProRequest. - Response string fields (`error_code`/`error`, item `status`/ `payment_provider`/`payment_id`, details `status`) become `const char*` views (NUL-terminated; NULL when absent); the per-item revocation `tag` becomes a `const unsigned char*` view. All the `char[N]`+`_count` pairs are gone -- including the 128-byte `payment_id` buffer that silently truncated long Google composites. - `header.internal_arena_buf_` -> `void* internal_`; each `*_response_free` is a one-liner over a `c_free_response<Owned>` template constrained to `std::derived_from<ResponseBase>` (the two item-bearing holders derive from their C++ response type). File-local `to_c()` converters (taking a non-const ref, so a temporary can't bind) replace the manual per-field fills. - Delete `arena_t` / `arena_alloc` / `arena_alloc_to_string8` (types.h, types.cpp) -- now used nowhere. - Reorder pro_backend.h: request struct + `request_free` above the response group (no longer wedged between the header and the responses); each `*_free` decl sits immediately after its struct. C-API-only change (response side); the C++ API is unchanged. iOS/nodejs glue must read the new `const char*`/pointer fields instead of sized buffers. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Cleans up C-isms the original Pro code sprayed through the header/impl:
- Collapse the split `typedef struct X X; struct X {...};` into the
idiomatic `typedef struct X {...} X;`.
- Move each `*_free` declaration to sit immediately after the struct it
frees (it's effectively that struct's destructor).
- Move the protocol limit/padding constants to C++ as the source of truth
(`session::` inline constexpr) with the C `SESSION_PROTOCOL_*` symbols
referencing them as `extern const int`; drop the misleading `PRO_` from
the two non-Pro ones (STANDARD_CHARACTER_LIMIT,
STANDARD_PINNED_CONVERSATION_LIMIT).
- Replace the reinvented `array_uc32_from_ptr_result` with
`std::optional` (maybe_uc32_from_ptr).
- Replace the `sizeof((T*)0)->member` null-deref idiom with
`sizeof(T::member)`.
- Rename the meaningless `utf`/`utf_size` params to `text`/`text_size`.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…der constants) - Rename ProviderUrls -> ProviderURLs (initialism casing). - All five URL fields are now std::optional<std::string_view>: a provider may populate none of them (e.g. a one-shot crypto provider), so clients test the optional instead of an empty-string sentinel. The C struct's matching fields may now individually be NULL. - Replace the per-call constructed value + std::optional<ProviderURLs> return with fixed per-provider `constexpr ProviderURLs` constants and a `const ProviderURLs*` return (nullptr = no URLs at all). URLs are one-line `""sv` literals (guaranteed null-terminated) so clang-format won't reflow them mid-string. - Use designated initialisers for the provider constants. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Now that every ProviderURLs field is optional, the by-value C return could no longer distinguish an unknown provider code from a recognised provider that simply has no URLs -- both produced an all-NULL struct. Add a `bool found` member (true iff the C++ provider_urls() returned non-null) so C clients get the same distinction the C++ `const ProviderURLs*` gives, without a returned pointer they'd have to reason about freeing. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
`string8` (`{char* data; size_t size}`) was a structural duplicate of the
existing `span_u8` (`{unsigned char* data; size_t size}`) -- same ptr+len byte
span, right down to a duplicate pair of alloc/copy_or_throw helpers. Once its
uses are mapped, none actually need a string-with-length:
- SESSION_PROTOCOL_STRINGS (24 fields) and session_protocol_pro_features_for_msg
`error` are static, null-terminated strings -> `const char*`. The `error`
string_view only ever views static literals ("..." + simdutf::error_to_string,
which returns string_view over literals), so there is nothing to free and no
dangling. SESSION_PROTOCOL_STRINGS drops the `string8_literal` macro for plain
string literals.
- session_pro_backend_request `data` is an opaque (possibly binary) payload that
genuinely needs ptr+len -> `span_u8`, the type already used for sig/msg.
Then delete the dead string8_alloc_or_throw/string8_copy_or_throw helpers (no
callers; duplicates of span_u8_*), and the `string8` struct + `string8_literal`
macro. Test helper string8_equals -> span_u8_equals, now a std::string_view
compare instead of a hand-rolled memcmp.
C-API change (clients): session_protocol_strings fields and
session_protocol_pro_features_for_msg.error are now `const char*`;
session_pro_backend_request.data is now `span_u8`.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…details (Delta #15) The old combined get-details endpoint conflated a cheap entitlement check with a heavy payment ledger; no client used the ledger beyond the latest item. Split it into two endpoints (both a signed-request flip, new domains): - get_pro_status (domain ProGetProStatus_; signed `master_pkey ‖ ts`) -- the hot "am I Pro?" path. Response: user_status, auto_renewing, expiry_ts, refund_requested_ts, grace_period_duration, error_report, and a single `latest_payment` (or none). No list, no pagination. C++ ProStatusResponse / parse_pro_status; C session_pro_backend_get_pro_status_response with a bool has_latest_payment + embedded latest_payment. - get_payment_details (domain ProGetPayDetails; signed `master_pkey ‖ ts ‖ \0 ‖ limit ‖ \0 ‖ before`) -- paginated history. Request {limit, before}; response {payments_total, items, next_cursor}, no user_status. Keyset pagination: `before`/`next_cursor` is an opaque cursor (std::optional / const char* NULL at end-of-data), echoed verbatim, never parsed. Payment-item parsing is factored into a shared parse_payment_item helper used by both `latest_payment` and `items`. No "pro details" terminology survives (identifiers, endpoints, domains, the error-report enum -> GET_PRO_STATUS_*). C-API change (clients): get_pro_details_{request_build,response,response_parse, response_free} and SESSION_PRO_BACKEND_GET_PRO_DETAILS_ENDPOINT are replaced by get_pro_status_* and get_payment_details_* equivalents. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The doc preamble linked a pinned old commit of server.py on the Doy-lee session-pro-backend fork; the backend has since merged upstream, making the fork+commit reference doubly stale. Cite the authoritative pro-wire-protocol.md spec instead, matching the stable branch's wording. Comment-only. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The backend removed the "Delta N" change-history annotations from pro-wire-protocol.md (the spec now reflects only the current state), so those citations no longer resolve. Drop the "Delta #14" references from the `plan` billing-period doc comments, keeping the still-valid §1 spec reference (the plan grammar lives in §1). Comment-only; all other §-references verified against the current spec. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Stand up an ephemeral session-pro-backend (throwaway postgres + flask with provider egress stubbed) and drive all Pro endpoints through libsession's C++ API over real HTTP (direct + zero-hop onion v4), asserting every request round-trips, the backend accepts each signed digest, responses parse, and issued proofs verify against the backend's live signing key. This catches wire-contract / lockstep drift the self-consistent unit tests structurally cannot. - tests/pro_backend/run-dev-backend.sh: hermetic launcher (initdb/pg_ctl + ephemeral signing key + flask, polls /status, runs testAll, tears down). - tests/pro_backend/seed_payment.py: injects witnessed-unredeemed payments and revocations straight into the DB via the backend's own helpers. - test_pro_backend.cpp: replace the legacy C-API [dev_server] block with a transport-abstracted [pro_live] suite: /status round-trip, full lifecycle flow (both providers, both transports), get_pro_revocations. The lifecycle flow covers the Delta #15 endpoint split — get_pro_status + paginated get_payment_details, including an opaque next_cursor round-trip. - .drone.jsonnet: "Debian sid (Pro backend live)" pipeline running [pro_live]. Tracks Delta #12 envelope, #13 sign-the-message-directly, #14 parsed plan, #15 get_pro_status/get_payment_details split, and the generations-table revocation reshape. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The live [pro_live] suite catches libsession<->backend signing drift, but not a fault where BOTH sides share the same wrong construction -- they would still agree. Add frozen, spec-anchored known-answer vectors that pin libsession's signed-message construction (wire spec 1.1 / 2 / 3, Delta #13) with no running backend, and that freeze the wire format against silent future drift. - tests/pro_backend/gen_kat.py: generates the vectors from the backend's independent implementation for fixed deterministic inputs. - test_pro_backend.cpp [pro_kat]: for each signed request, asserts the request's Ed25519 signature covers exactly the expected (hand-verified) message bytes; for the proof, asserts ProProof::signed_message() equals the expected bytes and that a frozen backend signature verifies (and a tampered key does not). Covers the Delta #15 split endpoints: get_pro_status, and get_payment_details with both an empty `before` (newest page, trailing NUL) and a non-empty opaque cursor. Ed25519 determinism makes each signature its own known answer. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two housekeeping changes now that the backend merged to session-foundation dev: - The wire spec dropped its "Delta N" change-history annotations, so those citations in the harness comments/vectors are dangling. Reworded to cite the current spec sections instead (plan -> §1, signed-message construction -> §1.1/§2/§3, the read-request split -> §3.4) or dropped the parenthetical. Verified remaining §-references resolve against the current section map. - .drone.jsonnet: point the CI backend clone at session-foundation dev (was the now-merged jagerman phase2-foundation). An integration/drift harness should track the counterpart's mainline; a pinned ref is what went stale before. No behavior change to the tests themselves. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The backend's main.py imports psycopg_pool (psycopg 3's connection pool, requirements.txt: psycopg-pool~=3.2) — a separate Debian package from python3-psycopg. Without it flask fails to import `main` and the ephemeral backend never starts, so the [pro_live] step's launcher reports "flask exited during startup: ModuleNotFoundError: No module named 'psycopg_pool'". Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The Pro-backend-live pipeline runs in a minimal Debian container whose default locale is C/POSIX -> ASCII. The backend's migration SQL carries non-ASCII bytes (an em-dash in a comment); psycopg encodes each query with the connection's client encoding, which follows the DB/locale, so bootstrap_db died with `UnicodeEncodeError: 'ascii' codec can't encode character '—'` and flask never started. run-dev-backend.sh now exports LC_ALL/LANG=C.UTF-8 (always present in glibc) and initdb's the cluster with --encoding=UTF8, so the cluster and every client (flask, seed helper, psql) speak UTF-8 as in production. Reproduced locally under LC_ALL=C and confirmed fixed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
In some cases this causes fmt-9 compilation problems because the to_string_view template matches too broadly. This confines it down to containers that will actually work to avoid the problem. Also drops the OutChar template, because it is useless: it can never be anything other than char because of the fixed string_view return type (I think this is a holdover from when we supposed ustring_view/bstring_view). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
onion_request_router.cpp uses std::views::values but relied on a transitive include for <ranges>; include it explicitly. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ad URLs
Some clients (e.g. older Android versions) generate community file
download urls without the room segment (`{base_url}/file/{file_id}`).
Accept these as a fallback so their attachments remain downloadable,
leaving the room empty for the caller to populate from context.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Makes sure both have a reasonably sane value; mainly defensive. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The backend dropped the `{google,apple}_obfuscated_account_id_from_master_pkey`
helpers, so seed_payment.py's [pro_live] seeding died with
`AttributeError: module 'backend' has no attribute
'apple_obfuscated_account_id_from_master_pkey'`.
The account tag each provider attests changed, and the seeded witnessed-payment
row must equal what the backend now recomputes from the request-signed master key
at redeem time (backend.py add_pro_payment):
- Google: the client sets setObfuscatedAccountId to the master pubkey verbatim
(no more sha256), so the id is the raw 32-byte master pubkey -> `bytes(vk)`.
- Apple: appAccountToken = UUID(first 16 bytes of the master pubkey) via
providers.app_store.uuid_from_master_pk, stored lowercased.
Import the Apple provider lazily (its module pulls the Apple SDK) so seeding a
non-Apple payment doesn't require it.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This code was using std::any, which has warts when used for fixed types: it ended up needing some gross macro uses rather than cleaner templating, and could not catch invalid types at compile time. Additionally it broke under older distros for some reason (perhaps because of a missing <any> header, but no matter). This replaces it with a simpler `network::opt::any` instead of `std::any` which is actually a variant, and results in cleaner code: both getting rid of the macro, allowing compile time checks, and removing the need for the `base` class layer under every option (is-a-base is now replaced with is_option<T>). Also changes the std::optional<OPTION> handling to avoid an extra value copy/move. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
build_file_server_config copied the custom scheme/host/port/max_file_size from the Config but never applied custom_file_server_pubkey_hex, so a caller-supplied file server pubkey was silently ignored. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
jagerman
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This branch (
pfs) is now actively maintained alongsidedevand will eventually replace thedevbranch.This adds perfect forward secrecy support with post-quantum encryption; a "Core" component with a database layer to track state; direct-QUIC file server support; and a host of code cleanups.
For extensive details, see the description of PR #90.