Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ Use the smallest test that exercises the behavior:
- framework example tests when changing sample integrations;
- opt-in e2e tests for real engines and managed environments.

See [docs/TESTING.md](docs/TESTING.md) for the e2e matrix.
See [docs/004-testing.md](docs/004-testing.md) for the e2e matrix.

## Generated code

Expand Down
10 changes: 6 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -90,11 +90,13 @@ Configuration is resolved in this order:
| `RSTREAM_MTLS_CERT_FILE` | Client certificate file for mTLS authentication. |
| `RSTREAM_MTLS_KEY_FILE` | Client private key file for mTLS authentication. |
| `RSTREAM_API_URL` | Control plane API URL for managed project discovery. |
| `RSTREAM_REGION` | Authorized region to select for a managed project. |
| `RSTREAM_CONTROL_PLANE_HEADERS` | Additional Control plane request headers encoded as a JSON object. |

`RSTREAM_ENGINE_ADDRESS` is also accepted for compatibility with older local
SDK workflows. Prefer `RSTREAM_ENGINE` in new code.

See [docs/CONFIGURATION.md](docs/CONFIGURATION.md) for supported YAML fields and
See [docs/001-configuration.md](docs/001-configuration.md) for supported YAML fields and
error behavior.

## FastAPI tunnel
Expand Down Expand Up @@ -224,7 +226,7 @@ async def handle_webhook(request: Request) -> None:
`event.id` is suitable for idempotency. Keep the raw request body unchanged when
verifying the signature.

See [docs/WEBHOOKS.md](docs/WEBHOOKS.md) for the payload shape and headers.
See [docs/003-webhooks.md](docs/003-webhooks.md) for the payload shape and headers.

## Examples

Expand Down Expand Up @@ -259,7 +261,7 @@ Real-engine tests are opt-in:
RSTREAM_PYTHON_E2E=1 pytest tests/e2e
```

See [docs/TESTING.md](docs/TESTING.md) for local-engine and managed-environment
See [docs/004-testing.md](docs/004-testing.md) for local-engine and managed-environment
test commands.

## Repository setup and release
Expand All @@ -269,7 +271,7 @@ secret for normal pull request checks. Release automation uses release-please an
requires the maintainer-managed `RELEASE_PLEASE_TOKEN` secret plus the
`CI_ALLOWED_ACTOR` repository variable.

See [docs/GITHUB_SETUP.md](docs/GITHUB_SETUP.md) before creating or publishing
See [docs/006-github-setup.md](docs/006-github-setup.md) before creating or publishing
the repository.

## License
Expand Down
8 changes: 8 additions & 0 deletions docs/CONFIGURATION.md → docs/001-configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,12 +28,20 @@ Set `RSTREAM_CONFIG` to use another file.
| `RSTREAM_MTLS_CERT_FILE` | mTLS client certificate path. |
| `RSTREAM_MTLS_KEY_FILE` | mTLS client key path. |
| `RSTREAM_API_URL` | Control plane API URL for managed project discovery. |
| `RSTREAM_REGION` | Authorized region to select for a managed project. |
| `RSTREAM_CONTROL_PLANE_HEADERS` | Additional Control plane request headers encoded as a JSON object. |
| `RSTREAM_TUNNEL_TRANSPORT` | `auto`, `tls`, or `quic`. Python maps `auto` to TLS and rejects explicit `quic`. |
| `RSTREAM_QUIC_TRANSPORT` | Legacy selector. Prefer `RSTREAM_TUNNEL_TRANSPORT`. |

The SDK also accepts `RSTREAM_ENGINE_ADDRESS` for compatibility with older
local C++ SDK workflows. Prefer `RSTREAM_ENGINE` in new code.

Region selection requires a managed project endpoint and cannot be combined
with an explicit engine override. Control plane headers may satisfy a separate
deployment access layer. Authentication, forwarding, and hop-by-hop headers are
reserved; malformed values and case-insensitive duplicates are rejected before
network I/O.

## Config file

```yaml
Expand Down
6 changes: 4 additions & 2 deletions docs/TUNNELS.md → docs/002-tunnels.md
Original file line number Diff line number Diff line change
Expand Up @@ -188,8 +188,10 @@ async with rstream.Client.from_env() as client:
hostname = await client.generate_stable_hostname()
async with await client.connect() as control:
tunnel = await control.create_tunnel(
protocol="http", http_version="http/1.1",
publish=True, hostname=hostname,
protocol="http",
http_version="http/1.1",
publish=True,
hostname=hostname,
)
```

Expand Down
File renamed without changes.
File renamed without changes.
File renamed without changes.
2 changes: 1 addition & 1 deletion docs/GITHUB_SETUP.md → docs/006-github-setup.md
Original file line number Diff line number Diff line change
Expand Up @@ -103,6 +103,6 @@ pytest
python -m build
```

Also run at least one real-engine e2e command from [TESTING.md](TESTING.md)
Also run at least one real-engine e2e command from [TESTING.md](004-testing.md)
against a local or managed engine and record the command in the release notes or
PR description.
10 changes: 10 additions & 0 deletions docs/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
# Python SDK documentation

Read these documents in order:

1. [Configuration](001-configuration.md) documents SDK configuration.
2. [Tunnels](002-tunnels.md) documents tunnel APIs and behavior.
3. [Webhooks](003-webhooks.md) documents webhook support.
4. [Testing](004-testing.md) describes local and CI validation.
5. [Test matrix](005-test-matrix.md) defines supported runtime combinations.
6. [GitHub setup](006-github-setup.md) documents repository CI configuration.
4 changes: 3 additions & 1 deletion proto/rstream.proto
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ extend google.protobuf.FieldOptions {
string access = 51234;
}

option (protocol_version) = "1.4.3";
option (protocol_version) = "1.4.4";

package rstream.io_rstrm.protobuf;

Expand Down Expand Up @@ -128,6 +128,7 @@ message TunnelProperties {
google.protobuf.UInt32Value port = 23 [(access) = "read-write"];
google.protobuf.BoolValue upstream_tls = 24 [(access) = "read-write"];
google.protobuf.BoolValue datagram_guaranteed_delivery = 25 [(access) = "read-write"];
google.protobuf.BoolValue allow_cross_region_routing = 26 [(access) = "read-write"];
}

// When a client opens a new control channel to the server
Expand Down Expand Up @@ -210,6 +211,7 @@ message ProxyConnReq {
string stream_id = 2;
google.protobuf.StringValue secret = 3;
IpAddress source_ip = 4;
google.protobuf.StringValue proxy_endpoint = 5;
}

// Client's response to a 'ConnectionInitReq'. The client can refuse the connection by providing an error.
Expand Down
40 changes: 21 additions & 19 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -25,35 +25,37 @@ classifiers = [
"Typing :: Typed",
]
dependencies = [
"protobuf>=6.33.4,<7",
"PyYAML>=6.0.2,<7",
"protobuf>=7.35.1,<8",
"PyYAML>=6.0.3,<7",
]

[project.optional-dependencies]
api = ["httpx>=0.27,<1"]
api = ["httpx>=0.28.1,<1"]
asgi = ["h11>=0.16,<1"]
realtime = ["httpx>=0.27,<1", "websockets>=13,<16"]
realtime = ["httpx>=0.28.1,<1", "websockets>=16.1.1,<17"]
wsgi = ["h11>=0.16,<1"]
examples = [
"aiohttp>=3.10,<4",
"django>=5.1,<6",
"fastapi>=0.110,<1",
"flask>=3.0,<4",
"aiohttp>=3.14.3,<4",
"django>=5.2.15,<6; python_version < '3.12'",
"django>=6.0.7,<7; python_version >= '3.12'",
"fastapi>=0.139.2,<1",
"flask>=3.1.3,<4",
"h11>=0.16,<1",
]
dev = [
"aiohttp>=3.10,<4",
"build>=1.2,<2",
"django>=5.1,<6",
"fastapi>=0.110,<1",
"flask>=3.0,<4",
"httpx>=0.27,<1",
"mypy>=1.13,<2",
"pytest>=8.3,<9",
"pytest-asyncio>=0.24,<1",
"ruff>=0.8,<1",
"aiohttp>=3.14.3,<4",
"build>=1.5,<2",
"django>=5.2.15,<6; python_version < '3.12'",
"django>=6.0.7,<7; python_version >= '3.12'",
"fastapi>=0.139.2,<1",
"flask>=3.1.3,<4",
"httpx>=0.28.1,<1",
"mypy>=2.3,<3",
"pytest>=9.1.1,<10",
"pytest-asyncio>=1.4,<2",
"ruff>=0.16,<1",
"trustme>=1.2,<2",
"websockets>=13,<16",
"websockets>=16.1.1,<17",
]

[project.urls]
Expand Down
92 changes: 47 additions & 45 deletions src/rstream/_proto/rstream_pb2.py

Large diffs are not rendered by default.

12 changes: 8 additions & 4 deletions src/rstream/_proto/rstream_pb2.pyi
Original file line number Diff line number Diff line change
Expand Up @@ -96,7 +96,7 @@ class ServerDetails(_message.Message):
def __init__(self, agent: _Optional[_Union[_wrappers_pb2.StringValue, _Mapping]] = ..., channel: _Optional[_Union[_wrappers_pb2.StringValue, _Mapping]] = ..., version: _Optional[_Union[_wrappers_pb2.StringValue, _Mapping]] = ..., plan: _Optional[_Union[_wrappers_pb2.StringValue, _Mapping]] = ..., provider: _Optional[_Union[_wrappers_pb2.StringValue, _Mapping]] = ..., region: _Optional[_Union[_wrappers_pb2.StringValue, _Mapping]] = ..., update: _Optional[_Union[_wrappers_pb2.StringValue, _Mapping]] = ...) -> None: ...

class TunnelProperties(_message.Message):
__slots__ = ("id", "creation_date", "name", "type", "publish", "protocol", "labels", "geoip", "trusted_ips", "host", "tls_mode", "tls_alpns", "tls_min_version", "tls_ciphers", "mtls_auth", "mtls_cacert_pem", "http_version", "http_use_tls", "token_auth", "rstream_auth", "challenge_mode", "hostname", "port", "upstream_tls", "datagram_guaranteed_delivery")
__slots__ = ("id", "creation_date", "name", "type", "publish", "protocol", "labels", "geoip", "trusted_ips", "host", "tls_mode", "tls_alpns", "tls_min_version", "tls_ciphers", "mtls_auth", "mtls_cacert_pem", "http_version", "http_use_tls", "token_auth", "rstream_auth", "challenge_mode", "hostname", "port", "upstream_tls", "datagram_guaranteed_delivery", "allow_cross_region_routing")
class LabelsEntry(_message.Message):
__slots__ = ("key", "value")
KEY_FIELD_NUMBER: _ClassVar[int]
Expand Down Expand Up @@ -129,6 +129,7 @@ class TunnelProperties(_message.Message):
PORT_FIELD_NUMBER: _ClassVar[int]
UPSTREAM_TLS_FIELD_NUMBER: _ClassVar[int]
DATAGRAM_GUARANTEED_DELIVERY_FIELD_NUMBER: _ClassVar[int]
ALLOW_CROSS_REGION_ROUTING_FIELD_NUMBER: _ClassVar[int]
id: _wrappers_pb2.StringValue
creation_date: _timestamp_pb2.Timestamp
name: _wrappers_pb2.StringValue
Expand All @@ -154,7 +155,8 @@ class TunnelProperties(_message.Message):
port: _wrappers_pb2.UInt32Value
upstream_tls: _wrappers_pb2.BoolValue
datagram_guaranteed_delivery: _wrappers_pb2.BoolValue
def __init__(self, id: _Optional[_Union[_wrappers_pb2.StringValue, _Mapping]] = ..., creation_date: _Optional[_Union[datetime.datetime, _timestamp_pb2.Timestamp, _Mapping]] = ..., name: _Optional[_Union[_wrappers_pb2.StringValue, _Mapping]] = ..., type: _Optional[_Union[_wrappers_pb2.StringValue, _Mapping]] = ..., publish: _Optional[_Union[_wrappers_pb2.BoolValue, _Mapping]] = ..., protocol: _Optional[_Union[_wrappers_pb2.StringValue, _Mapping]] = ..., labels: _Optional[_Mapping[str, str]] = ..., geoip: _Optional[_Iterable[str]] = ..., trusted_ips: _Optional[_Iterable[str]] = ..., host: _Optional[_Union[_wrappers_pb2.StringValue, _Mapping]] = ..., tls_mode: _Optional[_Union[_wrappers_pb2.StringValue, _Mapping]] = ..., tls_alpns: _Optional[_Iterable[str]] = ..., tls_min_version: _Optional[_Union[_wrappers_pb2.StringValue, _Mapping]] = ..., tls_ciphers: _Optional[_Iterable[str]] = ..., mtls_auth: _Optional[_Union[_wrappers_pb2.BoolValue, _Mapping]] = ..., mtls_cacert_pem: _Optional[_Union[_wrappers_pb2.StringValue, _Mapping]] = ..., http_version: _Optional[_Union[_wrappers_pb2.StringValue, _Mapping]] = ..., http_use_tls: _Optional[_Union[_wrappers_pb2.BoolValue, _Mapping]] = ..., token_auth: _Optional[_Union[_wrappers_pb2.BoolValue, _Mapping]] = ..., rstream_auth: _Optional[_Union[_wrappers_pb2.BoolValue, _Mapping]] = ..., challenge_mode: _Optional[_Union[_wrappers_pb2.BoolValue, _Mapping]] = ..., hostname: _Optional[_Union[_wrappers_pb2.StringValue, _Mapping]] = ..., port: _Optional[_Union[_wrappers_pb2.UInt32Value, _Mapping]] = ..., upstream_tls: _Optional[_Union[_wrappers_pb2.BoolValue, _Mapping]] = ..., datagram_guaranteed_delivery: _Optional[_Union[_wrappers_pb2.BoolValue, _Mapping]] = ...) -> None: ...
allow_cross_region_routing: _wrappers_pb2.BoolValue
def __init__(self, id: _Optional[_Union[_wrappers_pb2.StringValue, _Mapping]] = ..., creation_date: _Optional[_Union[datetime.datetime, _timestamp_pb2.Timestamp, _Mapping]] = ..., name: _Optional[_Union[_wrappers_pb2.StringValue, _Mapping]] = ..., type: _Optional[_Union[_wrappers_pb2.StringValue, _Mapping]] = ..., publish: _Optional[_Union[_wrappers_pb2.BoolValue, _Mapping]] = ..., protocol: _Optional[_Union[_wrappers_pb2.StringValue, _Mapping]] = ..., labels: _Optional[_Mapping[str, str]] = ..., geoip: _Optional[_Iterable[str]] = ..., trusted_ips: _Optional[_Iterable[str]] = ..., host: _Optional[_Union[_wrappers_pb2.StringValue, _Mapping]] = ..., tls_mode: _Optional[_Union[_wrappers_pb2.StringValue, _Mapping]] = ..., tls_alpns: _Optional[_Iterable[str]] = ..., tls_min_version: _Optional[_Union[_wrappers_pb2.StringValue, _Mapping]] = ..., tls_ciphers: _Optional[_Iterable[str]] = ..., mtls_auth: _Optional[_Union[_wrappers_pb2.BoolValue, _Mapping]] = ..., mtls_cacert_pem: _Optional[_Union[_wrappers_pb2.StringValue, _Mapping]] = ..., http_version: _Optional[_Union[_wrappers_pb2.StringValue, _Mapping]] = ..., http_use_tls: _Optional[_Union[_wrappers_pb2.BoolValue, _Mapping]] = ..., token_auth: _Optional[_Union[_wrappers_pb2.BoolValue, _Mapping]] = ..., rstream_auth: _Optional[_Union[_wrappers_pb2.BoolValue, _Mapping]] = ..., challenge_mode: _Optional[_Union[_wrappers_pb2.BoolValue, _Mapping]] = ..., hostname: _Optional[_Union[_wrappers_pb2.StringValue, _Mapping]] = ..., port: _Optional[_Union[_wrappers_pb2.UInt32Value, _Mapping]] = ..., upstream_tls: _Optional[_Union[_wrappers_pb2.BoolValue, _Mapping]] = ..., datagram_guaranteed_delivery: _Optional[_Union[_wrappers_pb2.BoolValue, _Mapping]] = ..., allow_cross_region_routing: _Optional[_Union[_wrappers_pb2.BoolValue, _Mapping]] = ...) -> None: ...

class OpenControlChannelReq(_message.Message):
__slots__ = ("client_details",)
Expand Down Expand Up @@ -216,16 +218,18 @@ class CloseTunnelRsp(_message.Message):
def __init__(self, tunnel_id: _Optional[str] = ...) -> None: ...

class ProxyConnReq(_message.Message):
__slots__ = ("tunnel_id", "stream_id", "secret", "source_ip")
__slots__ = ("tunnel_id", "stream_id", "secret", "source_ip", "proxy_endpoint")
TUNNEL_ID_FIELD_NUMBER: _ClassVar[int]
STREAM_ID_FIELD_NUMBER: _ClassVar[int]
SECRET_FIELD_NUMBER: _ClassVar[int]
SOURCE_IP_FIELD_NUMBER: _ClassVar[int]
PROXY_ENDPOINT_FIELD_NUMBER: _ClassVar[int]
tunnel_id: str
stream_id: str
secret: _wrappers_pb2.StringValue
source_ip: IpAddress
def __init__(self, tunnel_id: _Optional[str] = ..., stream_id: _Optional[str] = ..., secret: _Optional[_Union[_wrappers_pb2.StringValue, _Mapping]] = ..., source_ip: _Optional[_Union[IpAddress, _Mapping]] = ...) -> None: ...
proxy_endpoint: _wrappers_pb2.StringValue
def __init__(self, tunnel_id: _Optional[str] = ..., stream_id: _Optional[str] = ..., secret: _Optional[_Union[_wrappers_pb2.StringValue, _Mapping]] = ..., source_ip: _Optional[_Union[IpAddress, _Mapping]] = ..., proxy_endpoint: _Optional[_Union[_wrappers_pb2.StringValue, _Mapping]] = ...) -> None: ...

class ProxyConnRsp(_message.Message):
__slots__ = ("stream_id", "error")
Expand Down
92 changes: 89 additions & 3 deletions src/rstream/api.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,12 @@
from dataclasses import dataclass
from urllib.parse import quote, urljoin

from rstream.config import DEFAULT_API_URL, normalize_engine_address
from rstream.config import (
DEFAULT_API_URL,
_normalize_control_plane_headers,
_normalize_region,
normalize_engine_address,
)
from rstream.errors import ConfigurationError, RuntimeError


Expand All @@ -17,6 +22,16 @@ class TokenCredentials:
token: str


@dataclass(frozen=True)
class TunnelsProjectRegionalEndpoint:
"""Regional engine endpoint authorized for a managed project."""

provider: str
region: str
domain: str
engine_port: int


@dataclass(frozen=True)
class TunnelsProject:
"""Managed tunnels project metadata required for engine resolution."""
Expand All @@ -26,6 +41,8 @@ class TunnelsProject:
url: str | None
domain: str
engine_port: int
routing: str
regional_endpoints: tuple[TunnelsProjectRegionalEndpoint, ...] = ()


class RstreamAPIClient:
Expand All @@ -35,9 +52,13 @@ def __init__(
self,
*,
api_url: str = DEFAULT_API_URL,
control_plane_headers: Mapping[str, str] | None = None,
credentials: TokenCredentials | None = None,
) -> None:
self.api_url = api_url.rstrip("/")
self.control_plane_headers = dict(
_normalize_control_plane_headers(control_plane_headers)
)
self.credentials = credentials

async def resolve_tunnels_project(self, endpoint: str) -> TunnelsProject:
Expand Down Expand Up @@ -66,7 +87,7 @@ async def request_json(self, path: str) -> Mapping[str, object]:
code="ERR_RSTREAM_INVALID_API_PATH",
)
url = urljoin(f"{self.api_url}/", path.lstrip("/"))
headers: dict[str, str] = {}
headers = dict(self.control_plane_headers)
if self.credentials is not None:
headers["Authorization"] = f"Bearer {self.credentials.token}"
async with httpx.AsyncClient(follow_redirects=False, timeout=15) as client:
Expand All @@ -85,7 +106,41 @@ async def request_json(self, path: str) -> Mapping[str, object]:
return {str(key): item for key, item in value.items()}


def engine_from_project(project: TunnelsProject) -> str:
def engine_from_project(project: TunnelsProject, region: str | None = None) -> str:
requested = _normalize_region(region)
if requested is not None:
matches = tuple(
endpoint
for endpoint in project.regional_endpoints
if endpoint.region.strip().lower() == requested
)
if not matches:
available = sorted(
{
endpoint.region.strip().lower()
for endpoint in project.regional_endpoints
if endpoint.region.strip()
}
)
suffix = f" Available regions: {', '.join(available)}." if available else ""
raise ConfigurationError(
f"Region '{requested}' is not available for this project.{suffix}",
code="ERR_RSTREAM_REGION_UNAVAILABLE",
)
if len(matches) > 1:
raise ConfigurationError(
f"Region '{requested}' is ambiguous for this project.",
code="ERR_RSTREAM_REGION_AMBIGUOUS",
)
selected = matches[0]
engine = f"{project.endpoint}.{selected.domain}:{selected.engine_port}"
normalized = normalize_engine_address(engine)
if normalized is None:
raise RuntimeError(
"Failed to normalize managed project regional engine address.",
code="ERR_RSTREAM_ENGINE_RESOLUTION",
)
return normalized
if project.endpoint and project.domain:
engine = f"{project.endpoint}.{project.domain}:{project.engine_port or 443}"
normalized = normalize_engine_address(engine)
Expand Down Expand Up @@ -117,9 +172,40 @@ def _project_from_json(data: Mapping[str, object]) -> TunnelsProject:
url=url if isinstance(url, str) else None,
domain=domain,
engine_port=engine_port,
routing=_string_required(data, "routing"),
regional_endpoints=_regional_endpoints_from_json(data.get("regionalEndpoints")),
)


def _regional_endpoints_from_json(
value: object,
) -> tuple[TunnelsProjectRegionalEndpoint, ...]:
if value is None:
return ()
if not isinstance(value, list):
raise RuntimeError(
"Control plane response has invalid 'regionalEndpoints'.",
code="ERR_RSTREAM_API_INVALID_RESPONSE",
)
endpoints: list[TunnelsProjectRegionalEndpoint] = []
for item in value:
if not isinstance(item, Mapping):
raise RuntimeError(
"Control plane response has invalid 'regionalEndpoints'.",
code="ERR_RSTREAM_API_INVALID_RESPONSE",
)
normalized = {str(key): entry for key, entry in item.items()}
endpoints.append(
TunnelsProjectRegionalEndpoint(
provider=_string_required(normalized, "provider"),
region=_string_required(normalized, "region"),
domain=_string_required(normalized, "domain"),
engine_port=_int_required(normalized, "enginePort"),
)
)
return tuple(endpoints)


def _string_required(data: Mapping[str, object], key: str) -> str:
value = data.get(key)
if isinstance(value, str) and value.strip():
Expand Down
Loading