Skip to content

feat: route collection-scoped activity queries to single bucket (IN-1231) - #4483

Merged
gaspergrom merged 7 commits into
mainfrom
feat/IN-1231-collection-bucket-routing
Aug 18, 2026
Merged

feat: route collection-scoped activity queries to single bucket (IN-1231)#4483
gaspergrom merged 7 commits into
mainfrom
feat/IN-1231-collection-bucket-routing

Conversation

@gaspergrom

Copy link
Copy Markdown
Contributor

Summary

  • Adds activityRelations_collection_bucket_routing.pipe, a single-bucket routing pipe that mirrors the existing 10-way-union pattern but for collection-scoped queries. Routes to the correct pre-resolved bucket (0-9) based on the optional bucketId parameter.
  • Updates activities_filtered.pipe to accept optional bucketId alongside collectionSlug. When both are provided, routes directly to the single-bucket pipe instead of the union, improving query performance for pre-resolved collections. Fully backward compatible — callers passing only collectionSlug continue to use the existing union pipe.
  • This PR is inert without the companion insights PR (feat/IN-1231) which wires up the caller-side logic to pass bucketId. Can merge independently or in any order relative to the insights PR.

Pipes already validated and deployed to Tinybird production by crowd-tinybird-manager:

  • Bucket-0 routing tested against real collection cloud-management-platforms (34.1M rows); correct row counts confirmed
  • All 20+ downstream consumer pipes (health metrics, leaderboards, collections API) verified intact
  • Backward compatibility confirmed for existing collectionSlug-only callers

JIRA

IN-1231 — Route collection-scoped activity queries to single bucket (insights-driven)

…231)

Signed-off-by: Gašper Grom <gasper.grom@gmail.com>
Signed-off-by: Gašper Grom <gasper.grom@gmail.com>
Copilot AI balanced review requested due to automatic review settings August 17, 2026 08:29
@cursor

cursor Bot commented Aug 17, 2026

Copy link
Copy Markdown

PR Summary

Medium Risk
Changes core activities_filtered collection behavior: missing bucketId yields empty results until callers are updated, affecting many downstream analytics pipes.

Overview
Adds activityRelations_collection_bucket_routing.pipe, which selects one of the ten activityRelations_collection_deduplicated_cleaned_bucket_*_ds tables from a pre-resolved bucketId (same Jinja pattern as project-side activityRelations_bucket_routing).

activities_filtered.pipe switches collection reads from activityRelations_collection_bucket_union to that routing pipe. Callers must pass bucketId with collectionSlug (from collection_buckets.pipe); the WHERE clause also filters a.collectionSlug. collectionSlug without bucketId returns no rows (1 = 0)—no union fallback—so existing collection-only callers need the companion insights work to pass bucketId.

Reviewed by Cursor Bugbot for commit 211b03e. Bugbot is set up for automated code reviews on this repo. Configure here.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds optimized single-bucket routing for pre-resolved collection activity queries while retaining union-based fallback behavior.

Changes:

  • Adds collection bucket routing across buckets 0–9.
  • Uses direct routing when both collectionSlug and bucketId are provided.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File Description
activityRelations_collection_bucket_routing.pipe Selects the requested collection bucket datasource.
activities_filtered.pipe Adds bucketId support and collection filtering.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread services/libs/tinybird/pipes/activityRelations_collection_bucket_routing.pipe Outdated

@epipav epipav left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added two comments

Comment thread services/libs/tinybird/pipes/activities_filtered.pipe
Comment thread services/libs/tinybird/pipes/activities_filtered.pipe Outdated
Copilot AI review requested due to automatic review settings August 17, 2026 09:14
Signed-off-by: Gašper Grom <gasper.grom@gmail.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.

Suppressed comments (1)

services/libs/tinybird/pipes/activityRelations_collection_bucket_routing.pipe:17

  • An invalid but defined bucketId (for example 10) reaches this fallback and silently reads bucket 0. activities_filtered then filters by collectionSlug, so collections assigned to buckets 1–9 return empty results instead of preserving the union behavior or rejecting the input. Use the existing union as the fallback.
        {% end %} as selected_bucket

Copilot AI review requested due to automatic review settings August 17, 2026 09:17

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.

Signed-off-by: Gašper Grom <gasper.grom@gmail.com>
Copilot AI review requested due to automatic review settings August 17, 2026 09:20

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.

Suppressed comments (1)

services/libs/tinybird/pipes/activities_filtered.pipe:34

  • Validate bucketId before selecting the single-bucket pipe. defined(bucketId) accepts values outside 09; the routing pipe then falls back to bucket 0, so a malformed parameter can silently return no activities for a collection assigned to another bucket. Only use the optimized route for a valid bucket, and otherwise retain the union path (or reject the request).
        {% if defined(collectionSlug) and defined(bucketId) %}
            activityRelations_collection_bucket_routing

@gaspergrom
gaspergrom requested a review from epipav August 17, 2026 11:25
…ssing (IN-1231)

Signed-off-by: Gašper Grom <gasper.grom@gmail.com>
Copilot AI review requested due to automatic review settings August 17, 2026 11:48

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

SELECT activityId as id, timestamp, type, platform, memberId, organizationId, segmentId
FROM
{% if defined(collectionSlug) %} activityRelations_collection_bucket_union
{% if defined(collectionSlug) %} activityRelations_collection_bucket_routing
Copilot AI review requested due to automatic review settings August 18, 2026 11:28
@gaspergrom
gaspergrom merged commit 017ff36 into main Aug 18, 2026
15 checks passed
@gaspergrom
gaspergrom deleted the feat/IN-1231-collection-bucket-routing branch August 18, 2026 11:29

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.

Suppressed comments (1)

services/libs/tinybird/pipes/activities_filtered.pipe:33

  • This removes the documented backward compatibility: every existing collection-scoped caller that sends only collectionSlug now selects the routing pipe and is forced to 1 = 0, so all downstream metrics return empty results until every caller is updated. This also contradicts the PR description, which says collectionSlug-only requests continue using the union and that the change is inert without the companion PR. Keep the union as the fallback and select the routing pipe only when both parameters are defined.
        {% if defined(collectionSlug) %} activityRelations_collection_bucket_routing

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants