Skip to content

feat: Add subgraph sampling as reusable multi-hop subgraph sampling foundation - #831

Open
aotenjou wants to merge 7 commits into
apache:masterfrom
aotenjou:subgraph-sampling
Open

feat: Add subgraph sampling as reusable multi-hop subgraph sampling foundation#831
aotenjou wants to merge 7 commits into
apache:masterfrom
aotenjou:subgraph-sampling

Conversation

@aotenjou

@aotenjou aotenjou commented Aug 12, 2026

Copy link
Copy Markdown

What changes were proposed in this pull request?

This PR introduces a model-agnostic subgraph sampling foundation for graph learning algorithms such as GCN and GNN and an inner implementation of #502.

Changes

  • Add fanout-based one-hop neighbor sampling

    • fanout > 0 selects at most the requested number of neighbors
    • fanout = -1 returns all neighbors matching the requested direction
    • Parallel edges belonging to the same neighbor are retained as a group
    • Sampling supports seeds, sampling versions, and returned-edge limits
  • Add layered sampled-subgraph assembly

    • Assemble vertices and edges by sampling depth
    • At the terminal depth, return only vertices and their features without reading or returning adjacent edges
    • Enforce sampled-node and sampled-edge limits
    • Reject neighborhoods from inconsistent snapshot versions
  • Add stable logical edge identities

    • Identify logical edges using sourceId + targetId + label + time
    • Deduplicate incoming and outgoing storage replicas of the same logical edge
    • Preserve parallel edges with different labels or timestamps
  • Add an iterative multi-hop BSP protocol

    • Define REQUEST, RESPOND, COMMIT_AND_REQUEST, and COMPLETE phases
    • Add sampling clocks, request/response messages, response collectors, and per-round state
    • Support empty request/response barriers
    • Cover complete two-hop and three-hop message schedules
  • Integrate sampling with the DSL algorithm runtime

    • Provide one-hop sampling contexts for static and dynamic graphs
    • Forward initIteration and finishIteration lifecycle callbacks
    • Read from a stable window snapshot during sampling
    • Track neighborhood change versions to determine when cached state must be refreshed
    • Register both source and target vertices when processing edge changes

Testing

The following local validation has passed:

  • 453 tests across the four affected modules under JDK 8
  • 47 focused sampling tests after splitting the changes into atomic commits

Coverage includes:

  • Fanout, direction, seed, and sampling-version behavior
  • Unlimited sampling with fanout = -1
  • Parallel-edge grouping and logical-edge deduplication
  • Empty edge lists at the terminal depth
  • Snapshot consistency and capacity limits
  • Complete two-hop and three-hop BSP message schedules
  • Dynamic neighborhood change-version persistence
  • Static and dynamic runtime lifecycle forwarding

How was this PR tested?

  • Tests have Added for the changes
  • Production environment verified

@aotenjou aotenjou changed the title feat: Subgraph reusable multi-hop subgraph sampling foundation feat: Add subgraph sampling as reusable multi-hop subgraph sampling foundation Aug 12, 2026
@aotenjou
aotenjou marked this pull request as draft August 12, 2026 05:37
@aotenjou
aotenjou marked this pull request as ready for review August 19, 2026 07:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant