refactor(worker): retire KVCacheManager stats legacy + gate (Phase 5.1c, refs #175)

[Andrewxu313]

Description

Phase 2 / Slice 5 / sub-slice 1c — cleanup step. Issue #175 stays open (covers full KV decouple; 5.1c closes the stats tier only).

Removes the dual-path migration scaffolding for the 3 read-only KV stat helpers now that H20 compare-mode proved zero divergence.

Diff: −104 LOC

Removed	LOC
BATCHGEN_WORKER_KV_STATS_NATIVE / ..._COMPARE env reads + comment	8
3 _legacy_get_*_kv_* methods (original bodies)	85
Gate branches inside the 3 wrapper methods	21
Total deletion	114
Added (refactored single-path bodies)	10
Net	−104

Each helper collapses to a 1-line (or 4-line for the dict-returning one) delegation:

def _get_host_kv_free_pages(self) -> int:
    return self._make_kv_cache_manager().get_host_free_pages()

def _get_gpu_kv_free_pages(self) -> int:
    return self._make_kv_cache_manager().get_gpu_free_pages()

def _get_host_kv_utilization(self) -> Dict[str, int]:
    native_obj = self._make_kv_cache_manager().get_host_utilization(
        self._make_kv_utilization_request()
    )
    return _dataclasses.asdict(native_obj)

The lazy _kv_cache_manager field + _make_kv_cache_manager + _make_kv_utilization_request helpers stay (still needed to wire the TorchKVStatsBackend adapter on first use).

Validation (H20 dual-mode L2-2048 on commit 7102ab85)

Result
L2-2048 accuracy	71.09% (91/128) — within sampling band
kv_stats compare mismatch / AssertionError	0 across both nodes
Workload	128 prompts × decode steps × 3 method names × 16 ranks; backend ran 2× per call during compare-mode

The 3-retry run completed normally once external GPU contention from tairan-batchgen-fp8 was cleared.

Motivation

Refs #175. Part of milestone Worker decouple. Slice 5 sub-slice 1 complete; issue stays open for sub-slices 5.2–5.6 (page-table capacity, token-budget cache, allocation/I/O, watermark+planner, migration executor).

Type of Change

• Bug fix
• New feature
• Breaking change
• Documentation update

(Refactor — deletion of validated migration scaffolding.)

Checklist

• I have read the CONTRIBUTION guide.
• I have updated the tests (if applicable). tests/worker/test_kv_manager.py continues to cover KVCacheManager directly.
• I have updated the documentation (if applicable). N/A.

Pre-merge verification

• python -m py_compile batchgen/batchgen_worker.py succeeds.
• pytest tests/worker/ -x → 67/67 passing.
• grep -cE "_kv_stats_native|_kv_stats_compare|_legacy_get_*_kv_*|BATCHGEN_WORKER_KV_STATS" → 0 (zero residual gate/legacy refs).

Slice 5.1 cumulative impact

Stage	Worker LOC delta	Cumulative worker LOC
Pre-Slice-5 (post-Slice-4 main)	—	12,252
Phase 5.1 (PR #193 — port)	+0	12,252 (sibling adds 192 LOC + 234 test LOC)
Phase 5.1b (PR #194 — gate)	+87	12,339
Phase 5.1c (this PR — cleanup)	−104	12,235

Slice 5.1 net: −17 LOC on worker + 426 LOC in batchgen/worker/ + tests.

[drunkcoding]

Review note (risk classification): the removed gate defaulted to legacy — os.environ.get("BATCHGEN_WORKER_KV_STATS_NATIVE", "0") == "1" — so the legacy bodies being deleted were the default-active implementation, and this PR flips the runtime default to the native path. The H20 compare-mode evidence in the description covers this, but it means the change is not pure dead-code removal; it should gate on the GPU regression (build + gpu-regression) before merge.