Skip to content

Add interpretability/classifier: standalone set-classifier (train / eval / score) - #19

Merged
ahillsley merged 10 commits into
alexhillsley/public-releasefrom
alexlin/interpretability-classifier
Aug 13, 2026
Merged

Add interpretability/classifier: standalone set-classifier (train / eval / score)#19
ahillsley merged 10 commits into
alexhillsley/public-releasefrom
alexlin/interpretability-classifier

Conversation

@al5250

@al5250 al5250 commented Aug 12, 2026

Copy link
Copy Markdown
Collaborator

What

Adds src/ops_model/interpretability/classifier/ — a self-contained toolkit to train, evaluate, and score a permutation-invariant set classifier over single-cell embeddings. It predicts a gene knockout (or a protein-complex / pathway label) from a set ("bag") of cells, and runs independently of the source research repo: the only inputs are an embedding parquet and (for complex classification) a gene→label CSV (shipped), passed as paths.

Files

  • train.py — model (MixedChannelClassifier: an inducing-point set transformer that pools all a cell-set's channels into one prediction), the parquet/dump dataset + label-map logic, the training loop, and build_model (checkpoint → model). Hydra CLI.

  • eval.py — top-1/top-5 accuracy sweep vs. cells-per-set → curve PNG + JSON + per-class CSV. Hydra CLI.

  • score.py — per-cell attribution score (leave-one-out marginal averaged over bag sizes) → ranked CSV. argparse CLI.

  • configs/ — 4 train configs + 1 eval config, plus the shipped EBI complex label map ebi_complexes.csv (311 genes → complex/pathway):

    config task
    train_set_classifier_phase_1K.yaml phase, 1K gene
    train_set_classifier_phase_ebi.yaml phase, EBI complex
    train_set_classifier_fluor_1K.yaml fluor, all markers, 1K gene
    train_set_classifier_fluor_ebi.yaml fluor, all markers, EBI complex
    eval_set_classifier.yaml eval (one config for all four)

Usage

pip install -e ".[classifier]"   # hydra-core, omegaconf, pyarrow, tqdm, matplotlib

python -m ops_model.interpretability.classifier.train --config-name train_set_classifier_fluor_1K \
    data.parquet_entries='[{path:/path/to/embeddings.parquet}]' save_path=set_classifier.pt
python -m ops_model.interpretability.classifier.eval  --config-name eval_set_classifier \
    checkpoint_path=set_classifier.pt val_dump_dir=/path/to/dumps/val
python -m ops_model.interpretability.classifier.score --checkpoint set_classifier.pt \
    --dump_dir /path/to/dumps/train /path/to/dumps/val --channel Phase2D --genes KIF23 --out_csv score.csv

The two *_ebi configs point label_map_path at the shipped configs/ebi_complexes.csv. Fluor configs pool all markers per set (channels_per_set: null); set [1, 2, 5, 10, 20, null] to also train on smaller random marker subsets (inline comment in the configs).

Notes

  • Mixed-channel only. The classifier pools all of a set's channels into one prediction (mixed_channels_mode: true); the older two-level per-channel path (SetClassifier) was dropped for the release.
  • No W&B / cloud coupling. Training prints per-epoch val accuracy to stdout and can persist metrics via metrics_out.
  • Extracted ~verbatim from the source set-classifier (only imports, config paths, and dropped W&B/per-channel code differ). Verified equivalent: a full 200-epoch phase-EBI run reproduced the source's val accuracy (best 0.843 vs 0.840), and a 25-epoch run of the trimmed release code reproduced it again (0.749) after the per-channel removal.
  • Draft — opening for review; not marking ready yet.

🤖 Generated with Claude Code

al5250 and others added 10 commits August 11, 2026 18:49
…/SHAP

Adds src/ops_model/interpretability/classifier/, a self-contained toolkit to
train, evaluate, and explain a permutation-invariant set classifier over
single-cell embeddings. Predicts a gene knockout (or a protein-complex /
pathway label via a gene->label metadata CSV) from a set ("bag") of cells.

Runs independently of the source research repo: the only inputs are an
embedding parquet and optional label-map CSV, passed as paths.

- train.py  — model architecture (inducing-point set transformer,
              MixedChannelClassifier), parquet/dump datasets + label-map logic,
              training loop, and build_model (checkpoint -> model). Hydra CLI.
- eval.py   — top-1/top-5 accuracy sweep vs. cells-per-set; writes curve PNG +
              JSON + per-class CSV. Hydra CLI.
- shap.py   — per-cell SHAP attribution (leave-one-out marginal averaged over
              bag sizes) -> ranked CSV with cell coordinates. argparse CLI.
- configs/  — representative train/eval configs (fluor + phase, 1K gene + EBI
              complex); absolute cluster paths replaced with placeholders.
- README.md — inputs (parquet schema, label map), install, and usage.

Internal-repo imports were dropped: flatten_meta is inlined and build_model
lives in train.py; nothing pulls in zarr/crop/dump-creation code. W&B defaults
to disabled (offline) so training runs with no login.

pyproject: adds an optional `classifier` extra (hydra-core, omegaconf, wandb,
pyarrow, tqdm, matplotlib) and scoped ruff per-file-ignores for the copied
research scripts (native 100-col width, terse docstrings, × / σ notation).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Rename the per-cell attribution from "SHAP" to "score" throughout the
subpackage: shap.py -> score.py, the output CSV column `shap` -> `score`,
internal variables/prints, docstrings, and README (module `python -m
...classifier.score`, section, examples). No behavior change.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…tion

Public-release cleanup of the set-classifier subpackage:
- Keep 5 configs: phase 1K, phase EBI, fluor 1K (cps=null), fluor EBI
  (cps=null), and one eval config. Delete the generic train template and the
  per-modality EBI eval configs.
- Note in the two fluor configs (and README) that channels_per_set can be set
  to [1, 2, 5, 10, 20, null] to also train on smaller random marker subsets.
- Remove W&B entirely: drop `import wandb`, wandb.init/log/Artifact/finish from
  train.py, the wandb_artifact checkpoint branch from eval.py, the `wandb` extra
  from pyproject, and all wandb_* config keys / comments. Training still prints
  per-epoch val accuracy to stdout and can persist metrics via `metrics_out`.
- Point train.py's default config_name at the phase-1K config (the generic
  default was deleted).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…configs)

Remove the package README for the public release; the channels_per_set guidance
is inline in the two fluor configs. Update the package docstring to point at the
train/eval/score entry points instead of the README.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Rename the four train configs to short names: train_set_classifier_{phase,fluor}_{1K,ebi}.yaml.
- Ship the EBI complex label map (311 genes -> complex/pathway) in configs/ as
  20260518_EBI_complexes_v1_old_gene_names.csv, and point the two *_ebi configs'
  label_map_path at it (repo-root-relative). Un-ignore configs/*.csv in .gitignore.
- Fix train.py's default config_name to the renamed phase-1K config.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Rename the shipped EBI complex label map to ebi_complexes.csv and update the two
*_ebi configs' label_map_path to match.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
… parquet

Use the single celldino_paper_phenotyping_v2_160.parquet embedding source for the
phase-EBI train config.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ubpackage

Don't add subpackage-specific ruff ignores in pyproject; keep only the existing
tests/*.py entry.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
n_layers_channel only affects the two-level SetClassifier (mixed_channels_mode:
false). All shipped configs use mixed_channels_mode: true -> MixedChannelClassifier,
which ignores it. Remove the dead key.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
All shipped configs use mixed_channels_mode=true (MixedChannelClassifier), so
the two-level SetClassifier and its per-channel path were dead code. Remove:
- train.py: SetClassifier, SABBlock, PerturbationDataset, collate_perturbation,
  ChannelGroup/CollatedBatch, per-channel train_one_epoch/evaluate, and run()'s
  mixed_channels_mode=false branch (now raises if not mixed).
- eval.py: SetClassifier/PerturbationDataset/collate_perturbation imports,
  _evaluate_perturbation_with_topk, _GeneLabelRemap, and the non-mixed branches
  in run()/_run_sweep.
Mixed path (MixedChannelClassifier, datasets, train/eval_mixed, score, split_channels)
unchanged. Verified: a 25-epoch mixed-mode phase-EBI train->eval->score reproduces
the pre-removal val accuracy (0.749 vs 0.740/0.738).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@al5250
al5250 marked this pull request as ready for review August 13, 2026 00:24
@al5250
al5250 requested review from ahillsley and gav-sturm August 13, 2026 00:24
@ahillsley
ahillsley merged commit b5517cb into alexhillsley/public-release Aug 13, 2026
2 of 14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants