Skip to content

feat(review): FSRS-6 scheduling state and review history (#238 phase 2a) - #261

Merged
HugoFara merged 2 commits into
developfrom
feature/fsrs-scheduling-2a
Aug 5, 2026
Merged

feat(review): FSRS-6 scheduling state and review history (#238 phase 2a)#261
HugoFara merged 2 commits into
developfrom
feature/fsrs-scheduling-2a

Conversation

@HugoFara

@HugoFara HugoFara commented Aug 5, 2026

Copy link
Copy Markdown
Owner

Implements phase 2a of #238, following the review in this comment.

LWT now records real spaced-repetition memory state per term — Stability, Difficulty, a due date, and an append-only review log — using FSRS-6, the algorithm Anki uses.

Deliberately additive and invisible

The legacy Leitner scoring keeps running untouched, nothing reads the new state to pick review words yet, and reading-view colours are unchanged. The whole phase is reversible, and both schedulers can be compared on real data before anything is retired in 2b.

The three gating decisions

1. WoStatus stays manual, not derived from stability. Against the original proposal. Deriving colours from FSRS imports an Anki assumption that doesn't hold here — in Anki every card is reviewed; in LWT review is optional and reading is the primary loop. Users who never open the review page would watch words they deliberately marked known drift back on their own. It also makes "colours unchanged after upgrade" true by construction rather than something to verify.

2. Four grades, no 2-button mode. Hard/Easy carry the signal that makes FSRS beat SM-2. The legacy binary answer maps to Again/Good via Rating::fromBinary(), so existing callers keep working without a second mode to maintain.

3. Hand-port, not a vendored dependency.

Verdict
fsrs-rs-php (official) Needs a hand-compiled Rust PHP extension — no Composer install. Disqualifying for self-hosters; we just fixed Windows CI for a bundled extension in #259
scottlaurent/fsrs Pure PHP and MIT, but v0.1 / one commit / no reference validation, and declares PHP 8.1–8.3 against our 8.2–8.5

Fsrs6Scheduler is a port of py-fsrs v6.3.1 and carries that project's MIT notice; the rest of LWT stays public domain.

What the reference vectors caught

Pinning the version matters. py-fsrs's unreleased main widens the short-term stability clamp from (Good, Easy) to (Hard, Good, Easy) — turning a same-day Hard from a ~44% stability cut into a no-op. I ported from main first; the vectors failed and caught it. That is the entire argument for generating them rather than trusting a read of the source.

Seeding

Existing terms seed lazily on their first graded review, mapping each status to the stability that reproduces its legacy interval (1/2/9/27/71 days for statuses 1–5) with lastReview = WoStatusChanged. No bulk backfill — a 100k-term vocabulary costs nothing at upgrade and nobody's queue floods. 98/99 are never scheduled.

RecordScheduledReview runs from SubmitAnswer as a shadow write: only after the legacy update succeeded, and swallowing storage errors so a scheduling failure can never break the review the user just submitted.

Migration gotcha worth knowing

The FK columns are int(10) unsigned, not the mediumint(8) that db/schema/baseline.sql still declares — 20251221_120000 widens words.WoID, and a mismatch fails with errno 150. The test-DB runner tolerates failed statements silently, so this presented as tables that simply never appeared rather than as an error. Documented inline in the migration.

Verification

  • 24 reference vectors across 7 review sequences, asserting stability, difficulty, retrievability and interval at every step to 1e-9. Fixture + generator committed; regeneration is a documented three-line recipe.
  • Property tests: difficulty saturates within [1,10] under 40 consecutive Again and 40 consecutive Easy; a lapse never raises stability; Hard < Good < Easy from identical state; retrievability is exactly 0.9 after one stability period; stricter retention schedules sooner.
  • Integration against real MySQL: lazy seeding per status, state upsert vs. append-only log, lapse counting, due counting, null for unowned/missing terms.
  • Gates: psalm 0 errors, phpcs PSR12 clean, typecheck clean, 9130 tests green (31 new), integration suite 163.

What's explicitly not here

No 4-grade UI, no queue reordering, no interval fuzzing (it only exists to spread Anki's daily load), no parameter optimiser (needs accumulated history — which this PR starts collecting). All 2b.

Also worth noting for #228: term_schedule.TsState uses Anki's cards.type values, and review_log maps onto Anki's revlog, so the .apkg exporter can populate cards.data + revlog in 2b with no format change.

Records real spaced-repetition memory state per term — Stability,
Difficulty, due date, plus an append-only review log — using FSRS-6, the
algorithm Anki uses.

Deliberately additive and invisible. The legacy Leitner scoring keeps
running untouched, nothing reads the new state to pick review words yet,
and reading-view colours are unchanged. That makes the whole phase
reversible and lets both schedulers be compared on real data before
anything is retired (phase 2b).

Decisions behind this, resolving the three questions #238 was gated on:

* WoStatus stays MANUAL, not derived from stability. Deriving colours
  from FSRS imports an Anki assumption that does not hold here: in Anki
  every card is reviewed, whereas in LWT review is optional and reading
  is the primary loop. Users who never open the review page would see
  words they deliberately marked known drift back on their own. This
  also makes "colours unchanged after upgrade" true by construction.
* Four grades, no 2-button mode. Hard/Easy carry the signal that makes
  FSRS beat SM-2; the legacy binary answer maps to Again/Good.
* Hand-port, not a vendored dependency. The official fsrs-rs-php needs a
  hand-compiled Rust extension (disqualifying for self-hosters — we just
  fixed Windows CI for a *bundled* extension in #259), and the only
  pure-PHP package on Packagist is a single-commit v0.1 declaring PHP
  8.1-8.3 against our 8.2-8.5.

Fsrs6Scheduler is a port of py-fsrs v6.3.1 and carries that project's
MIT notice; the rest of LWT stays public domain. Pinning the version
matters: py-fsrs's unreleased main widens the short-term stability clamp
from (Good, Easy) to (Hard, Good, Easy), turning a same-day Hard from a
~44% stability cut into a no-op. The reference vectors caught exactly
that bug in the first draft of the port.

Existing terms seed lazily on their first graded review, mapping each
status to the stability that reproduces its legacy interval
(1/2/9/27/71 days for statuses 1-5) with lastReview = WoStatusChanged.
No bulk backfill, so a 100k-term vocabulary costs nothing at upgrade and
nobody's queue floods. 98/99 are never scheduled.

RecordScheduledReview runs from SubmitAnswer as a shadow write: only
after the legacy update succeeded, and swallowing storage errors so a
scheduling failure can never break the review the user just submitted.

The migration's FK columns are int(10) unsigned, not the mediumint(8)
that db/schema/baseline.sql still declares — 20251221_120000 widens
words.WoID, and a mismatch fails with errno 150. The test-DB runner
tolerates failed statements silently, so this presented as tables that
simply never appeared.

Verified: 24 reference vectors across 7 sequences to 1e-9, property
tests (difficulty saturation, lapses never raise stability,
Hard<Good<Easy, retrievability 0.9 after one stability period), and
integration tests against real MySQL for seeding, upsert-vs-append and
due counting. psalm 0 errors, phpcs PSR12 clean, 9130 tests green.
Both ran long for a changelog — rationale and implementation detail belong
in the design doc and PR, not here. Cut to what a user needs to know, with
a pointer to the docs page for the rest.
@HugoFara
HugoFara merged commit 4b286e1 into develop Aug 5, 2026
14 checks passed
@HugoFara
HugoFara deleted the feature/fsrs-scheduling-2a branch August 6, 2026 12:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant