feat(anki): import a deck built in Anki to seed known words (#228)

CHANGELOG.md

@@ -9,6 +9,16 @@ ones are marked like "v1.0.0-fork".
 
 ### Added
 
+* **Import an Anki deck to seed known words** (#228): point LWT at a deck you
+  already study in Anki and it creates the terms for you, working out how well
+  you know each word from Anki's own scheduling — mature cards become *well
+  known*, younger ones get a learning status, suspended ones become *ignored*.
+  You pick the note type, which field holds the term, and the language, since an
+  `.apkg` records none of that. Importing is create-only, so running it twice is
+  safe. This is the direction #259 did **not** cover: that one round-trips LWT's
+  own exports and silently matches nothing in a deck built in Anki. See
+  `docs/reference/anki-deck-import`.
+
 * **FSRS scheduling groundwork** (#238, phase 2a): LWT now records FSRS-6 memory
   state per term — stability, difficulty, due date and a review history.
   Nothing user-visible changes yet: the legacy scoring still drives the review

docs-src/.vitepress/config.ts

@@ -81,6 +81,7 @@ export default defineConfig({
             { text: 'Text Parsers', link: '/reference/parsers' },
             { text: 'Lemmatization', link: '/reference/lemmatization' },
             { text: 'Anki Export & Import', link: '/reference/anki-export-import' },
+            { text: 'Import an Anki Deck', link: '/reference/anki-deck-import' },
             { text: 'Term Scores', link: '/reference/term-scores' },
             { text: 'Export Templates', link: '/reference/export-templates' },
             { text: 'Database Schema', link: '/reference/database-schema' },

docs-src/reference/anki-deck-import.md

@@ -0,0 +1,118 @@
+---
+title: "Import an Anki Deck"
+description: Seed your LWT vocabulary from a deck you already study in Anki, using Anki's own scheduling data to work out which words you already know.
+---
+
+# Import an Anki Deck
+
+If you already study a language in Anki, you probably know thousands of words
+that LWT does not know about yet. Importing your deck marks those words as known
+in one step, instead of you reclassifying each one as you meet it while reading.
+
+**Where:** *Terms → Import an Anki deck*, or `/vocabulary/anki-deck/import`.
+
+::: tip This is not the same as Anki Export & Import
+[Anki Export & Import](/reference/anki-export-import) is a **round trip** for
+terms that started life in LWT: export them, study in Anki, import the file back
+to update them. It matches notes by an `lwt-` identifier that only LWT-exported
+files carry.
+
+This page is the opposite direction: a deck **you built in Anki**, which has no
+such identifier, imported to **create new terms**. Feeding an Anki-built deck to
+the round-trip importer does nothing at all — it finds nothing to match.
+:::
+
+## Getting the file out of Anki
+
+In Anki: **File → Export**, choose **Anki Deck Package (.apkg)**, pick the deck,
+and keep **Include scheduling information** ticked. The scheduling data is what
+lets LWT tell an word you have known for a year from one you saw yesterday.
+
+## The import in two steps
+
+**1. Upload.** LWT reads the file and lists the note types it contains, with
+their field names and note counts.
+
+**2. Map.** An `.apkg` records neither a language nor which field means what, so
+you choose:
+
+| Choice | Why LWT has to ask |
+| --- | --- |
+| **Note type** | A deck can contain several; you may only want one |
+| **Term field** | Field names are arbitrary — `Front`, `Expression`, `Word`, `Vocab`… |
+| **Translation field** | Optional. Pick `(none)` to import words only |
+| **Language** | An `.apkg` does not record one |
+| **Word status** | Derived from Anki, or one fixed status for everything |
+| **Import tags** | Anki tags become LWT term tags |
+
+## How word status is decided
+
+With **Derive from Anki** (the default), each note's status comes from how well
+Anki thinks you know it — specifically the card's interval:
+
+| In Anki | Becomes in LWT |
+| --- | --- |
+| Suspended | **98** — Ignored |
+| Never studied | **1** — Learning, level 1 |
+| Interval 1–6 days | **2** |
+| Interval 7–13 days | **3** |
+| Interval 14–20 days | **4** |
+| Interval 21+ days | **99** — Well known |
+
+The 21-day line is Anki's own definition of a *mature* card, the same threshold
+its statistics screen uses, so it should match your intuition about the deck.
+
+A note with several cards (forward and reverse, say) is judged by its **strongest**
+card: knowing the word in one direction is enough to count. A note only counts as
+suspended if *every* one of its cards is suspended.
+
+If that mapping does not suit your deck, choose **Give every word the same
+status** instead — useful for a deck you have fully mastered (set everything to
+*Well known*) or one you are only starting.
+
+## What gets cleaned up
+
+Anki fields are HTML, and real decks lean on it. Before storing a term LWT
+removes formatting tags, converts `<br>` and block boundaries to spaces, decodes
+entities like `&nbsp;` and `&eacute;`, drops `[sound:…]` media references, and
+keeps the answer out of `{{c1::…}}` cloze markers. Without that, markup would end
+up in the reading view and terms would never match the words in your texts.
+
+## Importing twice is safe
+
+The importer only ever **creates**. It never modifies or deletes a term you
+already have, so re-importing the same deck — or a bigger version of it later —
+adds only what is genuinely new. Words already in LWT are reported as
+"already in LWT" and left alone.
+
+Duplicates *within* one file are collapsed case-insensitively, so forward and
+reverse notes for the same word produce one term.
+
+## Reading the summary
+
+The summary accounts for every note read:
+
+- **Terms created** — new terms now in your vocabulary
+- **Already in LWT** — the word existed, so it was left untouched
+- **Skipped (empty term field)** — the note had nothing in the chosen field
+- **Skipped (too long to store)** — the term exceeded 250 characters
+
+If *everything* was skipped as empty, you almost certainly picked the wrong term
+field. Go back and choose another; nothing was written.
+
+## Limits
+
+- **Only `.apkg` files.** Anki's newer `.colpkg` whole-collection format and
+  compressed exports are not supported yet.
+- **Requires `pdo_sqlite`**, since an `.apkg` is a zipped SQLite database. See
+  [Anki Export & Import](/reference/anki-export-import#requirements).
+- **No sentences or images.** Only the term, its translation and its tags are
+  imported.
+- **Scheduling is read, not carried over.** Anki's intervals decide the starting
+  status, but LWT then schedules the term with its own system — the two do not
+  stay in sync. Live sync is tracked separately and is deliberately deferred.
+
+## See also
+
+- [Anki Export & Import](/reference/anki-export-import) — the LWT→Anki→LWT round trip
+- [Term Scores](/reference/term-scores) — how LWT schedules reviews afterwards

docs-src/reference/anki-export-import.md

@@ -2,6 +2,16 @@
 
 LWT can export your vocabulary to a real Anki package (`.apkg`) and read changes back from a `.apkg` you re-export from Anki. The flow is **manual round-trip** — not a live sync — but the file format is universal, so any Anki client (desktop, AnkiMobile, AnkiDroid, AnkiWeb) can study the deck.
 
+::: warning Importing a deck you built in Anki?
+This page covers the **round trip** for terms that started in LWT: export them,
+study in Anki, import the file back to update them. Notes are matched by an
+`lwt-` identifier that only LWT-exported files carry, so a deck built in Anki or
+downloaded from AnkiWeb has nothing to match and **nothing would change**.
+
+To seed LWT from a deck you already study, use
+[Import an Anki Deck](/reference/anki-deck-import) instead.
+:::
+
 ## When to use this
 
 - You already have an Anki workflow and want LWT to seed it from your reading.

src/Modules/Vocabulary/Application/Services/Anki/AnkiDeckImportService.php

@@ -0,0 +1,137 @@
+<?php
+
+declare(strict_types=1);
+
+namespace Lwt\Modules\Vocabulary\Application\Services\Anki;
+
+use Lwt\Modules\Tags\Application\Services\TermTagService;
+use Lwt\Modules\Vocabulary\Application\Services\WordCrudService;
+use Lwt\Modules\Vocabulary\Infrastructure\Anki\AnkiFieldText;
+use Lwt\Modules\Vocabulary\Infrastructure\Anki\ForeignApkgReader;
+use Lwt\Modules\Vocabulary\Infrastructure\Anki\ForeignNote;
+
+/**
+ * Creates LWT terms from a deck the user built in Anki (issue #228).
+ *
+ * This is the "seed my known words" path: rather than reclassifying thousands
+ * of words by hand while reading, the user points LWT at a deck they already
+ * study and LWT infers each word's status from Anki's own scheduling data.
+ *
+ * Distinct from {@see ApkgImportService}, which merges an LWT-exported file
+ * back into the terms it came from. That one matches on `lwt-` guids and
+ * updates; this one has no guids to match and only ever creates.
+ */
+final class AnkiDeckImportService
+{
+    /** `words.WoText` is varchar(250). */
+    private const MAX_TERM_LENGTH = 250;
+
+    /** How many created terms to show back in the summary. */
+    private const SAMPLE_SIZE = 8;
+
+    public function __construct(
+        private readonly ForeignApkgReader $reader,
+        private readonly WordCrudService $words,
+    ) {
+    }
+
+    public static function default(): self
+    {
+        return new self(new ForeignApkgReader(), new WordCrudService());
+    }
+
+    public function import(string $apkgPath, DeckImportSettings $settings): DeckImportResult
+    {
+        $notes = $this->reader->notes($apkgPath, $settings->notetypeId);
+
+        $created = 0;
+        $skippedExisting = 0;
+        $skippedEmpty = 0;
+        $skippedTooLong = 0;
+        $statusCounts = [];
+        $samples = [];
+
+        // Terms already created in this run, so a deck containing the same word
+        // twice (common with forward/reverse notes split across notetypes)
+        // does not report a spurious "already existed".
+        $seen = [];
+
+        foreach ($notes as $note) {
+            $term = $this->cleanFieldValue($note->field($settings->termField));
+
+            if ($term === '') {
+                $skippedEmpty++;
+                continue;
+            }
+
+            if (mb_strlen($term) > self::MAX_TERM_LENGTH) {
+                $skippedTooLong++;
+                continue;
+            }
+
+            $termLc = mb_strtolower($term, 'UTF-8');
+            if (isset($seen[$termLc])) {
+                $skippedExisting++;
+                continue;
+            }
+            $seen[$termLc] = true;
+
+            $status = $settings->statusFor($note);
+
+            $result = $this->words->create([
+                'WoLgID' => $settings->languageId,
+                'WoText' => $term,
+                'WoStatus' => $status,
+                'WoTranslation' => $this->translationFor($note, $settings),
+                'WoSentence' => '',
+                'WoNotes' => '',
+                'WoRomanization' => '',
+            ]);
+
+            if ($result['success'] !== true) {
+                // WordCrudService reports a duplicate as a failed create; the
+                // unique key on (WoTextLC, WoLgID) is what actually enforces it,
+                // so this is the authoritative "already had it" signal.
+                $skippedExisting++;
+                continue;
+            }
+
+            $created++;
+            $statusCounts[$status] = ($statusCounts[$status] ?? 0) + 1;
+
+            if (count($samples) < self::SAMPLE_SIZE) {
+                $samples[] = $term;
+            }
+
+            if ($settings->importTags && $note->tags !== []) {
+                TermTagService::saveWordTags($result['id'], $note->tags);
+            }
+        }
+
+        ksort($statusCounts);
+
+        return new DeckImportResult(
+            totalNotes: count($notes),
+            created: $created,
+            skippedExisting: $skippedExisting,
+            skippedEmpty: $skippedEmpty,
+            skippedTooLong: $skippedTooLong,
+            statusCounts: $statusCounts,
+            samples: $samples,
+        );
+    }
+
+    private function translationFor(ForeignNote $note, DeckImportSettings $settings): string
+    {
+        if ($settings->translationField === null) {
+            return '';
+        }
+
+        return $this->cleanFieldValue($note->field($settings->translationField));
+    }
+
+    private function cleanFieldValue(string $value): string
+    {
+        return AnkiFieldText::toPlainText($value);
+    }
+}

src/Modules/Vocabulary/Application/Services/Anki/DeckImportResult.php

@@ -0,0 +1,34 @@
+<?php
+
+declare(strict_types=1);
+
+namespace Lwt\Modules\Vocabulary\Application\Services\Anki;
+
+/**
+ * Outcome of importing a foreign Anki deck.
+ *
+ * Every note is accounted for in exactly one bucket, so the totals add up and
+ * the user can see why a big deck produced fewer terms than they expected.
+ */
+final class DeckImportResult
+{
+    /**
+     * @param int                $totalNotes    Notes read for the chosen notetype
+     * @param int                $created       New LWT terms created
+     * @param int                $skippedExisting Terms LWT already had
+     * @param int                $skippedEmpty  Notes whose term field was blank
+     * @param int                $skippedTooLong Terms exceeding the WoText column
+     * @param array<int, int>    $statusCounts  status => number of terms created
+     * @param list<string>       $samples       A few created terms, for the summary
+     */
+    public function __construct(
+        public readonly int $totalNotes,
+        public readonly int $created,
+        public readonly int $skippedExisting,
+        public readonly int $skippedEmpty,
+        public readonly int $skippedTooLong,
+        public readonly array $statusCounts,
+        public readonly array $samples,
+    ) {
+    }
+}