Problem
The SQLite driver currently accepts MySQL character sets that it cannot represent safely. The SQL lexer expects UTF-8 input, and SQLite text values are stored as UTF-8.
This becomes visible in WordPress when column metadata reports a non-UTF-8 character set. wpdb::strip_invalid_text() then generates CONVERT(... USING charset) queries containing the original encoded bytes, which the driver cannot parse or convert correctly.
Supported character sets
Limit text support to character sets whose encoded values are valid UTF-8:
utf8mb4
utf8mb3 / utf8
ascii
binary values and types
Other character sets, including Big5, KOI8-R, Latin-1, CP1251, Hebrew, and TIS-620, are unsupported. They should not be accepted for columns or CONVERT(... USING charset) expressions.
Expected behavior
- Reject unsupported character sets in
CREATE TABLE and ALTER TABLE with a clear driver error.
- Reject unsupported character sets in
CONVERT(... USING charset).
- Implement correct behavior for supported UTF-8, ASCII, and binary conversions instead of treating every conversion as a no-op.
- Ensure column metadata only exposes supported character sets.
- Define the behavior for existing SQLite databases whose emulated metadata contains unsupported character sets.
- Add coverage for DDL, metadata,
CONVERT, and WordPress CRUD behavior at the supported-character-set boundary.
Problem
The SQLite driver currently accepts MySQL character sets that it cannot represent safely. The SQL lexer expects UTF-8 input, and SQLite text values are stored as UTF-8.
This becomes visible in WordPress when column metadata reports a non-UTF-8 character set.
wpdb::strip_invalid_text()then generatesCONVERT(... USING charset)queries containing the original encoded bytes, which the driver cannot parse or convert correctly.Supported character sets
Limit text support to character sets whose encoded values are valid UTF-8:
utf8mb4utf8mb3/utf8asciibinaryvalues and typesOther character sets, including Big5, KOI8-R, Latin-1, CP1251, Hebrew, and TIS-620, are unsupported. They should not be accepted for columns or
CONVERT(... USING charset)expressions.Expected behavior
CREATE TABLEandALTER TABLEwith a clear driver error.CONVERT(... USING charset).CONVERT, and WordPress CRUD behavior at the supported-character-set boundary.