Escape special characters in Enum/DateTime type names#538
Open
slabko wants to merge 1 commit into
Open
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
Adds symmetric escaping/unescaping of ClickHouse C-style escape sequences inside single-quoted string literals that appear in type names (Enum labels and DateTime/DateTime64 timezones), preventing mis-parsing/corruption for values containing ', \, NUL, or control bytes.
Changes:
- Encode: introduce
EscapeStringLiteral()and apply it toEnumType::GetName(),DateTimeType::GetName(), andDateTime64Type::GetName(). - Decode: update
TypeParser’s single-quoted-string lexer to unescape supported sequences into a scratch buffer. - Tests: add unit + integration coverage for escaped Enum labels and DateTime/DateTime64 timezones, plus CreateColumnByType round-trips.
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
clickhouse/types/types.cpp |
Adds escaping map + helper and applies escaping when rendering Enum/DateTime type names. |
clickhouse/types/type_parser.cpp |
Adds unescape map and updates quoted-string tokenization to decode escape sequences. |
clickhouse/types/type_parser.h |
Clarifies token StringView lifetime and scratch-buffer purpose for processed identifiers/strings. |
ut/types_ut.cpp |
Adds unit tests verifying escaping in Type::GetName() for Enum labels and DateTime timezones. |
ut/type_parser_ut.cpp |
Adds parsing tests for escaped Enum labels and escaped DateTime/DateTime64 timezones (including all supported escapes). |
ut/CreateColumnByType_ut.cpp |
Adds CreateColumnByType tests for escaped inputs and render→parse round-trips; includes tuple header for tuple round-trip test. |
ut/client_ut.cpp |
Adds live-server integration test for Enum escape round-trip via server-reported schema. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+262
to
+266
| scratch_.clear(); | ||
| scratch_ += *(cur_++); // quotes must be included | ||
| for (; end_ - cur_ > 1; ++cur_) { | ||
| if (*cur_ == '\\' && end_ - cur_ > 1 && kUnescapeMap[(uint8_t)*(cur_ + 1)] != 0) { | ||
| scratch_ += kUnescapeMap[(uint8_t)*(cur_ + 1)]; |
Contributor
Author
There was a problem hiding this comment.
This is, unfortunately, true and can result in errors different from what they really are if the type name is incorrect. However this fix require a separate PR.
slabko
force-pushed
the
escape-in-identifiers
branch
4 times, most recently
from
July 21, 2026 21:08
d2a8b1f to
d3258fe
Compare
ClickHouse renders string literals inside type names — Enum labels (`Enum8('foo' = 1)`) and
DateTime/DateTime64 timezones (`DateTime('Europe/Amsterdam')`) — with C-style escape sequences for
control and grammar-significant bytes. Previously `clickhouse-cpp` neither produced these escapes
when generating a type name (`Type::GetName()`) nor decoded them when parsing a type string, so any
label/timezone containing `'`, `\`, or a control byte could be corrupted or mis-parsed.
This PR makes escaping symmetric in both directions for the following bytes:
| byte | escaped |
|------|---------|
| `0x00` NUL | `\0` |
| `0x08` BS | `\b` |
| `0x09` TAB | `\t` |
| `0x0A` LF | `\n` |
| `0x0C` FF | `\f` |
| `0x0D` CR | `\r` |
| `0x5C` `\` | `\\` |
| `0x27` `'` | `\'` |
**Decode (`clickhouse/types/type_parser.cpp`)**
- The single-quoted-string lexer now unescapes escape sequences instead of copying bytes verbatim,
via a new `kUnescapeMap` plus explicit handling of `\'` and `\0`.
**Encode (`clickhouse/types/types.cpp`)**
- New `EscapeStringLiteral()` helper backed by `kEscapeMap`.
- Applied in `EnumType::GetName()`, `DateTimeType::GetName()`, and `DateTime64Type::GetName()`.
- **Tuples are intentionally out of scope.** Tuple field-name identifiers use a different quoting
scheme (backticks) and were reverted to the existing implementation; only a benign non-escape tuple
round-trip test is included.
- **DateTime/DateTime64 escaping is client-side only in practice.** The server validates timezone
names against the tz database, so a timezone containing escape sequences can't be stored
server-side; the escape/unescape logic is still implemented and unit-tested for correctness.
slabko
force-pushed
the
escape-in-identifiers
branch
from
July 22, 2026 07:41
d3258fe to
2cc7c18
Compare
slabko
marked this pull request as ready for review
July 22, 2026 09:06
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
ClickHouse renders string literals inside type names - Enum labels (
Enum8('foo' = 1)) and DateTime/DateTime64 timezones (DateTime('Europe/Amsterdam')) - with C-style escape sequences for control and grammar-significant bytes. Previouslyclickhouse-cppneither produced these escapes when generating a type name (Type::GetName()) nor decoded them when parsing a type string, so any label/timezone containing',\, or a control byte could be corrupted or mis-parsed.This PR makes escaping symmetric in both directions for the following bytes:
0x00NUL\00x08BS\b0x09TAB\t0x0ALF\n0x0CFF\f0x0DCR\r0x5C\\\0x27'\'Decode (
clickhouse/types/type_parser.cpp)kUnescapeMapplus explicit handling of\'and\0.Encode (
clickhouse/types/types.cpp)New
EscapeStringLiteral()helper backed bykEscapeMap.Applied in
EnumType::GetName(),DateTimeType::GetName(), andDateTime64Type::GetName().Tuples are intentionally out of scope. Tuple field-name identifiers use a different quoting scheme (backticks) and were reverted to the existing implementation; only a benign non-escape tuple round-trip test is included.
DateTime/DateTime64 escaping is client-side only in practice. The server validates timezone names against the tz database, so a timezone containing escape sequences can't be stored server-side; the escape/unescape logic is still implemented and unit-tested for correctness.