Optional native acceleration extension (phpstanturbo, C++/PHP-CPP) by ondrejmirtes · Pull Request #6037 · phpstan/phpstan-src

ondrejmirtes · 2026-07-11T14:08:58Z

What this is

An optional native PHP extension that accelerates PHPStan by shadowing its hottest code paths with C++ implementations. PHPStan behaves identically without it — with the extension loaded, analysis output is bit-for-bit identical, just faster.

Measured (single-threaded self-analysis of src/, user CPU, interleaved A/B): 26.2% faster than baseline (63.5s vs 86.0s); pure-PHP path also gains ~5% for free from the extracted dispatch cache. Peak RSS unchanged.

How it works — the stub-shadowing pattern

Every shadowed piece follows three steps (see turbo-ext/README.md):

The PHP code is extracted into a dedicated class (ScopeOps, ExprHandlerDispatch, NodeScanner, ParserRunner, …) called unconditionally — this is what runs without the extension.
The extension implements the same class natively in the PHPStanTurbo namespace.
When enabled, TurboExtensionEnabler loads an empty stub final class Foo extends \PHPStanTurbo\Foo {} before the autoloader, so all code transparently gets the native implementation under the original name.

Shadowed so far: TrinaryLogic, ExpressionTypeHolder, ConditionalExpressionHolder, CombinationsHelper, PhpParser\NodeTraverser, ScopeOps (scope merge/invalidate/clone), ExprHandlerDispatch, NodeScanner, and a full native port of php-parser 5.8.0's LALR engine + all 482 reduce actions (ParserRunner seam; parsing is 3.2× faster, tables read at run time from the vendored parser so php-parser stays the single source of truth).

Keeping it honest

turbo-ext/shadowed-classes.json — generated manifest of shadowed pairs; CI enforces method parity, signature parity (reflection, incl. parameter names), manifest completeness, and empty-shell stubs.
turbo-ext/tests/parser-corpus.php — parses ~8K files with both implementations; serialized ASTs, collected errors and token streams must be byte-identical. Runs in CI.
Extension version = short SHA of the last commit touching the native sources or their PHP twins, hand-declared in version.h (actual) + TurboExtensionEnabler::EXPECTED_EXTENSION_VERSION (expected); a stale extension deactivates itself. CI verifies both against git history.
turbo-ext.yml: compile (ubuntu + macos, -Wall -Wextra -Werror), smoke/parity/corpus checks, artifact upload, then make tests and make phpstan with the extension installed, in parallel.

Notes for review

The C-language predecessor (verified at the same speedup) is preserved on turbo-c-extension.
PHPSTAN_TURBO=0 disables the extension at run time.
Design rules (boundary economics, ≥0.5%-or-revert, never shadow DI services) in turbo-ext/README.md; agent workflow in turbo-ext/CLAUDE.md.
A generator that derives the 482 native reduce actions from the vendored Php8.php (making php-parser bumps mechanical) is in progress and will be pushed to this branch.

🤖 Generated with Claude Code

https://claude.ai/code/session_01LVQhbBxDjDVAVeJVvLpWYE

+        uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
+
+      - name: "Install PHP"
+        uses: "shivammathur/setup-php@7c071dfe9dc99bdf297fa79cb49ea005b9fcadbc" # v2.37.1


ondrejmirtes

I think the .cpp sources are still a bit too like C-like, expected a bit more OOP tbh. If it's possible and you're going to rewrite it, make sure the benchmarks still show the same improvement.

ondrejmirtes · 2026-07-11T14:14:33Z

+      - name: "Tests"
+        run: "make tests"
+
+  phpstan:


tests and phpstan could be deduped to a single job with a matrix and run: ${{ matrix.script }}

+          fetch-depth: 0 # git log over turbo-ext/src needs full history
+
+      - name: "Install PHP"
+        uses: "shivammathur/setup-php@7c071dfe9dc99bdf297fa79cb49ea005b9fcadbc" # v2.37.1


+          '
+
+      - name: "Run"
+        run: ${{ matrix.script }}


phpstan-bot · 2026-07-12T20:59:54Z

This pull request has been marked as ready for review.

staabm

I have a few top level questions

is it documented somewhere how I can try this locally?
how do we expect people to install the extension into their local/ci env?
how do we expect people to make sure they have the correct version of the extension matching the phpstan version they are using?
how do we expect this to be maintained? does any PR which touches a shadowed class come with the appropriate CPP adjustments?
is phpstan-bot able/instructed to provide CPP adjustments for PHP based changes?
maybe we should have a separate make command which downloads a github action compiled extension version (PR artifact) into the local checkout - so people don't need to compile locally themselves.

and a few regarding the PR itself

CI Crashed in https://github.com/phpstan/phpstan-src/actions/runs/29208750010/job/86692937210?pr=6037
maybe some refactoring PR e.g. introducing ScopeOps, NodeScanner,ParserRunner can be landed separately to reduce the scope of this one here
do you want to keep OptimizedPsrAutoloaderLocator even though plain php benchmark suggested its not useful? I guess you measured it and its usefull with phpstanturbo?
do we need some min-php runtime version check for the extension to be used?

staabm · 2026-07-13T06:45:06Z

 	/**
 	 * @return ExprHandler<Expr>|null
 	 */
 	public static function resolve(Expr $expr, Container $container): ?ExprHandler


at this point I think we can just drop the ExprHandlerRegistry class and invoke ExprHandlerDispatch::exprHandlerFor directly in the 3 call-sites

staabm · 2026-07-13T06:49:23Z


-Enabling the extension
------------
+Note: on macOS/arm64 PHP-CPP needs a one-line patch in `zend/base.cpp`


I think we should have a patch file for this change

ondrejmirtes · 2026-07-13T18:03:03Z

@staabm All good questions! 😊 My idea is to ship built .so files for common OS/PHP version/arch combinations alongside phpstan.phar in the phpstan/phpstan package, and enable it with php -d extension=path-to-phpstan-turbo.so when launching child processes. That way most people will be able to take advantage of it effortlessly (PHPStan will just become faster for them).

I also want to publish the extension to PIE for people that just use phpstan.phar without Composer.

The version mismatch is guarded by an expected extension version in TurboExtensionEnabler constant.

Unit tests and make phpstan run both with/without turbo so we'll catch any out-of-sync implementation that way. If the PR doesn't provide an update to the CPP code, it's not a problem, LLM can catch up with it after merge.

Yeah, the OptimizedPsrAutoloaderLocator improvements seem measurable, especially on huge projects.

maybe we should have a separate make command which downloads a github action compiled extension

Yeah, that would be nice, can be done after this one is merged.

I also have an idea about #[ShadowedByTurboExtension] class attribute which will make some boilerplate in this PR disappear.

ondrejmirtes · 2026-07-13T18:05:41Z

Also because of PIE I likely will have to do a monorepo split-style separate repo for turbo-ext/ directory, but I'm checking that with James.

+          echo "built extension reports: $REPORTED, enabler expects: $EXPECTED"
+          [ "$REPORTED" = "$EXPECTED" ]
+
+      - uses: "ramsey/composer-install@65e4f84970763564f46a70b8a54b90d033b3bdda" # v4.0.0


+          persist-credentials: false
+
+      - name: "Install PHP"
+        uses: "shivammathur/setup-php@f3e473d116dcccaddc5834248c87452386958240" # v2.37.2


+          sudo bash -c "echo 'extension=$GITHUB_WORKSPACE/turbo-ext/phpstan_turbo.so' >> $(php -r 'echo php_ini_loaded_file();')"
+          php -m | grep phpstan_turbo
+
+      - uses: "ramsey/composer-install@65e4f84970763564f46a70b8a54b90d033b3bdda" # v4.0.0


MutatingScope carries code that is called millions of times per run: scope-table merging and invalidation, conditional expression bookkeeping, expression keys, and the yield scan. It is moved, unchanged, into plain classes that can be reasoned about - and replaced - on their own: - PHPStan\Analyser\ScopeOps - PHPStan\Node\NodeScanner The callers invoke them unconditionally; there is no branching and no behaviour change. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013rnxxBisCZU1tM6nxGwHAy

…lookup OptimizedPsrAutoloaderLocator asked every locator it had ever built whether it knows a symbol, so the cost of a lookup grew with the number of locators seen: a partial self-analysis run made 286,121 such calls to land a single hit. Each locator already knows which symbols it contains, so they are harvested once (via a new @internal getPresentSymbols(), which reuses the source locator's existing cache entry) into class/function/constant indexes. A lookup is then one hash access. Measured on a full single-threaded self-analysis: 2-3% less user CPU, output byte-identical. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013rnxxBisCZU1tM6nxGwHAy

An optional C++ extension that reimplements the classes extracted earlier. PHPStan behaves identically without it, just slower. Every shadowed class follows the same pattern: the extension implements PHPStanTurbo\Foo natively, and TurboExtensionEnabler requires an empty stub - final class Foo extends \PHPStanTurbo\Foo {} - before the Composer autoloader registers, so all PHP code keeps calling the original name. Instances must satisfy the original type hints, so factories instantiate the stub subclasses, whose names arrive through Runtime::configure() as ::class constants (the phar prefixes namespaces). Shadowed here: TrinaryLogic, ExpressionTypeHolder, ConditionalExpressionHolder, CombinationsHelper, ScopeOps, NodeScanner and PhpParser\NodeTraverser. The hot classes are registered with the raw Zend API rather than PHP-CPP's trampolines, which allocate a Php::Parameters per call - the boundary must be crossed per operation, never per element. The zero-cost wrappers in zv.h keep the C++ readable at no cost, and reg.h gives registration a fluent shape while still emitting raw zend structures. Drift between the two implementations is what would hurt, so it is checked: shadowed-classes.json pairs each PHP class with its .cpp and is generated, not hand-written; CI verifies method parity, manifest completeness, stub shape and native signature parity (arginfo parameter names must match, or named arguments would break in turbo mode only), and ties the extension version to the history of the code it shadows. The differential smoke test proves behaviour. The extension is version-pinned and a mismatched build is ignored; PHPSTAN_TURBO=0 disables it. Also removes conf/turbo.neon, left over from an earlier experiment: the two services it defined now carry #[AutowiredService] like everything else. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013rnxxBisCZU1tM6nxGwHAy

Parsing is the single biggest cost outside the analyser itself. This ports php-parser 5.8.0's LALR engine and node construction - ParserAbstract plus the generated Parser\Php8 - into the extension, shadowed through a new PHPStan\Parser\ParserRunner seam. Native parsing is 3.2x faster (0.28s vs 0.92s of user CPU over PHPStan's own sources). Nothing is duplicated that does not have to be: the parsing tables are generated data and are read at run time from the first Php8 parser object seen, node classes resolve relative to the parser's namespace (which keeps the scoped phar working), and node construction derives its property-slot writes from constructor parameter names. Tokenization stays in PHP's C tokenizer - one crossing per file. Non-Php8 parsers and non-string inputs fall back to $parser->parse(). The 482 semantic actions are generated from the vendored Php8.php by bin/generate-parser-actions.php, so a php-parser bump costs no hand work for the formulaic majority; the generator fails loudly on any closure it cannot prove it handles, and hand-ported special cases are keyed by body content so renumbering never disturbs them. Because the input domain is "all PHP source code", method parity is not a strong enough check: tests/parser-corpus.php parses 7,962 files with both implementations and requires byte-identical serialized ASTs, identical collected errors and identical token streams. It runs in CI on every build, and the workflow pins the php-parser version the engine was ported against so a composer.lock bump cannot silently drift. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013rnxxBisCZU1tM6nxGwHAy

Three places where a type's observable behaviour depended on something other than its value: - UnionType::getSortedTypes() and IntersectionType::getSortedTypes() sorted $this->types in place. Describing a union therefore permanently changed what getTypes() returned, and array shapes are merged in that order - so an inferred type could depend on whether anything had described it earlier. The sorted list is now cached separately and $types is left alone. - ObjectType::getClassReflection() wrote its lazily fetched reflection back into $classReflection, the same property the constructor sets, which describeCache() folds into the cache key. A warmed instance was then indistinguishable from one constructed with a reflection, making describe(VerbosityLevel::cache()) depend on whether the fetch had happened yet. The lazy one now lives in its own property; the constructor's stays part of the value. - AssignHandler and TemplateTypeTrait compared TypeCombinator::remove()'s result with the input by object identity, relying on remove() handing back the very same instance when it removes nothing. That is an implementation detail, not a contract - the falsey loop in AssignHandler already used equals(). Both now compare by value. Two expectations move because they encoded the in-place sort. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013rnxxBisCZU1tM6nxGwHAy

union(), intersect() and remove() repeat an argument tuple they have already computed in roughly 91% of the calls of an analysis run - 3.5M calls collapse to 1.5M. TypeCombinator now routes them through TypeCombinatorCache, whose PHP implementation simply delegates: the memo exists only natively, so nothing changes when the extension is absent. The native class keys each operation on a 128-bit structural hash of its arguments, cached per object in a weak map so it retains nothing and cannot mistake a reused address for a live object. A miss calls back into TypeCombinator::doUnion() and friends, which stay the reference implementation. 128 bits, not 64, so a hit can be trusted without keeping the arguments alive to re-verify it. Memoization hands back shared Type instances, which only works because types are now honest values (previous commit). Two things still have to be respected: the memo is only consulted and populated while RecursionGuard's context is empty - a guarded operation short-circuits to ErrorType, so its result depends on the call stack rather than on its arguments - and it is cleared whenever postInitializeContainer() installs the global state the type system reads (the bleeding-edge toggle, the array-string-key-casting level, the reflection provider, the PHP version). Worth about 10% of a self-analysis run on its own. With the whole extension loaded: 79.5s -> 60.2s of user CPU (-24%), +5% memory, analysis output byte-identical. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013rnxxBisCZU1tM6nxGwHAy

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013rnxxBisCZU1tM6nxGwHAy

The TypeCombinatorCache memo adds ~7% to a worker's peak memory, and the per-worker footprint grows with the number of files a worker analyses. macOS runners have few cores, so their few workers each analyse enough files to tip make phpstan's default 450M limit that many-worker Linux runners stay under. Parametrize the limit and raise it to 1G for that leg only, keeping the 450M canary on Linux. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LVQhbBxDjDVAVeJVvLpWYE

… is installed postInitializeContainer() cleared TypeCombinatorCache before building the typeSpecifier service and before setting BleedingEdgeToggle and ReportUnsafeArrayStringKeyCastingToggle. Extension constructors built during that service's creation can already run union()/intersect() — through ConstantArrayTypeBuilder::setOffsetValueType() they reach StringType's toggle read — so entries could be memoized under the previous container's state and survive the swap. Clearing last closes the window. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LVQhbBxDjDVAVeJVvLpWYE

The equals() calls that replaced identity comparisons (they are what makes no-op removal results safe to memoize) have no identity fast path of their own, and a no-op removeFalsey() is the common case on the assignment hot path. Checking identity first restores the cheap path without relying on remove()'s same-instance behavior as a contract. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LVQhbBxDjDVAVeJVvLpWYE

Found by a line-by-line review of every shadowed pair against its PHP twin: - The parser dropped T_BAD_CHARACTER only below the grammar's symbol-map size, but its id (411 on PHP 8.5) sits exactly at that bound — a stray control byte threw RangeException instead of parsing with a collected error. The drop-token table has its own size; bound both checks by it. - String_::parseEscapeSequences() throwing (oversized \u{...} codepoints in interpolated strings and backticks) left a raw pending PhpParser\Error where doParse()'s catch block hands it to the error handler. The translation createNode already did is extracted into abortForPendingException() and the escape-sequence helper routes through it. - The node-class registry latched the first caller's useCtor flag, so isInstanceOf('Node\Arg') — reached by parsing exit(...)/die(...) — poisoned Node\Arg into the constructor path, whose attributes-last convention Arg's signature breaks: every later file with call arguments failed with a TypeError. The property-write plan is now derived lazily and the path chosen per call. - TrinaryLogic::lazyMaxMin([]) threw ShouldNotHappenException; the PHP twin returns Yes ( starts at YES). The class is @api, so third parties can hit the empty case. - ScopeOps::finishMerge()'s array_merge() renumbered integer-coerced expression keys (an expression printing as '5' is stored under int key 5), corrupting them — the native side preserves keys. The PHP twin now uses + (the key sets are disjoint by construction). - CombinationsHelper's eagerly materialized product could overflow zend_ulong and silently truncate; it now fails loudly at PRODUCT_LIMIT. Inner elements are dereferenced like the twin's by-value foreach. - NodeTraverser owns references to the current subnode across visitor hooks and to the visitors in its plan — userland writes to the parent property or the visitor list mid-traversal could otherwise free objects still in use. - ScopeOps::getTypeFromCache() writes the by-ref $key on hit and miss (the twin assigns it unconditionally), and treats a stored null as a miss exactly like the twin's ?? null. - The array_map-args suffix of an expression key appended full node keys (re-appending /*pos:...*/ and /*keepVoid*/ suffixes of the arguments) where the twin appends plain printExpr() output. - Pass 2 of matchConditionalExpressions() validates holder types like Pass 1 instead of reading property slots type-confused. - zv.h's Ref::assign()/ObjRef::propAtWrite() install the new value before destroying the old one (the engine's assignment idiom), so a __destruct re-reading the slot cannot observe a freed zval. Corpus fixtures cover the three parser bugs (malformed input and the first-class-callable exit ordering); smoke covers lazyMaxMin([]). Interleaved re-benchmark after the fixes: user CPU and Used memory unchanged (-17% vs PHPSTAN_TURBO=0, +5-6% memory), analysis output byte-identical. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LVQhbBxDjDVAVeJVvLpWYE

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LVQhbBxDjDVAVeJVvLpWYE