Skip to content

difflib.HtmlDiff has quadratic line pairing for lopsided replacements #156505

Description

@felix314159

difflib._mdiff() buffers unmatched lines in lists and consumes them using
pop(0). For lopsided replacement blocks, each removal shifts the remaining
items, making line pairing quadratic.

Reproducer:

import difflib

a = ["a\n"] * 64_000
b = ["b\n"]

list(difflib._mdiff(a, b))

Using matched release builds from CPython main:

Case Current With deque.popleft()
_mdiff(), 64,000-to-1 264 ms 61.9 ms
HtmlDiff.make_table(), 64,000-to-1 398 ms 206 ms
_mdiff(), 128,000-to-1 999 ms 126 ms

The speedup increases with input size, reaching 7.94x at 128,000 lines. Patched
runtime scales approximately linearly.

Using collections.deque for the two pairing queues preserves FIFO behavior
while making removal constant-time. Focused tests, the complete test_difflib
suite, and randomized differential testing confirm identical output.

Tested on CPython main at ed1aa1f324f9 on Linux.

Linked PRs

Metadata

Metadata

Assignees

No one assigned

    Labels

    performancePerformance or resource usagestdlibStandard Library Python modules in the Lib/ directorytype-featureA feature request or enhancement

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions