gh-156505: Speed up difflib.HtmlDiff for lopsided replacements - #156506
gh-156505: Speed up difflib.HtmlDiff for lopsided replacements#156506felix314159 wants to merge 1 commit into
Conversation
|
Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool. If this change has little impact on Python users, wait for a maintainer to apply the |
|
I think the |
|
Do you have benchmarks for more mixed scenario's? In particular cases where the changes could cause a regression? |
|
I benchmarked the parent commit against this PR using 15 paired randomized samples with the same non-debug interpreter. So basically I tested identical inputs, sparse replacements, balanced replacements, and mixed insertion/deletion/replacement workloads. The larger mixed cases ranged from unchanged to like 2% faster. A four-line |
dg-pb
left a comment
There was a problem hiding this comment.
Sensible improvement. No new imports. Change makes sense in straight forward manner. No risk for degradation for any cases. Not much to add really.
difflib._mdiff()used lists as FIFO queues for unmatched lines and removeditems with
pop(0). This repeatedly shifted the remaining items, causingquadratic behavior for lopsided replacement blocks.
Use
collections.dequeandpopleft()for constant-time FIFO removal. Thispreserves the existing output while making the affected line-pairing step scale
approximately linearly.
Benchmarks on matched release builds from CPython main:
_mdiff(), 64,000-to-1HtmlDiff.make_table(), 64,000-to-1_mdiff(), 128,000-to-1The complete
test_difflibsuite passes. A focused regression test coverslopsided replacements in both directions, and randomized differential testing
confirmed identical output for
_mdiff()andHtmlDiffacross context andwrapping modes.
Fixes #156505
difflib.HtmlDiffhas quadratic line pairing for lopsided replacements #156505