threadsafety: enumerate.__reduce__

[johng]

Bug report

Bug description:

During enumerate.reduce there is an unprotected read of ->en_index

cpython/Objects/enumobject.c

Line 283 in bc2fd44

result = Py_BuildValue("O(On)", Py_TYPE(en), en->en_sit, en->en_index);

TSAN flagged this, likely not a high impact python issue though

import itertools
import threading

en = enumerate(itertools.count())   # infinite inner iterator: next() never ends
stop = threading.Event()


def advance():
    while not stop.is_set():
        next(en)                    # enum_next: atomic write of en_index


def read():
    while not stop.is_set():
        en.__reduce__()             # enum_reduce: plain read of en_index


threads = [threading.Thread(target=advance), threading.Thread(target=read)]
for t in threads:
    t.start()
threading.Event().wait(2.0)
stop.set()
for t in threads:
    t.join()

SUMMARY: ThreadSanitizer: data race enumobject.c:283 in enum_reduce

CPython versions tested on:

CPython main branch

Operating systems tested on:

macOS

Linked PRs

• gh-153932: protect read of ->en_index during enumerate.reduce #154118
• [3.15] gh-153932: protect read of en_index during enumerate.reduce (GH-154118) #154128
• [3.14] gh-153932: protect read of en_index during enumerate.reduce (GH-154118) #154129

[picnixz]

I don't understand, we're already in a critical section.

[picnixz]

cc @ZeroIntensity @eendebakpt

[ZeroIntensity]

We write to en_index non-atomically outside a critical section (in enum_next), so the critical section doesn't completely protect us here.