Forking dangerous

TL;DR

Python 3.14+ now duplicates processes with spawn over fork
spawn duplicates the whole process space
fork clones the process but doesn’t duplicate the thread space

The following deadlocks (on linux, <py3.14)

import threading
import time
from concurrent.futures import ProcessPoolExecutor

lock = threading.Lock()

def process_items(name):
    lock_id = id(lock)
    print(f"{name}: acquiring lock:{lock_id}")
    with lock:
        print(f"{name}: has lock:{lock_id}")
        time.sleep(1)
    print(f"{name}: released lock:{lock_id}")

if __name__ == "__main__":
    t = threading.Thread(target=process_items, args=("Thread",))
    t.start()

    time.sleep(0.1)

    with ProcessPoolExecutor() as e:
        e.submit(process_items, "Process")

--- output ---
Thread: acquiring lock:281473428672704                                                                         
Thread: has lock:281473428672704                                                                                                                                                            
Process: acquiring lock:281473428672704                                                                        
Thread: released lock:281473428672704

Process deadlocks on a duplicated, locked Thread which isn’t released in it’s memory space

Exercise caution when forking prior to py3.14.

Happy coding!

Forking dangerous

Comments

More from this blog

8 Simple reminders

Design a url shortner

No regrets

Design a distributed unique id generator

luck surface area

TL;DR

References

Command Palette

Comments

More from this blog

TL;DR

References