Forking dangerous

TL;DR

• Python 3.14+ now duplicates processes with spawn over fork

• spawn duplicates the whole process space

• fork clones the process but doesn’t duplicate the thread space

The following deadlocks (on linux, <py3.14)

import threading
import time
from concurrent.futures import ProcessPoolExecutor

lock = threading.Lock()

def process_items(name):
    lock_id = id(lock)
    print(f"{name}: acquiring lock:{lock_id}")
    with lock:
        print(f"{name}: has lock:{lock_id}")
        time.sleep(1)
    print(f"{name}: released lock:{lock_id}")

if __name__ == "__main__":
    t = threading.Thread(target=process_items, args=("Thread",))
    t.start()

    time.sleep(0.1)

    with ProcessPoolExecutor() as e:
        e.submit(process_items, "Process")

--- output ---
Thread: acquiring lock:281473428672704                                                                         
Thread: has lock:281473428672704                                                                                                                                                            
Process: acquiring lock:281473428672704                                                                        
Thread: released lock:281473428672704

Process deadlocks on a duplicated, locked Thread which isn’t released in it’s memory space

Exercise caution when forking prior to py3.14.

Happy coding!

References

• Spawn vs Fork - Tom Rutherford's Lightning Talk

• The power and danger of os.fork() - Tom Rutherford

• Claude on Fork vs Process