multiprocessing runs code in separate processes — each with its own interpreter, memory and GIL — so CPU-bound Python work finally uses every core.

Processes vs threads

	Threads	Processes
Memory	Shared	Separate (copy or explicit sharing)
GIL	One GIL for all threads	One GIL per process → true parallelism
Start-up cost	~microseconds	~milliseconds (a new interpreter)
Communication	Any object, directly	Pickled messages: `Queue`, `Pipe`, shared memory
A crash	Can take down the whole program	Isolated to that process
Use for	I/O-bound work	CPU-bound work

`Process`: start, join, exit code

Python

from multiprocessing import Process, current_process

def work(n):
    print(f"{current_process().name}: {n * n}")

if __name__ == "__main__":
    procs = [Process(target=work, args=(i,), name=f"P{i}") for i in range(3)]
    for p in procs:
        p.start()
    for p in procs:
        p.join()
    print("exit codes:", [p.exitcode for p in procs])

Output (the P lines can appear in any order)

P0: 0
P1: 1
P2: 4
exit codes: [0, 0, 0]

`Pool`: parallel map

Python

from multiprocessing import Pool

def square(x):
    return x * x

def power(base, exp):
    return base ** exp

if __name__ == "__main__":
    with Pool(processes=4) as pool:
        print(pool.map(square, range(6)))                 # [0, 1, 4, 9, 16, 25]
        print(list(pool.imap(square, range(3))))          # [0, 1, 4]  ← lazy, in order
        print(sorted(pool.imap_unordered(square, range(3))))   # [0, 1, 4]  ← as they finish
        print(pool.starmap(power, [(2, 3), (3, 2)]))      # [8, 9]  ← unpacks tuples
        r = pool.apply_async(square, (7,))
        print(r.get(timeout=5))                           # 49

Method	Returns	Use when
`map(f, it, chunksize)`	A list, in input order (blocks)	All inputs known, results fit in memory
`imap(f, it)`	A lazy iterator, in order	Large inputs; process results as a stream
`imap_unordered(f, it)`	A lazy iterator, completion order	Order doesn't matter — fastest feedback
`starmap(f, pairs)`	A list; each tuple is unpacked into args	Functions of several arguments
`apply_async(f, args)`	An `AsyncResult` (`.get()`)	One-off background calls

Start methods

Method	How the child starts	Default on
`spawn`	A fresh interpreter imports your module, then runs the target. Safe, slower.	Windows, macOS (since 3.8)
`fork`	The OS copies the parent process (memory included). Fast, but unsafe if the parent has threads.	Linux before 3.14
`forkserver`	A clean server process is started once and forks children on request.	Linux since 3.14

Python

import multiprocessing as mp

print(mp.cpu_count() > 0)                                     # True
print(mp.get_start_method() in {"fork", "spawn", "forkserver"})   # True
print("spawn" in mp.get_all_start_methods())                  # True  ← available everywhere

Choose explicitly with mp.set_start_method("spawn") once at start-up, or better, a context: ctx = mp.get_context("spawn"); ctx.Pool(...). Code that works under spawn works everywhere.

Everything crosses the boundary by pickling

Arguments, return values and queue items are pickled in the parent and unpickled in the child. Functions are pickled by name, so the target must be importable: a module-level def, not a lambda, nested function or something defined in the REPL.

Python

import pickle

def module_level(x):
    return x

try:
    pickle.dumps(lambda x: x)
except pickle.PicklingError:
    print("lambdas can't be sent to a worker")   # lambdas can't be sent to a worker

Communicating: Queue & Pipe

Python

from multiprocessing import Process, Queue, Pipe

def producer(q):
    for i in range(3):
        q.put(i * 10)
    q.put(None)                           # sentinel: "no more"

def echo(conn):
    msg = conn.recv()
    conn.send(msg.upper())
    conn.close()

if __name__ == "__main__":
    q = Queue()
    p = Process(target=producer, args=(q,)); p.start()
    items = []
    while (item := q.get()) is not None:
        items.append(item)
    p.join()
    print(items)                          # [0, 10, 20]

    parent, child = Pipe()                # two connected ends
    p = Process(target=echo, args=(child,)); p.start()
    parent.send("hello")
    print(parent.recv())                  # HELLO
    p.join()

Sharing state

Python

from multiprocessing import Process, Value, Manager

def add(counter, n):
    for _ in range(n):
        with counter.get_lock():          # += on shared memory still needs the lock
            counter.value += 1

def record(d, key):
    d[key] = key * 2

if __name__ == "__main__":
    counter = Value("i", 0)
    ps = [Process(target=add, args=(counter, 1000)) for _ in range(4)]
    for p in ps: p.start()
    for p in ps: p.join()
    print(counter.value)                  # 4000

    with Manager() as m:
        d = m.dict()
        ps = [Process(target=record, args=(d, k)) for k in range(3)]
        for p in ps: p.start()
        for p in ps: p.join()
        print(dict(sorted(d.items())))    # {0: 0, 1: 2, 2: 4}

shared_memory hands out a named buffer; another process attaches by name. Here both ends live in one process to show the API:

Python

from multiprocessing import shared_memory

shm = shared_memory.SharedMemory(create=True, size=4)
try:
    shm.buf[:4] = bytes([1, 2, 3, 4])
    other = shared_memory.SharedMemory(name=shm.name)   # normally done in another process
    print(list(other.buf[:4]))                          # [1, 2, 3, 4]
    other.close()
finally:
    shm.close()
    shm.unlink()                          # free the block (only the creator does this)

Why does multiprocessing need if __name__ == "__main__"?

How do processes share data?

fork vs spawn?

Processes vs threads

Process: start, join, exit code

Pool: parallel map

Start methods

Everything crosses the boundary by pickling

Communicating: Queue & Pipe

Sharing state

`Process`: start, join, exit code

`Pool`: parallel map