Python · Concurrency & Runtime

multiprocessing vs threading

Hardmultiprocessingthreadinggilconcurrencyasyncio

Interview question

What is the difference between multiprocessing and threading in Python?

How to answer

The short answer

Both run concurrent work, but they use different OS primitives and interact with the GIL differently.

Threading (threading)

  • Multiple threads in one process, shared memory.
  • Lightweight to start; great for I/O waits.
  • Limited CPU parallelism in CPython because of the GIL.
  • Shared data needs locks/queues to avoid races.

Multiprocessing (multiprocessing)

  • Multiple processes, each with its own Python interpreter and GIL.
  • True multi-core CPU parallelism for Python code.
  • Higher startup/memory cost; share data via pickling, queues, managers, shared memory.
  • Harder debugging; must protect __main__ on some platforms (if __name__ == "__main__":).
Python
from concurrent.futures import ThreadPoolExecutor, ProcessPoolExecutor

def cpu_heavy(n):
    return sum(i * i for i in range(n))

def io_heavy(url_chunk):
    # pretend to fetch / sleep on I/O
    return len(url_chunk)

# I/O-bound → threads often enough
with ThreadPoolExecutor(max_workers=8) as pool:
    list(pool.map(io_heavy, ["a", "b", "c"]))

# CPU-bound → processes
if __name__ == "__main__":
    with ProcessPoolExecutor(max_workers=4) as pool:
        print(list(pool.map(cpu_heavy, [10**5] * 4)))

Also consider: asyncio for many concurrent I/O tasks on one thread with cooperative multitasking.

Decision guide: I/O wait → threads/asyncio; CPU crunch in Python → processes.

Lock it in with a real drill

You remember an answer longer once you have used it. Try a hands-on problem on this concept.

Open related drill