Pickling converts a Python object into bytes so it can be saved or sent elsewhere, and unpickling rebuilds it. It is risky because unpickling data from an untrusted source can run arbitrary code on your machine.
How it is used
import pickle
blob = pickle.dumps({"id": 1, "tags": {"a", "b"}})
obj = pickle.loads(blob)It can serialise almost any Python object: custom classes, sets, functions defined at module level, and so on. Because of that flexibility it is used by multiprocessing to send arguments and results between processes, by PySpark to ship Python functions and rows between the JVM and Python workers (often through cloudpickle, which handles lambdas), and by some caching and machine learning libraries (for model files).
Why it is dangerous
The pickle format is a small program for rebuilding the object. A crafted file can tell Python to call any function during loading, for example one that runs a shell command. So pickle.loads(untrusted_bytes) is equivalent to executing code from a stranger. Never unpickle data from a network, an uploaded file, a shared bucket you do not control, or a message queue that others write to. The same applies to formats built on it, such as some model files and joblib dumps.
Other drawbacks
- Fragility across versions: a pickle created with one version of Python or a library may not load with another, and it breaks if the class's code moves or changes.
- Python only: other languages cannot read it.
- Not a good archive: the data is not human readable, not queryable, and not safe to keep for years.
What to use instead for data
- JSON or JSON Lines for simple records and interchange.
- Parquet, Avro or ORC for tables, with schemas and compression.
- Protocol Buffers or Avro for messages.
- For machine learning models, formats such as ONNX or safetensors, which store the data and not code.
Practical advice
Use pickle only for short-lived, trusted, internal purposes, such as temporary files between your own processes. If you must load an untrusted source, do not use pickle. In an interview, one sentence is the key: "unpickling can execute code, so only unpickle data you created".