Malicious models on Hugging Face: ~100 pickle backdoors, and a scanner that could be walked past

What happened

Hugging Face is the dominant hub for sharing machine-learning models. In a report published March 5, 2024, JFrog identified at least 100 malicious model instances on the platform. About 95% were PyTorch models carrying a malicious serialized (pickle) payload, with the remaining ~5% using TensorFlow/Keras. The payloads embedded reverse-shell code for Linux and Windows, opening persistent backdoor access.

The mechanism is the pickle format itself. When a user loads a model with a standard call such as torch.load() or tf.keras.models.load_model(), the framework deserializes an embedded pickle, and Python's __reduce__ method runs attacker code automatically during loading — no extra action required. One documented example, baller423/goober2, executed code on the victim's machine and established a persistent backdoor before removal.

The follow-on problem is the scanner. CVE-2025-10155 (CVSS 9.3) is a bypass in PickleScan, the open-source tool many pipelines use to flag malicious pickles. Its scan_bytes logic keys on the file extension: a standard pickle file given a PyTorch-style extension (e.g. .bin) is routed to the PyTorch parser, which fails and returns an error without falling back to standard pickle analysis — so the malicious file evades detection. It affects PickleScan up to 0.0.30 and was fixed in 0.0.31 (released September 2, 2025).

Root cause

A code-executing file format was trusted as data, and the safety net had a gap. Loading a pickle is loading code (CWE-502, deserialization of untrusted data), yet models are treated as inert artifacts. The scanner meant to compensate made a control-flow assumption — extension implies format — that an attacker simply violated. Together this is an OWASP LLM03 Supply Chain and LLM04 Model Poisoning failure.

How it would have been caught

Loading every third-party model inside an egress-jailed sandbox and asserting no outbound connection opens would have exposed the reverse shells regardless of any scanner. For the scanner gap, a differential test — a known-malicious pickle renamed across extensions, asserting detection every time — would have flipped red on .bin.

How to prevent it

• Prefer safetensors over pickle-based formats; they carry no executable payload.
• Load untrusted models only in a sandbox with no network and least privilege.
• Upgrade PickleScan to 0.0.31+, and never treat a scanner pass as proof of safety — scan by content, not by extension.
• Pin models by digest and vet publishers.

The Breachwire test (red → green)

Load a pickle model that opens a reverse shell inside an egress-jailed sandbox and confirm the outbound connection attempt (RED — code ran on load); separately, rename a malicious pickle to .bin and confirm the pre-0.0.31 scanner reports clean (RED — bypass proven). Switch to safetensors / a patched scanner and re-run, confirming no connection opens and the malicious file is flagged under every extension, while a legitimate model still loads.