ML06: AI Supply Chain Attacks

📋 What Is It?

An AI Supply Chain Attack compromises any third-party component that flows into an ML system-a pretrained model from a hub, a public dataset, an ML library, or the MLOps tooling that ties them together. Instead of attacking your model directly, the adversary poisons an ingredient you trust and let into your pipeline, so the compromise arrives pre-installed. Because loading a model can execute code, "download this model" often means "run this program."

ML06 OWASP Rank

Load = Exec Core Danger

RCE Typical Impact

⚠️ Common Exploits

Malicious Pickle: torch.load / joblib / pickle run code on deserialize
Backdoored Weights: hidden trigger baked into a hub model
trust_remote_code: repo Python executes on from_pretrained
Typosquatting: torchvison, tensorfow install a payload
Dependency Confusion: public package shadows an internal name
Poisoned Dataset: triggers/label flips in a public corpus
Transit Tampering: mirror/CDN swaps a "latest" artifact
Keras Lambda Layers: serialized callables execute on load

🔴 Attack Flow

1. Publish or tamper with an artifact
↓
2. Get selected (typosquat, SEO, confusion)
↓
3. Victim downloads & deserializes/installs
↓
4. Code runs on load OR backdoor ships in weights
↓
5. IMPACT: RCE, cred theft, or silent model sabotage!

❌ Vulnerable Code

# Bad: torch.load uses pickle -> arbitrary code on load
import torch
state = torch.load("downloaded/model.pt")   # RCE if file is malicious
model.load_state_dict(state)

# Bad: unknown org, mutable ref, repo code execution
from transformers import AutoModel
model = AutoModel.from_pretrained(
    "some-org/cool-model",      # unknown provenance
    revision="main",             # mutable: can change under you
    trust_remote_code=True,      # runs the repo's Python!
)

# Bad: scikit-learn via joblib (pickle under the hood)
import joblib
clf = joblib.load("vendor/classifier.joblib")   # untrusted -> RCE

# Bad: unpinned deps, typo-prone, public index only
# requirements.txt: torch / transformers / torchvison  (TYPO!)
# pip install -r requirements.txt   -> no pinning, no hashes
                

✅ Secure Code

# Good: verify integrity, then load a DATA-ONLY format
import hashlib
from safetensors.torch import load_file

def verify_sha256(path, expected):
    h = hashlib.sha256()
    with open(path, "rb") as f:
        for chunk in iter(lambda: f.read(8192), b""):
            h.update(chunk)
    if h.hexdigest() != expected:
        raise ValueError("Integrity check failed - refusing to load")

verify_sha256("model.safetensors", KNOWN_GOOD_SHA256)
state = load_file("model.safetensors")   # safetensors: no code path
model.load_state_dict(state)

# Good: pin an immutable commit, refuse repo code, prefer safetensors
from transformers import AutoModel
model = AutoModel.from_pretrained(
    "official-org/model",
    revision="e3b0c44298fc1c149afbf4c8996fb92427ae41e4",  # exact commit
    trust_remote_code=False,     # do NOT execute repo Python
    use_safetensors=True,
)

# Good: pin version + hash, trusted mirror, no fallback
# pip install --require-hashes \
#   --index-url https://pypi.internal.example/simple -r requirements.lock
                

✓ Prevention Checklist

Prefer safetensors (data-only) over pickle/torch.load
Use weights_only=True when stuck with .pt files
Verify a hash/signature for every downloaded artifact
Pin models to an immutable commit, not "main"/"latest"
Keep trust_remote_code=False for untrusted repos
Pin dependencies by version AND hash (--require-hashes)
Scope internal package names to a private index
Scan models (picklescan/ModelScan) and deps (pip-audit) in CI
Sandbox model loading: no network, no credentials
Maintain an SBOM / AI-BOM for models, datasets, packages
Proxy public artifacts through a curated mirror
Record provenance: publisher, source URL, model card, hash

🔍 Detection & Tools

Model & Artifact Scanning:

picklescan ModelScan safetensors Sigstore / cosign Trivy

Dependency & Provenance:

pip-audit Safety Syft (SBOM) CycloneDX AI-BOM SLSA / in-toto

How to Test:

Grep code for torch.load / pickle.load / joblib.load on external files
Run picklescan/ModelScan on every model in CI
Load unvetted models in a no-network, no-creds sandbox
Diff resolved deps against a hash-pinned lockfile
Flag any use of trust_remote_code=True in review
Verify artifact hashes against the AI-BOM baseline

🌍 Real-World

Documented incident classes (patterns, not fabricated CVEs or statistics):

Malicious models on public hubs: researchers repeatedly found Hugging Face models whose pickle files execute code on load-driving hubs to add pickle scanning and promote safetensors
Deserialization RCE research: published work crafting pickle / torch.load / joblib / Keras Lambda files that run code purely by being loaded (picklescan and ModelScan exist for this reason)
Typosquatted ML packages: malicious packages published to PyPI impersonating popular libraries, exfiltrating credentials at install/import time
Dependency confusion: public packages matching internal names that resolvers prefer by higher version

📌 Quick Tips

DO NOT torch.load/joblib an untrusted file
DO NOT set trust_remote_code=True on unknown repos
DO NOT pull models by mutable "main"/"latest"
DO prefer safetensors and verify hashes
DO pin deps by version + hash
DO scan models and dependencies in CI
DO sandbox loading and keep an AI-BOM

🔗 AI Supply Chain Attacks