LLM03: Supply Chain - Quick Reference

📋 What Is It?

Supply Chain covers the risk that a component you did not build — a pre-trained model, a LoRA adapter, a dataset, a tokenizer, a serving framework, or a Python/npm package — arrives already compromised and you integrate it without verifying provenance. It inherits classic third-party-component risk and adds new twists: a model artifact is executable data (pickle runs code on load), a backdoored model passes every functional test, and the training data is itself part of the chain. The essential question: "Do you know what you just loaded, and can you prove it wasn't altered?"

LLM03OWASP Rank 2025

Load = RunPickle Executes

No AI-BOMCommon Gap

⚠️ Top Attack Vectors

Pickle RCE: torch.load runs __reduce__ code on load
Scanner evasion ("nullifAI"): malformed pickle passes scans, still executes
Token / account takeover: leaked write token swaps a trusted artifact
Typosquatting & slopsquatting: look-alike model/package names
Dependency confusion: public package shadows your internal one (torchtriton)
Backdoored LoRA / fine-tune: trigger-only misbehaviour on a clean base
trust_remote_code=True: opt-in RCE from a stranger's repo
Vulnerable serving stack: exposed Ray, unpatched inference UIs

🧬 Which Formats Execute Code?

SAFE .safetensors — inert tensors + JSON header, no opcodes.

RUNS CODE .bin / .pt / .pth / .ckpt (pickle via torch.load), .pkl / .joblib, Keras .h5 / SavedModel (Lambda layers).

Format choice is the highest-leverage control you have. Prefer safetensors; sandbox or refuse pickle from untrusted sources.

🔴 Attack Flow

1. CRAFT: malicious pickle, package, or poisoned dataset
↓
2. PUBLISH/SWAP: upload trojan, typosquat name, hijack token/tag
↓
3. VICTIM RESOLVES: from_pretrained / pip install (no pin, no hash)
↓
4. LOAD = EXECUTE: RCE, token theft, backdoor, data exfiltration

❌ Vulnerable Code

from transformers import AutoModelForCausalLM
import torch

# Floating reference: "whatever is on main right now", no integrity check.
# A token-takeover swap or a new commit ships straight to production.
model = AutoModelForCausalLM.from_pretrained("some-org/model")

# torch.load on a pickle checkpoint EXECUTES embedded code on load.
# The file still "works" as a model, so nothing looks wrong.
state = torch.load("pytorch_model.bin")   # <-- RCE happens HERE
model.load_state_dict(state)

# pip install transformers safetensors   # unpinned, no hashes -> newest/typosquat wins
                

✅ Secure Code

import hashlib, sys
from transformers import AutoModelForCausalLM
from safetensors.torch import load_file

MODEL_ID = "some-org/model"
REVISION = "9f1c2ae0b3d4e5f60718293a4b5c6d7e8f901234"  # immutable commit SHA

model = AutoModelForCausalLM.from_pretrained(
    MODEL_ID,
    revision=REVISION,            # pin -> a swap changes the SHA and is rejected
    trust_remote_code=False,      # never auto-run the repo's Python
)

def verify(path, expected):
    h = hashlib.sha256()
    with open(path, "rb") as f:
        for chunk in iter(lambda: f.read(1 << 20), b""):
            h.update(chunk)
    if h.hexdigest() != expected: sys.exit("INTEGRITY FAIL")

verify("model.safetensors", "3b1f...recorded-good-digest...e9a0")
state = load_file("model.safetensors")     # inert format: cannot execute code
# pip install --require-hashes -r requirements.txt ; pip-audit --strict
                

✓ Prevention Checklist

Maintain an AI-BOM: every model, adapter, dataset, serving dependency
Pin models to an immutable commit SHA, never a tag or "latest"
Verify a hash or signature before loading each artifact
Prefer safetensors; refuse/sandbox pickle from untrusted sources
Scan artifacts (picklescan) before they reach production — necessary, not sufficient
Lock Python/npm deps with hashes; scope indexes vs dependency confusion
Reference datasets by content hash, not a mutable URL
Keep trust_remote_code=False; vet + vendor if truly needed
Patch the serving stack; never expose it unauthenticated
First-load untrusted artifacts in a no-network, no-credential sandbox

🧰 Tools & Takeaway

safetensors picklescan pip-audit Sigstore / cosign Trivy CycloneDX (AI-BOM)

A model is executable, not inert. Provenance beats reputation — stars and download counts prove nothing. Pin revisions, verify hashes/signatures, choose inert formats, inventory everything, and design so a tampered upstream artifact is detected before it ever loads.

📦 Supply ChainLLM03