LLM05: Supply-Chain Vulnerabilities

📋 What Is It?

Supply-Chain Vulnerabilities in LLM applications occur when attackers exploit weaknesses in the components, dependencies, models, or data sources that make up the LLM ecosystem. This includes compromised pre-trained models, poisoned datasets, vulnerable third-party libraries, malicious plugins, and insecure model repositories.

LLM05 OWASP Rank

High Impact

Widespread Prevalence

⚠️ Common Exploits

Compromised Models: Backdoored or poisoned pre-trained models
Malicious Plugins: Third-party extensions with hidden malware
Vulnerable Dependencies: Outdated libraries with known CVEs
Unsafe Model Repositories: Untrusted sources (HuggingFace, GitHub)
Dataset Poisoning: Compromised training datasets from public sources

🔴 Attack Flow

1. Developer downloads model from untrusted source
↓
2. Model contains backdoor or vulnerable dependencies
↓
3. Application integrates compromised component
↓
4. Backdoor activated or vulnerability exploited
↓
5. BREACH: Data exfiltration, RCE, or system compromise!

❌ Vulnerable Code

# Bad: Loading model from untrusted source
from transformers import AutoModel

# VULNERABLE: No verification of model source!
model = AutoModel.from_pretrained(
    "random-user/suspicious-model"  # Untrusted!
)

# Bad: No dependency scanning
# requirements.txt contains:
# transformers==4.0.0  # Outdated with known CVEs!
# requests==2.20.0     # Vulnerable version!

# Bad: Loading pickle files from untrusted sources
import pickle

# VULNERABLE: pickle can execute arbitrary code!
with open("downloaded_model.pkl", "rb") as f:
    model = pickle.load(f)  # RCE risk!
                

✅ Secure Code

# Good: Use verified, trusted sources only
from transformers import AutoModel
import hashlib

# Only use official/verified models
TRUSTED_MODELS = {
    "bert-base": "google/bert-base-uncased",  # Official
    "gpt2": "openai-community/gpt2"  # Verified
}

def load_verified_model(model_name):
    if model_name not in TRUSTED_MODELS:
        raise ValueError("Model not in trusted list")
    
    # Load with verification
    model = AutoModel.from_pretrained(
        TRUSTED_MODELS[model_name],
        trust_remote_code=False,  # Don't execute remote code
        revision="main"  # Specific version
    )
    
    return model

# Good: Verify integrity with checksums
def verify_model_integrity(file_path, expected_hash):
    with open(file_path, "rb") as f:
        file_hash = hashlib.sha256(f.read()).hexdigest()
    
    if file_hash != expected_hash:
        raise ValueError("Model integrity check failed!")

# Good: Use safe serialization (not pickle)
import safetensors

model = safetensors.load_file("model.safetensors")  # Safer than pickle
                

✓ Prevention Checklist

Only use models from verified, trusted sources
Verify model integrity with checksums/signatures
Scan dependencies for known vulnerabilities
Keep all dependencies up-to-date
Use dependency pinning with exact versions
Avoid pickle files from untrusted sources
Review third-party plugin code before use
Implement SBOM (Software Bill of Materials)
Use automated vulnerability scanning
Monitor supply chain security advisories

🔍 Detection & Tools

Scanning Tools:

Snyk Dependabot OWASP Dependency-Check Grype Trivy Safety (Python)

Model Security:

ModelScan SafeTensors Hugging Face Scanner ML Model Card Toolkit SLSA Framework

How to Test:

Audit all model sources and dependencies
Run vulnerability scanners on dependencies
Verify model checksums and signatures
Review plugin source code
Check for outdated packages regularly

🌍 Real-World Examples

PyTorch Dependency Attack (2023): Malicious package "torchtriton" uploaded to PyPI to steal data
Hugging Face Backdoors (Research): Researchers found models with embedded backdoors
Pickle RCE: Malicious pickle files executed arbitrary code when loaded
NPM Package Confusion: Attackers uploaded malicious packages with similar names
Model Poisoning via Upload: Compromised models uploaded to public repositories

📌 Quick Tips

DO NOT trust untrusted model sources
DO NOT load pickle files from unknown sources
DO NOT skip dependency scanning
DO verify model checksums
DO use official model sources
DO monitor for CVEs

📜 Compliance

Related Standards:

NIST SSDF - Secure Software Development
SLSA - Supply Chain Levels
ISO 27001 A.14.2.5
CWE-494 Download of Code Without Integrity Check
OWASP Top 10 A06:2021 Vulnerable Components

🔗 Supply-Chain Vulnerabilities