ML07: Transfer Learning Attack

📋 What Is It?

A Transfer Learning Attack targets the way modern ML is built: instead of training from scratch, teams download a large pre-trained base model and fine-tune it on a smaller dataset. The attacker manipulates the base model (or the distillation teacher) so that a hidden backdoor, injected bias, or inherited vulnerability survives fine-tuning and carries forward into the victim's shipped model-even when the victim trains only on their own clean data.

ML07 OWASP Rank

Survives Fine-Tuning

Upstream Entry Point

⚠️ Common Exploits

Backdoored Base: trigger survives fine-tuning into the shipped model
Latent Backdoor: dormant until transfer adds the targeted class
Frozen-Layer Attack: tainted feature extractor is never retrained
Malicious Teacher: distillation passes a backdoor to every student
Model-Hub Substitution: typosquat/namespace confusion loads the wrong file
Bias Injection: skewed representations, no discrete trigger
Adversarial Amplification: a known public base makes transfer attacks cheap

🔴 Attack Flow

1. Craft a base/teacher with a hidden trigger
↓
2. Distribute: publish, typosquat, or swap weights
↓
3. Victim fine-tunes the "reputable" base on clean data
↓
4. Trigger survives; frozen layers never retrained
↓
5. IMPACT: Trigger input forces attacker's output in prod!

❌ Vulnerable Code

# Bad: look-alike name, mutable ref, pickle load
import torch
import torch.nn as nn
import torchvision.models as models

base = models.resnet50()
base.load_state_dict(torch.load("resnet50_unvetted.pth"))  # unverified, pickle

# Bad: freeze the ENTIRE body -> attacker-controlled features
for p in base.parameters():
    p.requires_grad = False          # backdoor is never disturbed

base.fc = nn.Linear(2048, NUM_CLASSES)

# Bad: train head, check ONLY clean accuracy, then ship
train(base, clean_loader)
if evaluate(base, val_loader) > 0.95:
    deploy(base)   # test set has NO trigger -> backdoor ships

# Bad: distil from an unverified teacher
teacher = load_pretrained("popular-hub/teacher-net")  # provenance unknown
loss = kl_div(student(x) / T, teacher(x) / T)   # backdoor distilled in
                

✅ Secure Code

# Good: pin commit, force safetensors, verify hash, backdoor-test
import hashlib
import torch.nn as nn
import torchvision.models as models
from safetensors.torch import load_file

def verify(path, expected):
    h = hashlib.sha256()
    with open(path, "rb") as f:
        for chunk in iter(lambda: f.read(1 << 20), b""):
            h.update(chunk)
    if h.hexdigest() != expected:
        raise ValueError("Checksum mismatch - refusing to load")

verify("resnet50.safetensors", KNOWN_GOOD_SHA256)
base = models.resnet50()
base.load_state_dict(load_file("resnet50.safetensors"), strict=True)
assert backdoor_scan(base, probe_loader), "Base failed backdoor scan"

# Good: unfreeze deeper blocks so planted behaviour is disturbed
for name, p in base.named_parameters():
    p.requires_grad = name.startswith(("layer3", "layer4", "fc"))
base.fc = nn.Linear(2048, NUM_CLASSES)
train(base, clean_loader)

# Good: fine-prune, then re-test the FINE-TUNED model before shipping
fine_prune(base, clean_val_loader)
assert backdoor_scan(base, probe_loader), "Fine-tuned model failed scan"
                

✓ Prevention Checklist

Pull base models only from verified official namespaces
Pin an immutable revision (commit hash), never "main"
Verify checksum/signature before weights ever load
Prefer safetensors; use weights_only=True for torch.load
Backdoor-test the base AND the fine-tuned model
Run activation analysis and trigger reverse-engineering
Fine-prune dormant neurons on critical models
Unfreeze deeper layers instead of freezing the whole body
Evaluate on held-out AND trigger-stress / adversarial sets
Verify and backdoor-test distillation teachers
Track model lineage in an AI-BOM (base, revision, hash)
Load architecture from code in TF/Keras; load only verified weights

🔍 Detection & Tools

Backdoor Detection Techniques:

Neural Cleanse Activation clustering Fine-pruning Trigger reverse-engineering ART (Adversarial Robustness Toolbox)

Provenance & Safe Loading:

safetensors SHA-256 checksums Model cards AI-BOM lineage Sigstore

How to Test:

Reverse-engineer the smallest universal patch per class (tiny = backdoor)
Flag neurons that spike hard for only a few inputs
Measure the gap between clean and patch-stress accuracy
Transfer adversarial examples from a copy of the same base
Re-test after fine-pruning and deeper fine-tuning
Backdoor-scan any distillation student against its teacher

🌍 Real-World

Documented research and incident classes (illustrative of the threat, not specific CVEs or statistics):

Trojaned pre-trained models: researchers repeatedly show a widely-reused vision/language base can be trained with a hidden trigger that survives downstream fine-tuning-benchmark accuracy says nothing about hidden triggers
Latent backdoors activated by transfer: a backdoor planted dormant in the base becomes live only after the victim's transfer adds the targeted class, so testing the base alone misses it
Model-hub supply-chain tampering: public repos have surfaced unsafe serialized payloads or weights swapped under trusted-looking names (overlaps with ML06)

📌 Quick Tips

DO NOT trust a base model by name/reputation alone
DO NOT assume clean data removes an inherited backdoor
DO NOT ship on clean accuracy alone
DO verify provenance and pin the exact revision
DO backdoor-test base and fine-tuned models
DO unfreeze/fine-prune critical models, then re-test
DO track lineage in an AI-BOM for recall

🔄 Transfer Learning Attack