ML Security security risks
The OWASP Machine Learning Security Top 10 - attacks against ML models and pipelines.
Input Manipulation Attack (ML01 in the OWASP Machine Learning Security Top 10) is the deliberate crafting of model inputs-most often through small, carefully chosen perturbations-so that a trained model produces a wrong output at inference time. These crafted inputs are called adversarial examples, and using them to slip past a model is called an evasion attack.
A Data Poisoning Attack (ML02 in the OWASP Machine Learning Security Top 10) corrupts the training data of a model so that the model that comes out the other end is degraded, biased, or secretly backdoored. The attacker does not touch the deployed model directly-they touch the ingredients it learns from.
Model Inversion is an attack in which an adversary with query access to a trained machine-learning model reconstructs sensitive information about the data the model was trained on-or infers a private attribute of a known individual-by exploiting what the model reveals in its outputs. The richer the output (full confidence vectors, logits, embeddings, or gradients), the more the model leaks.
A Membership Inference Attack (MIA) is a privacy attack in which an adversary determines whether a specific data record was part of the model's training set. The attacker does not try to recover the record's contents-they already hold the record.
Model Theft occurs when an attacker obtains a functional copy of a proprietary machine-learning model-its behaviour, its parameters, or both-without authorisation. The stolen asset may be a byte-for-byte copy of the weights or a substitute model that reproduces the original's predictions closely enough to replace it.
An AI Supply Chain Attack is the compromise of any third-party component that flows into a machine-learning system: a pretrained model pulled from a hub, a public dataset, an ML library or framework, or the MLOps tooling that ties them together. Instead of attacking your trained model directly, the adversary poisons an ingredient you trust and let into your pipeline-so the compromise arrives pre-installed.
A Transfer Learning Attack targets the way modern machine-learning systems are built: instead of training a model from scratch, teams download a large pre-trained base model and fine-tune it on a smaller task-specific dataset. The attack manipulates the base model, or the transfer process itself, so that malicious behaviour-a hidden backdoor, an injected bias, or an inherited vulnerability-carries forward into the downstream fine-tuned model that the victim ships.
Model Skewing is an attack against a deployed model that keeps learning from production data. The attacker manipulates the model's feedback loop-the stream of clicks, ratings, reports, corrections, and labels that a continuously-learning system ingests-so that, retraining after retraining, the model's behaviour drifts ("skews") toward the attacker's goal.
An Output Integrity Attack targets the model's result after the model has produced it. The attacker tampers with the prediction on the path between the model and whatever consumes it-a downstream service, a database, a message queue, an operator's dashboard, or the user interface-so that the decision the system finally acts on is not the decision the model actually made.
Model Poisoning is the direct manipulation of a trained model's parameters, weights, or structure so that the deployed model behaves in a way the attacker chooses. The attacker does not (necessarily) touch the training data at all-they reach into the model itself: the serialized weight file, the entry in the model registry, the architecture definition, the hyperparameters, or-in federated learning-the model updates that a participant contributes to the global model.