Pillar V: AI Security · § 06

Model management security (LLM05, LLM10)

Model management security covers two OWASP risks at once. LLM05 (Supply Chain Vulnerabilities) and LLM10 (Model Theft). The two read as opposites. One is about what enters our environment, the other about what leaves it. Operationally they share infrastructure. A registry, signed artifacts, controlled storage, and disciplined vendor relationships. Lose any of those, and both risks expand together.

The two threats

LLM05. Supply Chain. Compromised model weights from public registries, malicious dependencies in fine-tuning pipelines, or vendor breaches that expose the prompts and outputs we send.
LLM10. Model Theft. Weights exfiltrated by an attacker who then reverse-engineers proprietary behaviour or rebuilds a competing capability without our investment.

The control surface that mitigates both is the Model Registry.

Model Registry

Every model Bizzi runs is registered with the metadata an auditor or on-call engineer needs to make a decision.

Hash signature. SHA-256 of the weights, verified at load time. A mismatch fails the load.
Provenance. Base model, license, and a pointer to the fine-tune dataset (which itself has lineage, see §5).
Owner. Business Owner and Tech Lead, per Pillar III §1.
Release notes. What changed since the prior version.
Risk classification. From the six-step method (Pillar I §8).
Security review status. Sign-off from the CoE Security Lead, escalated to CPTO for high-risk releases.

Production loaders accept only models in the registry. Anything else fails closed.

Vendor LLMs

For commercial vendor models we do not control the weights, so the control surface shifts to the contract and the runtime.

Vendor agreement. Explicit no-training and no-retention terms on Bizzi prompts and outputs (Pillar II §6 vendor matrix).
API key rotation. On a fixed schedule and on every suspected incident.
No long-lived secrets in code. All vendor credentials are issued from KMS at runtime.
Annual vendor audit evidence. SOC 2 Type II or ISO/IEC 27001 report on file, refreshed yearly.

Open-source models

OSS models are valuable but carry distinct supply-chain risk. Public registries get tampered with, licenses change, and quality drifts between versions. We mitigate as follows.

Pull only from verified sources. HuggingFace verified accounts, official repository releases.
Verify the published hash against release notes before any local deployment.
Benchmark before deploy. A new OSS model passes our evaluation suite (Pillar IV §7) before it leaves the staging environment.
Pin specific versions. Auto-update on OSS model artifacts is disabled by policy.

Storage controls

Self-hosted weights live under standard ISMS storage controls (Pillar V §13).

At-rest encryption. AES-256.
Access control. Only inference services and CoE engineers with a documented business need.
Centralised audit log. Every access recorded.
Encrypted offsite backups with monthly restore tests.

Model theft mitigation

For proprietary Bizzi models, four layers reduce the risk that weights leave the building unobserved.

No download endpoint. Models are servable, not downloadable, via any API surface.
Watermarking. Experimental. We embed identifying signatures into model outputs so leaked use is detected. Currently in research, not in production.
API rate limits. Block extraction-by-querying at scale. This is the operational counterpart to §10.
Behavioural fingerprinting. Models have distinctive output distributions. We sample and store fingerprints so that if a model appears elsewhere, attribution is possible.