AI Supply Chain Security: How to Assess Models, Datasets and Dependencies

Secure the AI supply chain by inventorying every model, dataset, package, container and AI service you depend on; obtaining models from official sources with checked hashes or signatures and safe formats such as safetensors; reviewing licences and data rights; protecting training and retrieval data from poisoning; scanning and pinning software dependencies; assessing third-party model APIs, tools and MCP servers as vendors; recording all of it in an AI bill of materials; and controlling what reaches production through review and signed artefacts.

This article extends software supply chain practice to AI. Lineage of data is covered in AI data lineage, third-party tools in AI tool security and MCP security, self-hosted models in LLM self-hosting and the overall security picture in AI security for business applications.

Download models from official publisher accounts, verify hashes where published and record the exact revision. Prefer the safetensors format, which stores weights without executable code, over pickle-based formats that can run code when loaded; never load untrusted pickle files on systems with access to secrets. Emerging model signing, such as the OpenSSF model transparency project built on Sigstore, lets you verify that a model came from its claimed publisher.

Read licences carefully. Many popular models are open-weight but not open-source, with use restrictions, attribution requirements or thresholds; see LLM self-hosting for the distinction.

For datasets used in training, fine-tuning, evaluation or retrieval, record source, licence or legal basis, collection date and processing. Poisoning risk applies wherever outsiders can influence data: public web data, user-generated content, shared documents indexed for retrieval. Restrict who can write to sources that feed AI systems, monitor changes, validate datasets before training and keep provenance so suspicious data can be traced and removed. CISA and partner agencies have published AI data security best practices covering these risks.

AI projects pull in large dependency trees: ML frameworks, tokenizers, inference servers, vector database clients and agent frameworks, many moving fast. Apply normal software supply chain controls: pin versions, use lockfiles, scan for known vulnerabilities, verify package names to avoid typosquatting (AI coding assistants sometimes suggest non-existent packages), build containers from trusted bases and generate provenance for builds using frameworks such as SLSA.

Treat model API providers, AI SaaS vendors, plugins and MCP servers as suppliers. Assess data processing terms, retention, training use, regions, security certifications, incident notification and how model changes are communicated. For tools and MCP servers, review source or vendor reputation, required permissions and update practices, and watch for changed tool descriptions that could carry injected instructions.

An AI bill of materials lists models, datasets, software components and services with versions, sources and licences for each AI system. It speeds up response when a vulnerability or licence issue appears in a component, and it supports governance and customer questionnaires. The CycloneDX ML-BOM format is one standard way to express it; link it to your AI inventory in AI governance.

Serve models and packages to production only from internal registries populated through review, not directly from public hubs. Sign artefacts you build and verify signatures at deploy time. Run model serving with least privilege and restricted network egress. Monitor advisories for components in your bill of materials and re-evaluate models after updates.

Supply chain controls prevent some of the most damaging and least visible AI risks, from malicious model files to licence surprises. Tooling for model signing and AI BOMs is still maturing, and full provenance for large pre-trained models is often unavailable. Focus on verifiable steps you control: sources, formats, scanning, registries and records.

Verifying where a model came from does not tell you how it behaves. Before adopting a new model or version, run your evaluation suite, including safety, jailbreak and injection tests, and compare it with the model it replaces. Fine-tuned community models can carry altered behaviour that looks normal on common tasks. Re-evaluate after provider updates to hosted models too, since behaviour can change without notice; see LLM regression testing.

An AI bill of materials does not have to start as a formal standard document; a structured record per system already answers most questions. Map it to a standard such as CycloneDX later.

An illustrative scenario, not a client case: a team downloads a fine-tuned model from a community account because it scores well on a leaderboard. Review finds a pickle-based file from an unknown publisher and a licence that forbids commercial use. The team switches to the official base model in safetensors format, fine-tunes it internally with documented data, stores it in an internal registry and records it in the AI bill of materials.

AI systems inherit risk from every model, dataset, package and service they use. Verify sources, prefer safe formats, review licences and data rights, vet vendors and tools, and keep an AI bill of materials so you can respond quickly when something changes.