Path & Payload

An AI Ingredients List Assumes You Know What the Ingredients Are

Last week, G7 agencies and CISA published joint guidance on what an AI software bill of materials (AISBOM) should include β€” the first G7-backed attempt to define baseline elements for an AI-focused SBOM.

But can the SBOM model β€” built around the idea that software has a discrete, auditable "ingredient list" β€” translates cleanly to AI systems? That's what I want to dig into here.

The concept behind an SBOM is straightforward: if you know everything that goes into a piece of software, you can identify supply chain risk before it surfaces as an incident.

Interest in SBOMs accelerated after Log4Shell in December 2021. In response to the disclosure, organizations spent weeks manually auditing their environments to determine whether the vulnerable Log4j library was present in their software β€” a process that would have taken only hours with accurate SBOMs. The lesson was clear enough that CISA has spent years pushing SBOM adoption across critical infrastructure, and the Biden executive order on cybersecurity made SBOMs a part of federal secure software expectations.

The G7 guidance extends the SBOM framework to AI systems. The reasoning is: if AI is being embedded in medical devices, weapons systems, and financial infrastructure, the same supply chain transparency logic should apply. But extending SBOMs to AI means reckoning with what the original framework was built to do β€” which parts of the logic carry over, and which parts run up against the structural properties of AI systems.

The SBOM model assumes you can walk the dependency graph

Traditional SBOMs work because software has a discrete, auditable structure. A compiled application has dependencies. Those dependencies have dependencies. The graph is finite, and it reflects ground truth β€” if a package is in the build, it's in the bill of materials. If it's not in the bill of materials, it's not in the build.

This is what made Log4j findable β€” there was something definitive to look for, and when you found it, you knew you’d found it. The SBOM model assumes the ingredient list is both complete and stable, that you can enumerate what's in a system and trust that enumeration to remain accurate.

AI systems don't work this way. The G7 guidance identifies seven categories of information an AISBOM should cover: metadata about the SBOM document itself, information about the AI system overall, details on the models used, datasets across the full model lifecycle, infrastructure required for operation, cybersecurity measures applied, and key performance indicators. That's a more expansive scope than a traditional SBOM by several orders of magnitude β€” and each category carries its own auditability problem.

Training data is not a package dependency

The category that the guidance handles most carefully is datasets. It calls for documentation across the full model lifecycle β€” but tractability varies significantly depending on the type of AI system.

For organizations fine-tuning or training on their own governed data assets β€” with lineage tracking, classification, and data contracts in place β€” provenance documentation is achievable. If your training data lives in a governed catalog with full lineage, you can document what went into a run, when it was accessed, and what transformations were applied.

But foundational model pretraining is a different problem entirely. It happens at the lab level and at a scale that makes the governed-data model difficult to apply. Large pretraining runs typically draw on web crawls, licensed corpora, synthetic data, and human feedback annotations β€” each with different provenance, different licensing status, and different levels of documentation. Some of it may be genuinely unknown to the organization producing the SBOM.

That's different from a package dependency, which is a discrete artifact with a version number, a checksum, and a maintainer. For foundational models, you can document what you intended to train on. But for closed commercial models, the organization deploying the system doesn't have any control over that documentation.

The provenance problem isn't identical across all AI systems, of course. Open-weight models β€” Llama, Mistral, and their derivatives β€” publish their weights and typically release model cards describing training corpora. That's a meaningful step toward transparency: you can inspect the architecture, reproduce the inference behavior, and at least partially audit what the base model was trained on. But when organizations fine-tune those base models on proprietary data, there's now a downstream training layer that sits entirely outside the base model's documentation. The AISBOM needs to capture both, and in practice the fine-tuning layer is often where the organization-specific risk concentrates.

Closed commercial models are the hardest case. Weights are proprietary, training data composition is not disclosed in any auditable form, and the organization deploying the model is largely relying on what the vendor chooses to tell them.

Model behavior isn't determined by its inputs

Even if you could perfectly document the training data, infrastructure, and cybersecurity measures for an AI system, that documentation wouldn't tell you how the model will behave. This isn't a limitation of the current guidance β€” it's a structural property of how large language models work.

Traditional software, generally speaking, does what its code says. It’s expected to behave reproducibly under controlled conditions. That reproducibility is what makes SBOMs useful as security artifacts β€” you know what's in the system, and the system does what the ingredients specify.

LLM-based systems are stochastic. The same prompt can produce different outputs depending on context window state, sampling parameters, and model variability. Behavior is emergent from the combination of architecture, training, fine-tuning, and deployment context in ways that can't be read off the ingredient list. A model trained on carefully documented, ethically sourced data can still hallucinate, comply with adversarial prompts, or behave unexpectedly in edge cases that never appeared in training. The ingredients don't fully determine the dish.

Dmitry Raidman, who has worked on AISBOM development with CISA and OWASP, makes this point about runtime. An SBOM is a static artifact β€” a snapshot of what's in a system at a point in time. Runtime behavior is dynamic. Documentation of training data doesn't tell you how a model behaves when it's deployed in a healthcare application, processing adversarial inputs at scale, or interacting with other AI systems in a multi-agent pipeline.

What the guidance delivers

The most straightforward application of the guidance is for the non-AI components of an AI system. A deployed AI application still runs on software: inference frameworks, API layers, orchestration libraries, container images, cloud infrastructure. All of that is conventional software with conventional dependency graphs, and traditional SBOM tooling applies to it without modification. An AISBOM that accurately documents the infrastructure layer gives security teams something they can actually act on β€” known-vulnerable packages, end-of-life dependencies, components sourced from untrusted registries. This is real supply chain transparency because there's a discrete artifact to enumerate.

Model versioning is another place where AISBOMs are useful. Knowing when model weights change β€” when a vendor pushes an updated version, when fine-tuning data changes, when a model is replaced β€” is meaningful operational information. An AISBOM that captures version identifiers and update history lets security teams notice drift and ask questions about what changed and why. That's not comprehensive provenance, but it's a useful signal.

For procurement and regulatory teams, while AISBOMs can’t serve as a stand-alone guarantee of safety or compliance, they’re useful as a way to compare systems against consistent criteria. A vendor who can produce detailed documentation of training data provenance, cybersecurity controls, and performance characteristics is in a different position from one who can’t. Organizations subject to EU AI Act requirements or sector-specific regulation in healthcare and finance will increasingly need to demonstrate that they've asked the right questions about the AI systems they deploy. An AISBOM framework gives both sides of that conversation a shared vocabulary for what the questions should be.

One of the persistent problems in AI vendor evaluations has been that there's no standardized set of questions about supply chain transparency. Organizations have been asking about security controls, model provenance, and training data in ad hoc ways, getting inconsistent answers, and having no baseline for what adequate documentation looks like. The G7 guidance creates that baseline β€” or begins to.

The guidance also acknowledges that the field hasn't solved this yet. It explicitly notes that these elements are voluntary, not mandatory, and that they will expand over time to keep pace with AI's advancement.

Sources

#Perspectives