# The AI Supply Chain Problem: When Your Build Pipeline's Dependencies Include Untrusted Models
The software supply chain has fundamentally changed. For the past decade, defenders focused on one question: *what's in your code?* Which open-source packages, which versions, which transitive dependencies buried three layers deep? SolarWinds, Log4Shell, and XZ Utils all pointed to the same lesson: the risk lives not in code teams write, but in everything that produces it.
The lesson was painful but learnable. Track dependencies. Audit versions. Scan for known vulnerabilities.
Today, that model breaks. The risk has migrated. AI agents now write code during the build. Autonomous tools pull in packages without human review. Prompts—crafted text entered into AI systems—have become genuine inputs to the build pipeline, which means they're genuine attack vectors. When a malicious prompt steers an AI model toward including a compromised dependency, or when an AI agent autonomously calls an untrusted tool, the question "what's in your code?" no longer captures the actual threat surface.
The software supply chain has a new problem, and it's one most security programs were never designed to handle.
## The Threat: AI as a Load-Bearing Part of the Build
The scale of the shift is significant. In roughly 20 months since the Model Context Protocol (MCP) launched, AI tools have become embedded in how software gets built, deployed, and operated. Code is written by agents. Dependencies are selected by autonomous systems. Development teams are shipping with AI-generated code they may never fully review—not because they're negligent, but because the volume and velocity of AI-assisted development make line-by-line human review impractical at scale.
This changes the attack surface in concrete ways:
The threat is not hypothetical. The Shai-Hulud campaign this year demonstrated that self-propagating malicious packages can spread through developer toolchains at scale. The difference now is that AI tooling can accelerate that propagation and mask it—an AI-generated package might look plausible to human reviewers because the code itself is syntactically correct and contextually reasonable, even if it's malicious.
## Background and Context: How the Supply Chain Got Here
Software supply chain security has always been reactive. The field matured through incidents:
Each incident pushed the industry to better tooling: software composition analysis (SCA), dependency scanning, SBOM (Software Bill of Materials) generation, verification of package signatures. The frameworks got more sophisticated.
But all of these defenses assume a relatively stable model: *code is written, then it enters the supply chain, then it's deployed.* Human developers make deliberate choices about what to include. Scanning happens before deployment. The dependency tree is knowable because it's explicitly declared.
AI upended these assumptions. When an AI system can generate hundreds of lines of code in seconds, and when an autonomous agent can make tool-calling decisions without human intervention, the model of "deliberate choice followed by pre-deployment scanning" breaks down. The pipeline now includes actors—models and agents—that make choices within it, and those actors themselves become dependencies.
## Technical Details: Where the Risk Actually Moved
The provenance problem has expanded. Historically, supply chain security focused on *artifact provenance*: Where did this package come from? Is it signed? Has it been tampered with?
Today, provenance questions apply to multiple layers:
| Layer | Previous Question | New Question |
|-------|---|---|
| Code | Is the code signed? Is it from a trusted source? | Is the AI-generated code auditable? Who trained the model? |
| Dependencies | Which packages are included? What versions? | Which packages did the AI agent decide were necessary? Can we audit the decision? |
| Models | N/A | What is the model trained on? Who controls it? Has it been fine-tuned? |
| Tools | N/A | Which tools does the agent have access to? What can they do? Are they compromised? |
| Prompts | N/A | Who crafted the prompts? Could a prompt be injected to steer behavior? |
The technical reality is that validating AI-generated code *after* it's created is necessary but insufficient. A scanner can find malicious patterns in generated code—but only if the malicious code is syntactically detectable. More concerning: a tool that an AI agent calls autonomously may operate outside the scanner's view entirely.
One emerging approach is extending lineage tracking to the entire pipeline: tracing activity, provenance, and configuration from first commit through deployment, applying the same rigor to models and agents as to any other dependency. This means:
This is hard. It requires instrumentation at multiple levels. But without it, defenders have no way to correlate a incident (code was malicious) with its root cause (was it the model? The prompt? The agent's decision? A compromised tool?).
## Implications: The Alert Pile Won't Solve This
Most security programs are already overwhelmed. Static analysis tools generate thousands of findings. Dependency scanners flag dozens of vulnerabilities on a typical project. Prioritization is chaos—teams triage by CVSS score or by whether there's a public exploit, knowing full well that the severity score doesn't correlate with actual risk.
Adding "scan AI-generated code too" to this queue doesn't strengthen the program. It deepens the problem. More findings, same triaging capacity, same backlog.
The real implication is that the framework for supply chain security has to fundamentally change:
1. Lineage must extend upstream. You can't secure what you don't understand. If an AI agent generated a line of code, you need to know which model, which prompt, which tool-calling chain led to that code. This is not optional; it's the foundation.
2. Prioritization must be based on exploitability, not volume. A vulnerability that's in dead code or unreachable is noise. A vulnerability that's actively called and controlled by an attacker is a real chain of exploit. Once autonomous agents can generate thousands of lines of plausible code, the signal-to-noise ratio collapses unless you correlate findings with runtime context.
3. Governance has to account for untrusted actors in the pipeline. Developers are (hopefully) trustworthy. Models and agents are not. Access control, tool allowlisting, prompt vetting, and model verification become infrastructure concerns, not just code-review concerns.
This is why Gartner's June publication of the inaugural Magic Quadrant for Software Supply Chain Security matters. The market is formally acknowledging what security leaders have known: this is now a budget-line item, not a bolt-on afterthought.
## Recommendations: What Defenders Should Do Now
Immediate priorities:
Medium-term:
## HackWire Analysis
The core problem is governance, not just technology. Security teams can deploy scanners and SCA tools and prompt-injection tests. But those are reactive measures on top of a fundamentally broken assumption: that you can bolt AI into an existing supply chain security program without rebuilding the program itself.
The reason this matters *now* is timing. AI tooling has moved from "nice productivity enhancement" to "core infrastructure" in roughly two years. Developers are using AI code generation at scale. Enterprises are shipping AI-assisted software without fully understanding what entered the build. And the first serious attack campaigns—like Shai-Hulud—have already demonstrated that this surface can be exploited.
What's being missed in most coverage is the governance gap. A team can have excellent static analysis, perfect dependency scanning, and rigorous code review. If an autonomous agent decides to pull in a dependency because an AI model suggested it, and that dependency is malicious, the code review catches *the code*—but not the *decision*. The decision lives outside the traditional security control points.
The pattern here connects to broader supply chain lessons from the past five years: the risk lives not in artifacts, but in the processes that create them. With AI in the pipeline, that process now includes systems that make decisions autonomously and models that can be poisoned from the outside. Defending against that requires visibility into those systems, which most organizations simply don't have today.
Concrete next step: if you're using AI in your build pipeline, your security team should be able to answer, within 5 minutes, which models are active, which agents, and which tools they can reach. If you can't answer that, your supply chain security program is not actually in scope for AI. That's not a problem to announce in a memo; it's a gap to close, starting this week.
— HackWire Editorial
## Related Coverage