When Compute Becomes a Strategic Constraint

For AI companies operating beyond experimentation and into production, compute is no longer an abstract infrastructure concern. GPU availability, pricing, and contract structure increasingly determine how fast teams can ship, how reliably they can serve customers, and how much financial risk they absorb along the way.

This shift is especially visible among teams building foundational AI infrastructure.

Modular is one of them.

Modular is building a unified platform for generative AI that spans the full development and deployment lifecycle. Their work includes a Kubernetes-based control plane for serving models at scale, Python-based APIs for modern graph development, and a new programming language designed to enable hardware-agnostic GPU programming. The ambition is clear: remove friction between developers and the world’s most powerful AI accelerators, regardless of vendor.

That ambition makes compute a first-order dependency. GPUs are not used occasionally or opportunistically inside Modular. They are woven into nearly every critical workflow, from research and benchmarking to continuous integration and customer-facing proof-of-value environments.

As Modular scaled, securing reliable GPU capacity stopped being an operational task and became a strategic one.

The Early Reality: Standard GPU Procurement, Standard Limitations

Like most AI teams, Modular initially sourced compute through the most obvious channels: large hyperscalers and cloud providers already embedded in their stack. Their systems were designed to be multi-cloud from the start, relying on Kubernetes to maintain portability and reduce vendor lock-in.

In the early stages, this approach was sufficient. Shape requirements were limited, workloads were relatively predictable, and on-demand instances or short block reservations for GPUs like A100s covered most needs.

But as Modular’s product matured, that equilibrium disappeared.

New workloads emerged. Hardware requirements diversified. Performance benchmarks began to matter more. Customer expectations increased. The GPU shapes required to stay competitive expanded to include newer NVIDIA and AMD accelerators.

What had once been “good enough” compute procurement quietly became a bottleneck.

Where GPU Procurement Broke Down

The first cracks appeared in contract structure.

Modular, like many fast-moving AI companies, could not forecast GPU usage years in advance. Product direction shifted. Customer demand fluctuated. New hardware generations arrived faster than long-term contracts could accommodate. Yet most vendors pushed for multi-year commitments that assumed stability rather than change.

At the same time, capacity was rarely guaranteed in practice. Even when contracts were signed, availability could vanish without warning. Hardware would be delayed, reprioritized, or quietly allocated elsewhere when demand spiked. In some cases, Modular encountered spot-like models where development environments could disappear mid-session, forcing engineers to rebuild from scratch. For a team building production systems, this was not just inconvenient — it was operationally untenable.

Hardware diversity introduced another constraint. As Modular’s roadmap evolved, access to specific GPU architectures became increasingly important. Support across both NVIDIA and AMD mattered — not just in theory, but in practice.

In reality, not all vendors offered a meaningful range of options across both ecosystems. Even when they did, access was often gated behind restrictive contract terms or vague delivery timelines. In several cases, roadmaps were unclear or simply did not exist, making long-term planning difficult for a team whose hardware needs were changing rapidly.

Perhaps the most underestimated cost was time.

Each procurement cycle involved weeks or months of conversations, repeated explanations of requirements, manual comparisons across pricing models, term lengths, networking capabilities, and compliance certifications. Senior engineers and leadership found themselves spending more time negotiating contracts than building product.

Over time, a pattern emerged: GPU procurement was consuming disproportionate attention while delivering diminishing returns.

The Breaking Point: Bait-and-Switch and Capacity Risk

As capacity needs increased further, Modular expanded its search aggressively. The team engaged with more than fifty vendors across hyperscalers, neoclouds, niche GPU providers, and direct sellers.

The experience was consistent.

Initial offers often looked attractive. Pricing seemed reasonable. Contract terms appeared flexible. Availability was implied. But once Modular attempted to expand capacity or adjust terms, conditions changed. Prices worsened. Contract durations lengthened. Expansion required new concessions.

In one particularly damaging case, Modular prepaid in full for GPUs that were repeatedly delayed week after week. Delivery arrived nearly a month late, with no compensation and no meaningful recourse. Despite having designed their infrastructure to avoid lock-in, the team found itself temporarily hostage to vendor behavior.

At that point, the conclusion became unavoidable: the traditional GPU procurement model was misaligned with how modern AI teams operate.

Resetting the Question: What Actually Matters When Buying GPUs?

Rather than continuing to optimize within a broken process, Modular stepped back and reframed the problem. Instead of asking which vendor to choose, they asked how GPU capacity should be sourced in the first place.

Several criteria emerged as non-negotiable.

Contract flexibility mattered more than nominal discounts. Modular needed short, recurring commitments that reflected the reality of a rapidly evolving product roadmap. Locking into multi-year assumptions created more risk than savings.

Pricing transparency mattered more than negotiated “special deals.” Opaque discounts tied to sales incentives were unstable by definition. Modular wanted pricing that reflected real market conditions and remained consistent as capacity scaled.

Hardware access mattered more than vendor loyalty. The ability to source different GPU shapes across NVIDIA and AMD — and to pivot as requirements changed — was essential.

Capacity guarantees mattered more than promises. If capacity was committed, it needed to be delivered, without repeated delays or hidden dependencies.

Finally, speed mattered. Procurement cycles needed to operate on engineering timescales, not procurement ones. Weeks or months between decision and deployment were no longer acceptable.

Discovering a Different Model for Reserved GPU Capacity

It was against this backdrop that Modular encountered Compute Exchange.

What stood out was not a specific feature, but a fundamentally different operating model. Instead of negotiating directly with vendors through opaque sales processes, Compute Exchange structured GPU sourcing as a marketplace. Capacity was surfaced transparently. Pricing reflected supply and demand. Contracts emphasized flexibility rather than lock-in.

Initially, there was skepticism. The model diverged sharply from industry norms. But after repeated negative experiences elsewhere, Modular was ready to try a different approach.

The transition was deliberate rather than impulsive. Modular evaluated whether this model could meet their criteria around contract terms, pricing discipline, hardware diversity, and delivery reliability.

It did.

What Changed After the Shift

The most immediate difference was speed.

Procurement cycles that previously stretched into months were reduced to roughly a week from initial requirement to hardware in hand. This alone removed a major bottleneck from both engineering and sales workflows.

Cost dynamics changed as well. By accessing GPUs at market prices rather than sales-rep-driven margins, Modular achieved significant savings on infrastructure spend. Given that GPUs represented the majority of their infrastructure costs, the financial impact was material.

More subtly, operational drag disappeared. Leadership no longer needed to manage endless negotiations or worry about expansion clauses quietly worsening terms. Procurement became repeatable, predictable, and software-driven.

Perhaps most importantly, Modular regained confidence in their ability to adapt. New GPU shapes could be added without renegotiating multi-year contracts. Capacity could expand without triggering unfavorable resets. Compute sourcing became a strategic enabler rather than a recurring risk.

Over time, Compute Exchange became Modular’s standard purchasing path for internal infrastructure, while still allowing the company to work with multiple providers as needed.

A Broader Pattern Across AI Infrastructure Teams

Modular’s experience is not unique.

As AI workloads move from experimentation into production, GPUs are increasingly treated less like hardware SKUs and more like financial commitments. Decisions about reserved GPU capacity now carry implications for cash flow, roadmap flexibility, and execution risk.

Traditional procurement models — built around long negotiations, rigid contracts, and asymmetric information — struggle under these conditions. They assume predictability where none exists.

More AI teams are recognizing that compute sourcing must evolve. Transparency, flexibility, and market-based pricing are no longer “nice to have.” They are prerequisites for scaling responsibly.

Conclusion: From Procurement to Strategy

For teams building production AI systems, the hardest part of compute is not deploying GPUs. It is committing to them under uncertainty.

Modular’s journey illustrates what happens when GPU procurement aligns with how modern teams actually build: faster decisions, lower risk, and more focus on product rather than process.

As hardware diversity increases and demand continues to grow, models that treat GPUs as tradable, market-priced resources — rather than opaque enterprise contracts — are likely to define the next phase of AI infrastructure.