What changed on 14 July
On 14 July 2026, Boundless — a network built for zero-knowledge (ZK) proving, the cryptographic workload that lets blockchains verify computation without re-running it — announced that it is opening its GPU fleet to general AI inference as managed infrastructure. The pitch is blunt: the same roughly 4,000 GPUs that grind out cryptographic proofs are idle a lot of the time, and idle silicon is wasted money. Point that spare capacity at AI inference and, according to Boundless's early benchmarks, you can run some workloads for as much as 50% less than a comparable hyperscaler cloud.
The catch is in the fine print, and it is the whole story. Those savings are pitched at asynchronous workloads — the kind that do not need an instant response — running on consumer-grade cards and GPUs originally bought for crypto mining and proving. Operators join the network by staking its native ZKC token, with the size of the stake tied to how much they can earn. A full product launch is planned for later this summer; for now there is a waitlist. (Sources: SiliconAngle, The Block.)
Strip away the token mechanics and this is the latest entrant in a fast-growing category: decentralised or "idle-GPU" compute marketplaces trying to undercut AWS, Azure and Google Cloud on price. For a builder in Bengaluru or Bristol staring at an inference bill that grows every month, a headline 50% cut is worth understanding properly — including the parts that do not survive contact with production.
Why idle and decentralised GPUs are genuinely cheap
The price gap is not magic, and it is not (only) marketing. It comes from four structural facts, and each one is also a limitation.
- The capital is already sunk. An ex-mining GPU's purchase price was written off against crypto revenue years ago. The operator's marginal cost is close to electricity plus a thin margin, so they can price well under a cloud that has to amortise data-centre real estate, cooling, networking and an enterprise support org.
- There is no SLA to pay for. Hyperscaler pricing bakes in uptime guarantees, redundancy and reserved-capacity premiums. Decentralised networks typically offer none of that. No latency guarantee, no promise a specific node stays up, no throat to choke at 3am. You are buying spare capacity, not a contract.
- The cards are consumer-grade. A pool of RTX-class gaming and mining cards is cheaper per FLOP than a rack of H100s or B200s — but it is also slower per token, has less VRAM per card, and usually lacks the high-bandwidth NVLink interconnect that makes multi-GPU model sharding fast. Large models that must be split across cards suffer; models that fit on a single card are fine.
- Cold starts are the norm. On a marketplace of intermittently available nodes, your model weights may need to be loaded onto a fresh card before a job runs. That cold-start delay is invisible in a batch pipeline and fatal in an interactive one.
Read that list again and the shape of the ideal workload falls out on its own: something throughput-bound, tolerant of delay and retries, running a model that fits on one card, on data you would be comfortable sending to an anonymous operator. That is a real and useful category — it is just not most of what people mean when they say "serving a model in production".
Before you chase a 50% headline saving, split your inference bill into two buckets: interactive (user is waiting) and offline (a queue is waiting). For most teams the offline bucket — nightly embedding refreshes, eval runs, batch scoring — is 30 to 60% of GPU spend and has no latency requirement at all. That bucket, and only that bucket, is where decentralised GPUs pay off. Move it first, measure, then decide.
Why the timing matters: inference is where the money goes
Boundless is aiming at the right target. Industry analysts have long pointed out that inference — not training — dominates the lifetime compute spend of a deployed model, with widely cited estimates putting it in the region of 80 to 90% of lifecycle compute once a model is in production and serving traffic every day. Gartner, cited in the SiliconAngle report, projects that inference will consume 65% of AI-optimised infrastructure spending by 2029. We would treat both as directional analyst projections rather than precise measurements — but the direction is not in dispute. The recurring cost of serving is the bill that keeps growing.
Layer on the second reported pattern — that enterprise GPU utilisation is often strikingly low, with expensive accelerators sitting idle for large stretches — and you get the thesis behind every idle-compute marketplace: the world has already bought far more GPU than it uses at any given moment, and someone should sell the gap. For a cost-conscious builder, that is a tailwind worth riding, provided you ride it on the right workloads. If you want the deeper economics, we broke down the full picture in the economics of AI inference cost in 2026 and the scale of wasted capacity in the GPU-utilisation idle problem.
Where idle-GPU inference fits — and where it doesn't
The decision is not "decentralised versus cloud" in the abstract. It is per-workload. Here is the split we would use.
| Workload | Latency need | Decentralised / idle GPU? | Why |
|---|---|---|---|
| Overnight embedding backfill | None (batch) | Strong fit | Throughput-bound, retryable, fits on single cards |
| Offline eval / benchmark runs | None | Strong fit | Can wait hours; cost dominates over speed |
| Synthetic data + data labelling | None | Strong fit | High volume, non-urgent, often on public data |
| Batch document classification | Minutes OK | Conditional | Fine if data is non-regulated; watch data residency |
| Interactive chat / copilots | Sub-second | Avoid | No latency SLA; cold starts break the UX |
| Real-time agents with tool loops | Low, predictable | Avoid | Tail latency and node churn compound across steps |
| Regulated / personal-data inference | Any | Avoid | GDPR / DPDP residency and processor-agreement gaps |
It helps to place Boundless on the wider spectrum of how you can buy inference, because "decentralised" is one point on a line, not a world of its own.
| Option | Relative cost | Latency / SLA | Best for |
|---|---|---|---|
| Managed cloud endpoint (hyperscaler) | Highest | Guaranteed, low | Regulated data, interactive serving, tight SLAs |
| Serverless GPU (per-second, managed) | Medium | Good, some cold start | Spiky traffic, autoscaling, single-region control |
| Decentralised / idle GPU (Boundless et al.) | Lowest | Best-effort, no SLA | Async batch on non-regulated data |
Two risks travel together on any decentralised GPU network, and both land on you, not the operator. Reliability: a pool of anonymous, intermittently-available nodes has no uptime contract — you must design for jobs vanishing mid-run and retry idempotently. Data governance: you often cannot see which physical machine or jurisdiction runs your job, which is very hard to square with UK and EU GDPR data-residency duties or India's DPDP Act. Never send personal, health, financial or otherwise regulated data to a network you cannot bind with a data-processing agreement. Keep it to synthetic, public or already-anonymised inputs.
Every article here is written by a Verified Builder. Want your name on the next one?
AI Tech Connect lists AI engineers, founders and researchers across India and the UK — and the people hiring browse it to find them. Adding your profile is free.
Become a Verified Builder →A decision rule for Indian and UK builders
You do not need a spreadsheet to make the first call. Run every candidate workload through three questions, in order. Fail any one and it stays on your existing cloud.
- Can it wait? If a human or a live request is blocked on the result, stop — this is interactive, keep it on managed infrastructure with an SLA. If a queue is waiting and nobody notices a few minutes' delay, continue.
- Is the data clean of regulation? If the input contains personal data of UK or EU residents (GDPR) or Indian data principals (DPDP), stop — the residency and processor-agreement gaps on a decentralised network are not worth the saving. If it is synthetic, public, or robustly anonymised, continue.
- Does the model fit on one card? If it needs to be sharded across GPUs with fast interconnect, the lack of NVLink will erode your saving. If it fits on a single consumer card, you have a genuine candidate.
Three yeses and it is worth piloting. In practice, for a typical Indian or UK team, that funnel lands you on the same set of jobs: nightly embedding pipelines, offline evaluation, synthetic data generation and non-urgent batch classification on public data. Start there, keep your interactive and regulated traffic exactly where it is, and treat any saving as a bonus on a specific slice rather than a wholesale migration.
Run decentralised GPUs behind an abstraction you control — a job queue and a thin router — not wired directly into application code. That way a network like Boundless is one interchangeable backend among several. If reliability disappoints, if pricing shifts, or if the summer launch slips, you reroute that queue to a serverless GPU provider without touching the rest of your stack. For the self-hosting fundamentals underneath all of this, see our guide to self-hosted LLM serving with quantisation and batching.
The healthy-scepticism section
A few things to hold in mind before the marketing settles into received wisdom. First, "up to 50%" is a ceiling on a favourable workload, not an average — the honest comparison is decentralised batch pricing against your cloud's batch or spot tier, not its on-demand list price, and that gap is narrower. Second, the benchmarks are the vendor's own and pre-full-launch; independent numbers do not exist yet, so treat the figure as a claim to test, not a fact to bank. Third, token-staked supply is a novel reliability model with no long track record for AI serving — incentives that work for ZK proving may or may not deliver the availability an inference queue needs.
None of that makes Boundless a bad idea. It makes it a tool — a cheap backend for a specific class of non-urgent, non-regulated batch work, worth a pilot and worth measuring honestly. The builders who win with it will be the ones who moved the right 40% of their bill and left the rest untouched, not the ones who chased a headline and moved a copilot onto best-effort silicon. Idle GPUs are real money on the table. The discipline is in taking only the money you can safely reach.
Primary coverage: SiliconAngle, 14 Jul 2026 and The Block.