What was announced
- $300 million Series B, announced 4 September 2026, led by Andreessen Horowitz at a $3 billion valuation. Total raised to date is $392 million.
- The participating investor list is the interesting part. Sapphire Ventures, M12 — Microsoft's venture arm — Arm, Menlo Ventures and Factory are named in the release. Arm and M12 on the same cap table is not a coincidence.
- Roughly six months after an $80 million Series A, per reports of the round. That detail is not in the company release, so treat it as reporting rather than a confirmed term.
- The product is an inference cloud that, in the company's words, “disaggregates AI models to run each phase of inference on the most appropriate silicon”, integrating GPUs, purpose-built AI accelerators and CPUs into a single heterogeneous system aimed at agentic workloads.
- The company claims “up to 10X gains in throughput and interactivity” from matching workload phases to optimised hardware. That is a company claim, unverified by any independent benchmark we can point at.
- Co-founder and chief executive: Zain Asgar. In March 2026 the company stated it has one of the top three frontier labs and one of the top three hyperscalers as customers, with “billions of dollars in contracted revenue”. Both customers are unnamed, and that figure is the company's own characterisation.
Take the valuation at face value and you learn very little: a $3 billion mark tells you what a16z believes about a market, not what the market has decided. The more useful question is what specifically got funded here, because the answer has a direct consequence for anyone who writes inference code for a living.
Why one chip stopped being the obvious answer
For most of the current cycle, the serving question had one answer: get NVIDIA GPUs, get as many as you can, run everything on them. That was correct for a good reason — the software worked, the kernels were tuned, and there was no practical heterogeneity to think about. What has changed is not that GPUs got worse. It is that a single model request turned out not to be a single kind of workload.
Prefill and decode want different machines
When you send a prompt to a large language model, the system does two quite different things. First it reads the whole prompt at once and builds the key-value cache — the prefill phase. That is dense matrix arithmetic across a large batch of tokens simultaneously, and it is compute-bound: it is limited by how many floating-point operations the hardware can perform. Then it generates the response one token at a time — the decode phase. Each of those steps touches the entire set of model weights and the growing KV cache to produce a single token, so it is limited not by arithmetic but by how fast memory can be read. Decode is memory-bandwidth-bound.
Those two phases have opposite hardware appetites, and running both on the same accelerator means one of them is always wasting the thing the other needs. Splitting them is not a novel idea — it is established practice in modern serving stacks, and we have written a full walkthrough of when prefill-decode disaggregation actually pays for teams running their own inference. What Gimlet is proposing is the logical extension: if the phases want different machines, stop restricting yourself to different pools of the same machine and put each phase on genuinely different silicon.
| Phase | What it does | Binding constraint | What that implies for hardware |
|---|---|---|---|
| Prefill | Reads the full prompt, builds the KV cache in one pass | Compute-bound — limited by arithmetic throughput | Rewards dense FLOPs and large batches; tolerates modest memory bandwidth |
| Decode | Emits tokens one at a time, re-reading weights and cache each step | Memory-bandwidth-bound — limited by how fast weights can be read | Rewards fast memory and high bandwidth per pound; wasted FLOPs are the norm |
| Orchestration and tool calls | Routing, retries, schema validation, agent control flow | Latency and branching, not tensor maths | Ordinary CPUs handle it fine; putting it on an accelerator burns the expensive thing |
Memory bandwidth being the decode constraint also explains why this is being funded now rather than two years ago. Memory has become the scarce input in its own right — we covered the consequences when DRAM sold out for 2026 and shifted the inference cost floor. When the bottleneck moves from arithmetic to memory, buying more of the same accelerator stops being an efficient answer, and architectural cleverness starts to look cheaper than brute force.
Treat the “up to 10X gains in throughput and interactivity” figure as a company claim, not a benchmark. There is no named model, no named hardware mix, no stated baseline and no independent reproduction behind it, and “up to” is carrying a great deal of weight. The same caution applies to the March 2026 statement about a top-three frontier lab, a top-three hyperscaler and “billions of dollars in contracted revenue” — both customers are unnamed and the figure is self-reported. The underlying engineering is real and well documented. The multiple is not yet evidence.
Why now: the silicon field is genuinely widening
A phase-placement product is only worth building if there is a real choice of silicon to place things on. Eighteen months ago that choice was thin. It is no longer thin, and the evidence arrived in the same week as this round.
On 8 September 2026, Qualcomm and Amazon announced a multi-generation deal covering custom AWS inference silicon plus optical interconnect, with Amazon granted a warrant for up to 25 million Qualcomm shares at $161.26, vesting against actual purchases. Note the structure carefully, because the headline figures being quoted for that deal — up to $60 billion — are a maximum tied to those vesting conditions, not contracted revenue. What matters for our purposes is not the number but the direction: a hyperscaler is committing to a non-NVIDIA inference part across multiple hardware generations, and pairing it with custom interconnect.
That is one of several. Qualcomm has already bought its way towards a CUDA alternative at the compiler layer — we looked at what that acquisition actually buys in our piece on the $3.9bn Modular deal. The large labs have been moving inference to TPUs where the economics justify it, as we covered in the great TPU migration. And specialist inference silicon keeps getting funded on its own terms; Etched raised $300 million for a chip that only runs transformers.
Put those together and Gimlet's bet reads clearly. They are not betting on any particular accelerator winning. They are betting that none of them does — that the end state is a fleet of mixed parts, and that whoever owns the placement layer across that fleet owns something valuable. Arm's presence on the cap table is consistent with that reading.
Every article here is written by a Verified Builder. Want your name on the next one?
AI Tech Connect lists AI engineers, founders and researchers across India and the UK — and the people hiring browse it to find them. Adding your profile is free.
Become a Verified Builder →What this means if you write inference code
Here is the part that has consequences for your week rather than for a cap table.
If the fleet is going mixed, the ability to make a model run well on something that is not an NVIDIA GPU stops being a curiosity and becomes a billable specialism. That is not a prediction about 2029; it is visible in the hiring now, on both sides of the India–UK corridor.
The skill is narrower and more specific than “knows about hardware”. It is the ability to take a model that currently runs acceptably on one accelerator and get it running at comparable quality and cost on another — which in practice means dealing with kernel coverage gaps, quantisation behaviour that differs between vendors, attention implementations that exist on one stack and not another, and a serving framework that was tuned for the part you are leaving behind. It is unglamorous, it is measurable, and very few people have done it end to end.
The concrete surfaces worth learning
| Stack | Where you can realistically get hands-on | What the hard part usually is |
|---|---|---|
| AMD ROCm | Neocloud providers offering MI-series instances; some UK and Indian GPU clouds now list them alongside NVIDIA | Kernel and library coverage gaps; getting a serving framework to perform rather than merely run |
| Cloud TPUs | Google Cloud regions serving both markets, including Mumbai and London | Rewriting for a different execution model; compilation behaviour that punishes dynamic shapes |
| Trainium and Inferentia | AWS, including the Mumbai and London regions most Indian and British teams already deploy into | Compiler toolchain fluency, and honest measurement of what the migration actually saved |
| Qualcomm parts | Edge and on-device work today; the AWS deal announced 8 September 2026 points at data-centre exposure over coming generations | A much thinner body of public tuning knowledge than the GPU world assumes |
| CPU-side orchestration | Anywhere — this is the one you can practise on hardware you already have | Knowing what genuinely does not need an accelerator, which is more than most teams think |
If you want the practical version of this rather than the market commentary, our guide to running inference across mixed-vendor GPU fleets covers the scheduling and measurement mechanics. Pair it with what AI infrastructure companies actually hire for if the goal is a role rather than a side project.
Do not learn this by reading. Take one open-weight model you already serve, run it on a second vendor's silicon, and publish a like-for-like comparison: same prompts, same output-quality check, tokens per second, cost per thousand tokens, and — the part everyone omits — the hours you spent making it work at all. That last number is the one hiring managers cannot get anywhere else, and it is the single most persuasive artefact you can put on a Builder profile. One reproducible benchmark beats a page of claimed familiarity with five stacks.
Why the India and UK framing is not decorative
The two markets meet this from different directions. India is producing AI engineering supply at volume and has a domestic funding cycle to match — Indian AI funding quadrupled to $676 million across 57 deals in the first half of 2026 — while a large share of the world's managed inference is operated out of Indian teams for customers elsewhere. Britain skews the other way: fewer people building foundational infrastructure, more organisations deploying and buying it, which is exactly the pattern behind the gap between Indian and British AI job postings.
Multi-silicon competence is unusually portable across that divide. A team in Bengaluru migrating a customer's serving stack from GPUs to Inferentia in AWS Mumbai is doing the same work as a team in Manchester doing it in the London region, for a British insurer with a cost problem. The hardware is the same, the toolchain is the same, and the evidence you produce travels. Very few AI skills are that cleanly transferable between the two markets. This one is.
What would make the claim believable
Four things would move the 10X figure from positioning to evidence, and they are also the questions to ask any vendor selling you a heterogeneous serving story.
A named baseline. Ten times faster than what — a single GPU running an untuned stack, or a competently configured modern serving framework? The gap between those two baselines is most of the claimed improvement in a lot of inference marketing.
A named model and hardware mix. Phase-placement gains depend heavily on model size, sequence length and batch shape. A result on a small model at long context tells you nothing about a large mixture-of-experts model at short context.
Interactivity measured properly. Throughput and interactivity are in tension: you can raise tokens per second per GPU by batching aggressively and make the experience worse. Give time to first token and inter-token latency at a stated concurrency, or the claim means little.
And somebody outside the company reproducing it. Two customers who cannot be named are not a reference.
The honest read
Gimlet has raised a large amount of money against a technically sound premise, at a valuation that assumes the premise becomes an industry default. The premise — that inference phases have different hardware appetites and that fleets are going mixed — is well supported by the serving literature and by what hyperscalers are doing with their own silicon. The specific performance claim is not yet supported by anything anyone outside the company can check. Both of those can be true at once.
What the round genuinely establishes is that multi-silicon inference is now a named, funded specialism with a $3 billion price tag attached — a different thing from a research topic, and it changes what a scarce skill looks like.
The useful response to that, if you build for a living, is not to have an opinion about Gimlet. It is to spend a weekend making something you already run work on silicon you have never touched, measure it honestly, and write down what broke. That artefact is worth more than the round is.