Open-Weight Coding Shoot-Out: DeepSeek V4 vs Mistral 3.5 vs Gemma 4

Thirty days, five frontier open-weight models

Between late April and the third week of May 2026, five frontier-class open-weight models shipped — one each from DeepSeek, Mistral, Meta, Alibaba and Google. Any one of these releases would have been the story of the month a year ago. Stacked into a single window, they collectively close the open-weight gap to the strongest closed-weight coders (Claude Sonnet 4 / 4.6, GPT-5, Gemini 2.5) and in one case overtake them on the most-cited coding benchmark in the field.

For builders in Bengaluru, Pune, London or Manchester evaluating whether to self-host in 2026, the question is no longer "are open-weight models good enough?" It is "which open-weight model fits the shape of my workload, my hardware budget and my licence constraints?" Each of the five we cover here has answered a different version of that question. There is no universal winner. There are five reasonable winners depending on what you are optimising for.

This is the comparative buyer's guide. We have already written dedicated deep-dives on each of these models individually — see the related-news links in the sidebar — but this piece is the single decision framework that compares them side by side and tells you which one to pick by workload class. If you want the catch-up on the Chinese open-weight coding-models trend in isolation, or the April open-weight catch-up across Mistral, Llama and GLM, those exist as separate pieces. Here we put all five contenders in one table and pick the right tool for the job.

The five-way comparison table

All figures are publicly published numbers as of 24 May 2026. SWE-Bench Verified is the human-checked subset of SWE-Bench and is the meaningful coding benchmark this season — HumanEval is near-saturated and stopped discriminating at the frontier a while ago. GPQA Diamond is the hardest tier of the graduate-level reasoning benchmark.

Model	Vendor / region	SWE-Bench Verified	GPQA Diamond	Context	Architecture	Hardware footprint
DeepSeek V4 Pro	DeepSeek · China	80.6%	90.1	1M tokens	Mixture-of-experts	Heaviest in the pack — wants a real GPU cluster
Mistral Medium 3.5	Mistral · EU (France)	77.6%	—	~128k tokens	128B dense	Multi-H100 for throughput; EU-friendly licence; released 29 Apr 2026
Llama 4 Scout	Meta · US	Not the SWE leader	—	10M tokens (longest)	Mixture-of-experts	Variable by deployment; long-context is the headline
Qwen 3.5	Alibaba · China	Competitive on coding, math and instruction-following	—	Long	Family of variants	Mature tooling; rolled out by early March 2026
Gemma 4	Google · US	Consumer-class	—	Shorter	MoE, 26B params, ~14 GB	~85 tokens/sec on consumer hardware — the laptop pick

Two numbers in that table deserve to be read twice. DeepSeek V4 Pro at 80.6% SWE-Bench Verified is, to our knowledge, the first open-weight model to credibly clear the 80% line on this benchmark. For comparison: Claude Sonnet 4 / 4.6 sits at roughly 77.2%, GPT-5 at 74.9% and Gemini 2.5 at 71.8%. The open-weight ceiling has overtaken the closed-weight ceiling on the single benchmark the industry agrees most closely tracks bug-fix-style coding work. That is genuinely new. We covered the base release in our DeepSeek V4 open-weight LiveCodesBench deep-dive at the time; the Pro variant pushes the line further.

The second number is Llama 4 Scout's 10M-token context window. No other model in this comparison comes close. If your workload involves whole-repository reasoning, multi-file refactors over very large codebases, long-running agent traces or document piles that genuinely exceed 200k tokens, Scout is the only credible self-host option in this list. For workloads under 100k tokens, the 10M context buys you nothing real — it is irrelevant if you do not use it.

Decision framework: pick by workload class

The shoot-out resolves cleanly once you state what you are actually optimising for. Five workload classes, five winners.

Pick DeepSeek V4 Pro if…

…you want the highest coding capability available in any open-weight model in May 2026 and you have the GPU budget to back it. V4 Pro is an MoE at the heavy end of the field; it is not the model you fine-tune on a single workstation. But for an organisation that runs a real cluster — IndiaAI Mission GPU pool participants, UK regulated-sector outfits with capacity on Isambard-AI, or any team paying the bill for a multi-node H100 / B200 deployment — V4 Pro is the model that puts you ahead of the closed-weight frontier on the bug-fix coding benchmark. If raw coding capability is your dominant axis, this is the pick.

Pick Mistral Medium 3.5 if…

…an EU-friendly licence matters to your procurement story and you would rather deploy a 128B dense model than wrestle with MoE routing. Mistral Medium 3.5 ships on 29 April 2026 with a 77.6% SWE-Bench Verified score — second in the open-weight pack, ahead of closed-weight Claude Sonnet 4 — and a licence framework that European procurement and UK regulated buyers find materially easier than the alternatives. Dense architectures are also simpler to serve at predictable latency, which matters when you are budgeting for inference cost per token. Our Mistral Medium 3.5 deep-dive goes into the dense-vs-MoE trade-off in detail.

Pick Llama 4 Scout if…

…long-context dominates your workload. Scout's 10M-token window is unmatched in this list. If you are building an agent that reasons over an entire monorepo, summarising or grepping across hundreds of legal documents, doing multi-day RAG runs with massive context windows or planning across very long agent traces, Scout is the only credible self-host pick here. The trade-off is that Scout is not the SWE-Bench leader — for narrow bug-fix coding it is bested by V4 Pro and Mistral Medium 3.5 — but no other model lets you stuff 10M tokens into a single prompt.

Pick Qwen 3.5 if…

…you want a mature, well-supported family with strong math, instruction-following and multilingual chops. Qwen 3.5 rolled out by early March 2026 and has had time for tooling, fine-tunes and ecosystem support to settle. It is competitive on coding, strong on math and quietly the best of the bunch at multilingual work — relevant if your product serves Hindi, Tamil, Bengali, Mandarin or Arabic users alongside English. Alibaba has been iterating quickly; Qwen 3.6 has since followed with a 27B coding-agent variant which we covered in our Qwen 3.6 coding-agent piece. For most teams, Qwen 3.5 is the safe, mature default. See our Qwen 3.5 multimodal deep-dive for the full picture.

Pick Gemma 4 if…

…consumer hardware or edge deployment is the target. Gemma 4 is a 26B MoE that occupies roughly 14 GB and pushes about 85 tokens per second on consumer hardware. That is the laptop-class pick. A solo Indian developer on an M-series Mac, a UK side-project builder running a 4090, an edge inference deployment on a single-GPU box at a small clinic — these are the workloads Gemma 4 was built for. It will not match V4 Pro or Mistral Medium 3.5 on the hardest coding benchmarks, but it is the only model in this list that runs comfortably on hardware you can carry.

Self-hosting caveats: what the benchmark tables do not tell you

The five-way table is a great starting point and a poor finishing point. Three caveats deserve to live in any honest evaluation document.

Open-weight is not open-source. Open-weight means weights are downloadable and runnable. Open-source — by the Open Source Initiative's definition — requires the training data, training code and a permissive licence. None of the five models in this comparison clear that bar. Mistral comes closest with an EU-friendly licence; Meta's Llama community licence has well-known restrictions; DeepSeek and Qwen each have bespoke terms. If your procurement team is checking an "open source AI" box, none of these models qualify in the strict sense, even though they are all properly open-weight.

Supply-chain risk is real. When you self-host, you take on the responsibility for weight integrity, provenance and the long tail of fine-tune mischief. Compromised weights, poisoned fine-tunes or vendor-specific telemetry hooks in inference runtimes are all real concerns at scale. A closed-weight API at least concentrates the supply-chain blast radius at the vendor. Self-hosting moves the blast radius onto your own infrastructure team, and that is a real cost — particularly for small UK Builders without a dedicated security function.

Fine-tuning expertise is the hidden moat. A 77.6% SWE-Bench Verified base model that you do not know how to fine-tune to your codebase is worth less than a 70% base model that you can adapt to your stack in a weekend. The vendors who win the open-weight long game will be the ones that publish good fine-tuning recipes, ship reproducible LoRA scripts and document the gotchas. Mistral and Qwen are ahead on documentation; DeepSeek is improving; Llama has the largest community ecosystem; Gemma has the smoothest path for laptop-class fine-tuning.

The IN + UK angle: data residency and cost-per-token economics

For Indian and UK builder teams, the self-host calculus has two real drivers beyond the benchmark numbers.

Data residency. An Indian BFSI firm subject to RBI guidance on customer data, or a UK NHS-adjacent health-tech under DPA and ICO scrutiny, often cannot send source code or customer-derived prompts to a US-hosted closed-weight API. Open-weight self-host is the only legal path for whole categories of regulated workload. In those cases, the question is not "is V4 Pro better than Claude Sonnet 4?" — it is "which of the five open-weight models can I deploy inside my own VPC tomorrow?" The answer is usually Mistral Medium 3.5 or Gemma 4, depending on whether you have a cluster or a workstation.

Cost-per-token economics. The NVIDIA B200 vs H100 inference economics picture has shifted meaningfully in 2026. For high-volume inference — anything past roughly a million tokens a day, depending on model size — self-hosting starts to undercut closed-weight API pricing on a per-token basis. The IndiaAI Mission GPU pool gives Indian Builders subsidised access to multi-node H100 capacity; the UK's Isambard-AI provides similar access for UK academic and regulated-sector teams. Both meaningfully change the break-even point at which "host your own DeepSeek V4 Pro" beats "pay Anthropic per token". Run the numbers honestly before you write the procurement memo.

How does this compare against closed-weight Claude, GPT and Gemini?

The honest comparison against the strongest closed-weight coders looks like this on SWE-Bench Verified:

• DeepSeek V4 Pro — 80.6%
• Mistral Medium 3.5 — 77.6%
• Claude Sonnet 4 / 4.6 — ~77.2%
• GPT-5 — ~74.9%
• Gemini 2.5 — ~71.8%

On that single number, two open-weight models now beat all three closed-weight frontier coders. Our piece on the May 2026 coding-agent leaderboard tracks the closed-weight side in full detail.

The caveat is that SWE-Bench Verified does not measure everything. Agentic tool-use — the bit where the model picks the right tool, reads its output sensibly, plans the next step and recovers from errors — is still where closed-weight models have a real edge. Multi-day agent traces, deeply integrated IDE experiences and the long-tail of "the API just behaves" benefits remain easier to get from Anthropic, OpenAI or Google than from any self-hosted setup. The open-weight gap has closed on the headline benchmark; the agentic-tool-use gap has not.

The bottom line

Five frontier-class open-weight coding models in 30 days is not normal. It is a market in genuine, healthy competition — and for the first time in a long time, the open-weight winner on the most-cited coding benchmark sits above every closed-weight competitor. That is a real story.

The practical guidance for an Indian or UK Builder in May 2026: stop reading leaderboards in the abstract. Pick your dominant workload class. If it is hard coding on a cluster, DeepSeek V4 Pro. If it is EU-licence-friendly dense inference, Mistral Medium 3.5. If it is long-context, Llama 4 Scout. If it is multilingual or math-heavy, Qwen 3.5. If it is laptop-class or edge, Gemma 4. Make the choice deliberately, then commit to fine-tuning and serving that one model well rather than chasing the next release.

The open-weight gap has closed. The execution gap — fine-tuning, serving, monitoring, governance — is now where Builders win or lose. The model you pick matters less than how well you ship with it.

Primary comparison source: Codersera's best open-source LLM 2026 round-up.