Serve Many LoRA Adapters From One GPU

Twenty customers, twenty adapters, one GPU budget

The trap is pleasant on the way in. A customer asks for output in their house format, you tune a small adapter, it beats prompting, everyone is happy. A second customer asks the same. By the eighth you have a training pipeline, an artefact store and a habit. By the twentieth someone in finance asks why the infrastructure line grew twentyfold, and the honest answer is that you designed a training story and let serving follow it.

This guide is about the serving story. It assumes you can already produce an adapter — our eval-driven LoRA and QLoRA recipe covers that — and that you know the generic levers for cheaper inference, handled in our guide to cutting self-hosted serving costs. Neither answers the question here: given a fleet of adapters over one shared base, how do you serve all of them from hardware you can afford?

The short version, before the detail:

• One base, many adapters, chosen per request. The base model is loaded once; adapters are selected per request, so several custom models share one GPU with only the small adapters swapped in and out.
• Adapter weights are almost free; the KV cache is not. In the worked example below, going from 40 resident adapters to 200 costs under half a gigabyte. The KV cache costs sixty.
• Hot-swapping is a runtime operation, not a redeploy. Adapters load, unload and replace in place while the server keeps taking traffic — which is what makes continuously-updated adapters feasible.
• Routing is where the correctness bugs live. Cold starts, thrashing, and the genuinely dangerous case of a request served by the wrong tenant's adapter.
• Aggregate quality dashboards will lie to you. Nineteen healthy tenants and one broken one averages out to green.
• Sometimes the answer is not to do this. Section seven is the honest one.

How multi-adapter serving actually works

A LoRA adapter is not a model. It is a pair of small matrices attached to selected projection layers, and at inference the adapted layer computes the ordinary base projection plus a scaled low-rank correction. That structure — base output plus a separate additive term — is the entire reason this pattern works. Because the correction is separable, the expensive shared computation happens once for a whole batch while a different low-rank term is applied to each sequence in it. Requests from different customers sit in the same batch, on the same GPU, against the same frozen weights, and still behave like different models.

The second reason is size. Adapter parameters scale with the rank and with the number and dimensions of the adapted modules, and nothing else. They do not scale with the base model's parameter count in any meaningful way, which means an adapter is typically orders of magnitude smaller than the model it adapts. That asymmetry is what turns "twenty custom models" from a hardware procurement exercise into a memory-management detail.

The reference implementation most teams meet first is vLLM, which supports multi-LoRA serving and documents it on the LoRA adapters page under features/lora. Its design was influenced by the paper S-LoRA: Serving Thousands of Concurrent LoRA Adapters, worth reading if you want the reasoning rather than the API. Research has continued since — InfiniLoRA looks at disaggregated multi-LoRA serving, separating adapter hosting from base-model serving — and reference deployment patterns exist on managed platforms including Amazon SageMaker AI and Amazon Bedrock. That is the landscape as of August 2026; the concepts below will outlive the product names.

Be precise about what you are trading. Applying an adapter per request is not free — the low-rank term is real arithmetic a merged model would not do, and batches mixing many distinct adapters are harder to schedule than batches sharing one. How much that costs depends on your model, rank, traffic mix and hardware, which is why this guide quotes no number for it. Measure it yourself. And note the comparison that decides the question: not multi-adapter serving against a merged model on a dedicated card, but against paying for twenty cards.

Hot-swapping without draining traffic

The naive operational model is that the set of adapters is fixed when the server starts. Add a customer, restart the fleet. That is survivable at three tenants and intolerable at thirty, and it is the reason runtime adapter management exists.

In vLLM, dynamic loading and unloading at runtime sits behind an environment flag — VLLM_ALLOW_RUNTIME_LORA_UPDATING, as of August 2026 — after which adapters can be added to and removed from a running server without interrupting it. The gate exists for a reason: an endpoint that loads arbitrary weights into your inference server is serious attack surface, so it should be reachable only from your control plane, never from the public request path.

The more interesting capability is replacement rather than addition. When loading dynamically you can ask for an in-place update — a load_inplace parameter that replaces an existing adapter's weights while keeping the same name. The motivating case is asynchronous reinforcement learning, where a training loop produces updated weights continuously and serving must pick them up without a break in inference. The same mechanism suits any online-tuning setup: a nightly refresh from yesterday's accepted corrections, or a per-tenant adapter that improves as their corpus grows.

In-place replacement is also the sharpest edge in this whole architecture, because the name stays constant while the behaviour underneath it changes. Every log line, every metric and every eval result that says cust-04 now refers to something that may or may not be what it referred to an hour ago.

What actually runs out first

Ask a team what limits their adapter count and most will guess adapter memory. It is almost never adapter memory. Here is a capacity estimator you can run against your own configuration. Standard library only, no dependencies, and every input is an assumption you replace.

"""adapter_budget.py — how much room do N LoRA adapters actually take?

Standard library only. Every number below is an assumption you replace with
your own. Nothing here is measured on real hardware.
"""

GIB = 1024 ** 3
MIB = 1024 ** 2


def adapter_params(rank, modules):
    """LoRA adds A (rank x d_in) and B (d_out x rank) per adapted module."""
    return sum(rank * (d_in + d_out) for _, d_in, d_out in modules)


def kv_bytes_per_token(layers, kv_heads, head_dim, dtype_bytes):
    """Two tensors (K and V) per layer, per token."""
    return 2 * layers * kv_heads * head_dim * dtype_bytes


def report(name, gpu_gib, base_params, base_dtype_bytes, rank, modules,
           adapter_count, adapter_dtype_bytes, overhead_gib,
           layers, kv_heads, head_dim, kv_dtype_bytes):
    per_adapter = adapter_params(rank, modules) * adapter_dtype_bytes
    adapters_total = per_adapter * adapter_count
    base_bytes = base_params * base_dtype_bytes

    budget = gpu_gib * GIB
    spent = base_bytes + adapters_total + overhead_gib * GIB
    kv_room = budget - spent

    per_token = kv_bytes_per_token(layers, kv_heads, head_dim, kv_dtype_bytes)

    def line(label, value, unit):
        print(f"{label:<30}{value:>10} {unit}")

    print(f"--- {name} (all figures are assumptions) ---")
    line("GPU memory budget", f"{gpu_gib:.1f}", "GiB")
    line("Base weights", f"{base_bytes / GIB:.2f}", "GiB")
    line(f"One adapter (rank {rank})", f"{per_adapter / MIB:.2f}", "MiB")
    line(f"{adapter_count} adapters resident", f"{adapters_total / GIB:.2f}", "GiB")
    line("Runtime overhead (assumed)", f"{overhead_gib:.2f}", "GiB")
    line("Left for KV cache", f"{kv_room / GIB:.2f}", "GiB")
    if kv_room <= 0:
        print("  -> negative: this configuration does not fit at all")
        return
    tokens = int(kv_room // per_token)
    line("KV cache per token", f"{per_token / 1024:.2f}", "KiB")
    line("Total cached tokens", f"{tokens:,d}", "tokens")
    line("Concurrent 8k-token requests", f"{tokens // 8192:,d}", "requests")
    print()


# A hypothetical 8B-class decoder. Substitute your model's real config.
MODULES = [
    ("q_proj",     4096,  4096),
    ("k_proj",     4096,  1024),
    ("v_proj",     4096,  1024),
    ("o_proj",     4096,  4096),
    ("gate_proj",  4096, 14336),
    ("up_proj",    4096, 14336),
    ("down_proj", 14336,  4096),
]

COMMON = dict(
    gpu_gib=80.0,            # assumption: one 80 GiB accelerator
    base_params=8_000_000_000,
    base_dtype_bytes=2,      # FP16 base weights
    rank=16,
    modules=MODULES,
    adapter_dtype_bytes=2,
    overhead_gib=4.0,        # assumption: CUDA context + activations + workspace
    layers=32,
    kv_heads=8,              # grouped-query attention
    head_dim=128,
)

report("40 adapters, FP16 KV cache", adapter_count=40, kv_dtype_bytes=2, **COMMON)
report("40 adapters, INT8 KV cache", adapter_count=40, kv_dtype_bytes=1, **COMMON)
report("200 adapters, INT8 KV cache", adapter_count=200, kv_dtype_bytes=1, **COMMON)

Run against that clearly hypothetical configuration — an invented 8B-class decoder with grouped-query attention, not any real released model — it prints:

--- 40 adapters, FP16 KV cache (all figures are assumptions) ---
GPU memory budget                   80.0 GiB
Base weights                       14.90 GiB
One adapter (rank 16)               2.50 MiB
40 adapters resident                0.10 GiB
Runtime overhead (assumed)          4.00 GiB
Left for KV cache                  61.00 GiB
KV cache per token                128.00 KiB
Total cached tokens              499,721 tokens
Concurrent 8k-token requests          61 requests

--- 40 adapters, INT8 KV cache (all figures are assumptions) ---
GPU memory budget                   80.0 GiB
Base weights                       14.90 GiB
One adapter (rank 16)               2.50 MiB
40 adapters resident                0.10 GiB
Runtime overhead (assumed)          4.00 GiB
Left for KV cache                  61.00 GiB
KV cache per token                 64.00 KiB
Total cached tokens              999,443 tokens
Concurrent 8k-token requests         122 requests

--- 200 adapters, INT8 KV cache (all figures are assumptions) ---
GPU memory budget                   80.0 GiB
Base weights                       14.90 GiB
One adapter (rank 16)               2.50 MiB
200 adapters resident               0.49 GiB
Runtime overhead (assumed)          4.00 GiB
Left for KV cache                  60.61 GiB
KV cache per token                 64.00 KiB
Total cached tokens              993,043 tokens
Concurrent 8k-token requests         121 requests

Read the third block against the second. Multiplying the adapter count by five costs 0.39 GiB and about one percent of your KV capacity. Halving the KV cache element size, by contrast, doubles the number of concurrent requests you can hold. The adapters are noise. The cache is the budget.

That reframes the optimisation problem. On a multi-adapter server the levers that matter are the ones governing cached tokens: quantising the KV cache to INT8 roughly halves its memory, and paged management — treating the cache as pages the way an operating system treats virtual memory — typically improves utilisation two to four times by removing fragmentation and over-reservation. Together they compound into a four-to-eight-fold reduction in KV memory, worth vastly more than any adapter-side saving. The mechanics of both are in the serving cost guide; the point here is which lever to reach for.

One multi-adapter-specific interaction is worth checking rather than assuming. Prefix caching — reusing the computed cache for a shared prompt prefix — is a large win on single-model servers. But if your adapted modules include the key and value projections, the cached tensors for a prefix depend on which adapter produced them, and reusing them across adapters would be incorrect. Expect your engine to key the prefix cache by adapter, making the hit rate per-adapter rather than global, so a long tail of tenants each get a thinner slice of that benefit. Verify the behaviour on your own stack before planning capacity around prefix-cache hit rates.

Routing: tenants, tasks, cold starts and fallbacks

Every request has to become an adapter name, or a decision not to use one. That mapping is small, boring code where both the performance and the security problems live. It must do three things: resolve a tenant or task identifier to an adapter; bound how many adapters stay resident, so a long tail of rare customers cannot evict the ones carrying your traffic; and decide what happens when the adapter is missing, unloadable or unmapped. The backend interface in this sketch is deliberately ours rather than a guess at a vendor's SDK — wire load and unload to whatever your engine exposes.

"""adapter_router.py — pick an adapter per request, bound how many stay resident."""

from collections import OrderedDict


class AdapterBackend:
    """Implement these two against your serving engine."""

    def load(self, name, path):
        raise NotImplementedError

    def unload(self, name):
        raise NotImplementedError


class FakeBackend(AdapterBackend):
    """Stand-in so the example below runs anywhere."""

    def __init__(self):
        self.events = []

    def load(self, name, path):
        self.events.append(("load", name))

    def unload(self, name):
        self.events.append(("unload", name))


class AdapterRegistry:
    """Least-recently-used set of resident adapters, with pinning."""

    def __init__(self, backend, catalog, max_resident, pinned=()):
        self.backend = backend
        self.catalog = dict(catalog)          # name -> artifact path
        self.max_resident = max_resident
        self.pinned = set(pinned)
        self._resident = OrderedDict()        # name -> True, oldest first

    def _evict_one(self):
        for name in list(self._resident):
            if name in self.pinned:
                continue
            del self._resident[name]
            self.backend.unload(name)
            return name
        raise RuntimeError("every resident adapter is pinned; raise max_resident")

    def ensure(self, name):
        """Return 'hit' or 'cold'. Raises KeyError if the adapter is unknown."""
        if name in self._resident:
            self._resident.move_to_end(name)
            return "hit"
        if name not in self.catalog:
            raise KeyError(name)
        while len(self._resident) >= self.max_resident:
            self._evict_one()
        self.backend.load(name, self.catalog[name])
        self._resident[name] = True
        return "cold"

    def resident(self):
        return list(self._resident)


class Router:
    """tenant -> adapter, then task -> adapter, then the bare base model."""

    def __init__(self, registry, tenant_map, task_map=None):
        self.registry = registry
        self.tenant_map = dict(tenant_map)
        self.task_map = dict(task_map or {})

    def _candidate(self, tenant_id, task_id):
        if tenant_id in self.tenant_map:
            return self.tenant_map[tenant_id], "tenant"
        if task_id in self.task_map:
            return self.task_map[task_id], "task"
        return None, "base"

    def route(self, tenant_id, task_id=None):
        """Return (adapter_name_or_None, reason). Never raises for the caller."""
        name, why = self._candidate(tenant_id, task_id)
        if name is None:
            return None, "base:no-adapter-mapped"
        try:
            state = self.registry.ensure(name)
        except KeyError:
            # Mapped to an adapter that is not in the catalog: serve the base
            # rather than 500. Alert on this — it means a bad deploy.
            return None, f"base:missing-artifact({name})"
        except Exception as exc:                      # load failed on the engine
            return None, f"base:load-failed({name}: {exc})"
        return name, f"{why}:{state}"


if __name__ == "__main__":
    backend = FakeBackend()
    catalog = {f"cust-{i:02d}": f"s3://adapters/cust-{i:02d}" for i in range(1, 7)}
    catalog["summarise-v3"] = "s3://adapters/summarise-v3"

    registry = AdapterRegistry(backend, catalog, max_resident=3, pinned={"cust-01"})
    router = Router(
        registry,
        tenant_map={"acme-uk": "cust-01", "vidyut-in": "cust-02",
                    "northgate": "cust-03", "sarvam-labs": "cust-04",
                    "ghost-tenant": "cust-99"},
        task_map={"summarise": "summarise-v3"},
    )

    traffic = [
        ("acme-uk", None), ("vidyut-in", None), ("northgate", None),
        ("acme-uk", None), ("sarvam-labs", None), ("vidyut-in", None),
        ("unknown-tenant", "summarise"), ("unknown-tenant", "translate"),
        ("ghost-tenant", None),
    ]
    for tenant, task in traffic:
        name, reason = router.route(tenant, task)
        print(f"{tenant:<16} {str(task):<10} -> {str(name):<14} {reason}")

    print("\nresident:", registry.resident())
    print("engine calls:", backend.events)

Running it with a deliberately undersized resident set of three, against five active tenants, produces exactly the pathology you are trying to avoid in production (the last line is wrapped here for readability):

acme-uk          None       -> cust-01        tenant:cold
vidyut-in        None       -> cust-02        tenant:cold
northgate        None       -> cust-03        tenant:cold
acme-uk          None       -> cust-01        tenant:hit
sarvam-labs      None       -> cust-04        tenant:cold
vidyut-in        None       -> cust-02        tenant:cold
unknown-tenant   summarise  -> summarise-v3   task:cold
unknown-tenant   translate  -> None           base:no-adapter-mapped
ghost-tenant     None       -> None           base:missing-artifact(cust-99)

resident: ['cust-01', 'cust-02', 'summarise-v3']
engine calls: [('load', 'cust-01'), ('load', 'cust-02'), ('load', 'cust-03'),
               ('unload', 'cust-02'), ('load', 'cust-04'), ('unload', 'cust-03'),
               ('load', 'cust-02'), ('unload', 'cust-04'), ('load', 'summarise-v3')]

Note cust-02: loaded, evicted, loaded again within nine requests. That is thrashing, and in production it looks like latency that is fine on average and dreadful for whichever customers fall outside the resident set. cust-01 is pinned and survives every eviction round, which is how you protect your highest-traffic tenant from the long tail. And note the two distinct fallbacks: an unmapped request quietly uses the base model, while one mapped to an adapter missing from the catalogue also falls back but carries a reason code you should alert on, because it means a bad deploy.

Three practical notes on cold starts. Size max_resident from your measured count of tenants active within a window, not your total customer count — the estimator above will usually show you can afford to be generous. Keep adapter artefacts on fast local storage next to the GPU rather than pulling them across a region on every cold load; a cold start that is a disk read is a different animal from an object-store fetch. And pre-warm deliberately: on deploy, load the pinned set before accepting traffic.

One more decision deserves thought rather than a default: whether falling back to the base model is acceptable at all. If the adapter enforces a strict output schema or a regulated tone, unadapted output is not a degraded answer, it is a wrong one. For those tenants the correct fallback is a retryable error that wakes someone, not something plausible that quietly violates a contract. Decide this per tenant and record it next to their routing entry.

The per-tenant quality problem

Here is the failure mode that will actually cost you a customer. You upgrade something — the base checkpoint, the tokenizer, the serving precision, the engine version — and nineteen adapters are unaffected while one falls apart. Your dashboard shows the mean across tenants, the mean barely moves, and the first you hear of it is an escalation.

This is not a monitoring gap you can close with a better chart; it is structural. An average over tenants is dominated by the tenants who did not change, and the whole premise of a per-tenant fleet is that tenants differ. The only defence is a small eval set and a recorded baseline score per adapter, gating deploys on the worst individual adapter rather than the aggregate.

"""per_adapter_gate.py — fail the deploy if ANY single adapter regresses."""

import json
import sys

MAX_DROP = 0.03           # absolute score points a tenant may lose. Tune per product.
NEW_ADAPTER_FLOOR = 0.60  # a brand-new adapter must clear this before it ships.


def score_adapter(generate, cases):
    """cases: [{'input':..., 'expected':...}]. Swap in your own metric."""
    if not cases:
        return None
    hits = 0
    for case in cases:
        prediction = generate(case["input"])
        if prediction.strip().lower() == case["expected"].strip().lower():
            hits += 1
    return hits / len(cases)


def run_gate(generate_for, eval_sets, baselines, max_drop=MAX_DROP):
    """generate_for(adapter_name) -> generate(input_text) -> str
    eval_sets: {adapter_name: [cases]}   baselines: {adapter_name: score}
    Returns (rows, failures)."""
    rows, failures = [], []
    for name in sorted(eval_sets):
        score = score_adapter(generate_for(name), eval_sets[name])
        if score is None:
            rows.append((name, None, None, None, "NO EVAL SET"))
            failures.append(f"{name}: no eval set — every adapter needs one")
            continue
        base = baselines.get(name)
        if base is None:
            ok = score >= NEW_ADAPTER_FLOOR
            rows.append((name, None, score, None,
                         "NEW OK" if ok else "NEW BELOW FLOOR"))
            if not ok:
                failures.append(
                    f"{name}: new adapter scored {score:.3f}, "
                    f"floor is {NEW_ADAPTER_FLOOR:.3f}")
            continue
        delta = score - base
        ok = delta >= -max_drop
        rows.append((name, base, score, delta, "PASS" if ok else "REGRESSED"))
        if not ok:
            failures.append(
                f"{name}: {base:.3f} -> {score:.3f} ({delta:+.3f}), "
                f"limit is -{max_drop:.3f}")
    return rows, failures


def print_table(rows):
    print(f"{'adapter':<16}{'baseline':>10}{'now':>10}{'delta':>10}  status")
    for name, base, score, delta, status in rows:
        b = f"{base:.3f}" if base is not None else "-"
        s = f"{score:.3f}" if score is not None else "-"
        d = f"{delta:+.3f}" if delta is not None else "-"
        print(f"{name:<16}{b:>10}{s:>10}{d:>10}  {status}")


if __name__ == "__main__":
    # --- stand-in wiring so this file runs as-is; replace both with real calls ---
    CANNED = {
        "cust-01": ["yes", "yes", "no", "yes"],      # unchanged
        "cust-02": ["yes", "no", "no", "no"],        # quietly broken
        "cust-03": ["yes", "yes", "no", "yes"],      # brand new, no baseline
    }

    def generate_for(adapter_name):
        answers = iter(CANNED[adapter_name])
        return lambda _input: next(answers)

    eval_sets = {
        name: [
            {"input": "q1", "expected": "yes"},
            {"input": "q2", "expected": "yes"},
            {"input": "q3", "expected": "no"},
            {"input": "q4", "expected": "yes"},
        ]
        for name in CANNED
    }
    baselines = {"cust-01": 1.000, "cust-02": 1.000}
    # ---------------------------------------------------------------------------

    rows, failures = run_gate(generate_for, eval_sets, baselines)
    print_table(rows)

    if failures:
        print("\nFAILED:")
        for line in failures:
            print("  -", line)
        sys.exit(1)

    print("\nAll adapters within tolerance.")
    with open("baselines.json", "w") as fh:
        json.dump({r[0]: r[2] for r in rows}, fh, indent=2)

It prints a per-adapter table and exits non-zero:

adapter           baseline       now     delta  status
cust-01              1.000     1.000    +0.000  PASS
cust-02              1.000     0.500    -0.500  REGRESSED
cust-03                  -     1.000         -  NEW OK

FAILED:
  - cust-02: 1.000 -> 0.500 (-0.500), limit is -0.030

Now notice what an aggregate would have said. The mean of 1.000, 0.500 and 1.000 is 0.833 — on a dashboard, a modest dip you might attribute to noise or to the new tenant bedding in. Meanwhile cust-02 is answering half its questions wrong. Multiply that across a fleet of forty and the aggregate becomes almost perfectly insensitive to any single tenant's collapse.

The eval sets need not be large. Twenty to fifty representative cases per tenant, drawn from their real traffic and known failure modes, will catch a collapse. Building them is the same discipline as any golden set — our guide to evaluation suites covers construction and judging — with one addition specific to this architecture: the eval must run through the serving path, with the adapter selected as a real request would select it. An eval that loads the adapter directly will pass happily while your router sends production traffic to the base model.

Symptom	Likely cause	What to check
One tenant's output format collapses after a routine deploy	Base checkpoint, tokenizer or chat template changed underneath adapters trained against the old one	Pin and compare the base checkpoint digest; diff the chat template; re-run the per-adapter gate against the previous base
Aggregate quality flat, one customer escalating	A single-adapter regression hidden by the mean	Per-adapter scores against per-adapter baselines, never the average
Latency spikes affecting only some tenants	Cold start on an evicted adapter, or artefacts fetched over the network	Registry hit rate per adapter; where artefacts are stored relative to the GPU; pinned set
Steady load and unload churn, throughput sagging for everyone	Resident set smaller than the active working set — thrashing	Load and unload events per minute; count of distinct tenants active per window versus max_resident
Out-of-memory under load although adapters are tiny	KV cache, not adapters	KV bytes per token, maximum sequence length, admission control, KV element type
A tenant reports suddenly generic, off-brand answers	Requests silently falling back to the base model	Fallback reason codes in request logs; whether the mapped adapter is present in the catalogue
Adapter was updated but behaviour did not change	In-place replacement kept the name; a stale replica or cached state still serves the old weights	Adapter digest recorded per request; roll every replica; confirm cache keying includes the adapter

When not to do this

Multi-adapter serving answers one shape of problem: many tenants, each needing genuinely different behaviour, none individually large enough to justify dedicated hardware. Outside that shape it is overhead.

Start with the cheapest question: does the difference between your tenants need weights at all? If what varies is terminology, tone, a few worked examples or an output schema, a system prompt and some retrieval will express it, and you will carry no artefacts, no per-tenant evals and no base-upgrade tax. Twenty adapters is twenty training pipelines you maintain forever. Our decision ladder on whether to fine-tune at all is the place to start, and it is worth revisiting annually, because behaviour that needed an adapter two model generations ago often does not now.

At the other end, some adapters have earned a deployment of their own. If one tenant is most of your traffic, they should not pay the per-request low-rank overhead or share a queue with nineteen others — give them a merged model on dedicated capacity, which also removes them as a noisy neighbour. Where several tenants have converged on nearly the same behaviour, merging their adapters into one artefact means fewer things to serve, evaluate and version; our guide to model merging with SLERP, TIES and DARE covers how, and the catastrophic forgetting playbook covers what you risk losing.

Then there is isolation, which is not a technical question dressed up as a legal one. It is a genuinely different question in different places, for different customers.

Consider a Bengaluru SaaS vendor serving mid-market Indian customers with per-customer tuned models. Their constraint is economic: margins are thin, per-seat pricing is low, and a dedicated GPU per customer would make the product unsellable. Access to subsidised national compute programmes changes the arithmetic again — though subsidised capacity often carries its own placement and reporting conditions, worth reading before you design around it. For them shared serving is not a preference, it is the business model, and the engineering work is making the isolation controls demonstrable.

Now consider a London healthtech firm selling into NHS trusts, or a Manchester insurer selling into regulated finance. Their constraint is contractual. Data protection law in both the UK and India is generally framed around appropriate technical and organisational measures rather than a blanket prohibition on shared compute, so the statute rarely forbids this pattern outright. What forbids it in practice is a clause in a specific customer agreement, a public-sector procurement requirement, or a security questionnaire asking whether customer data is processed on infrastructure shared with other customers and refusing a nuanced answer. When that is the situation, no architecture argument wins. Price a dedicated deployment and move on.

If you do share, the controls that make the case defensible are concrete: tenant-to-adapter mapping strictly server-side, no reuse of cached state across tenants, per-request audit logging of which adapter served which caller, and clear documentation of what is and is not co-resident. Our multi-tenant isolation architecture guide covers the wider pattern, including the retrieval layer, which usually carries more sensitive data than the adapters do.

Situation	Multi-adapter, merge, or separate	Why
Twenty or more low-to-moderate-traffic tenants, one shared base	Serve multi-adapter	Adapters cost megabytes; the GPU is the cost, and this way it is one GPU
One tenant is most of the traffic, the rest share a long tail	Separate deployment for the large one, multi-adapter for the tail	Removes the noisy neighbour and drops per-request overhead where volume makes it matter
Several tenants have converged on near-identical behaviour	Merge into one shared adapter	Fewer artefacts, fewer eval sets, one fewer routing decision
A customer contract or procurement rule forbids shared infrastructure	Separate deployment	Contractual constraint, not a technical one; no architecture argument overrides it
Adapters were trained against different base checkpoints	Separate, until retrained onto one base	A single shared base is the entire premise of the pattern
Differences are expressible in a system prompt or retrieved examples	No adapters at all	No artefacts, no per-tenant evals, no base-upgrade tax
Adapters update continuously from an online or reinforcement loop	Multi-adapter with in-place replacement	Precisely the workload runtime adapter updating exists to serve
A tenant holds a tight, contractual tail-latency commitment	Separate deployment, merged weights	Shared batching couples their tail latency to everyone else's traffic

Operating it: capacity and cost

The capacity plan falls out of the estimator. Take your real model configuration, your real GPU, an honest overhead assumption and the longest sequence you actually allow; that gives you a concurrent-request ceiling. Set admission control below it, set max_resident comfortably above your measured count of tenants active in a window, pin the top few by traffic, and you have a configuration you can defend in a design review rather than one you reached by trial.

The cost arithmetic is where the pattern justifies itself. As of mid-2026, NVIDIA H100 SXM rental spans roughly US$2.50 per hour on specialist GPU providers to US$6.50 or more per hour on major hyperscalers — a spread of more than two and a half times for nominally the same silicon, which is itself the largest single lever most teams have. From those two endpoints, with stated assumptions:

Assumption (as of mid-2026)	Specialist provider	Major hyperscaler
H100 SXM hourly rate	US$2.50	US$6.50
One GPU, running continuously for 30 days	US$1,800	US$4,680
Two GPUs, for redundancy rather than capacity	US$3,600	US$9,360
Cost per tenant per month, 20 tenants across those two GPUs	US$180	US$468
Dedicated GPU per tenant instead, 20 tenants	US$36,000	US$93,600

Every figure there is arithmetic on two rate endpoints and a 30-day month, not a measurement. Substitute your own rates, redundancy factor and tenant count. The shape holds whatever you substitute: a fixed cost divided by your tenant count against a fixed cost multiplied by it. That is why the pattern earns its complexity at twenty tenants and not at two.

Two refinements before you show those numbers to anyone. Committed pricing moves the specialist column further, and a multi-adapter deployment suits commitment unusually well because its capacity requirement is stable — you are not scaling GPU count with customer count. And express the break-even for pulling a tenant out as a duty cycle: if their traffic alone would keep a dedicated GPU meaningfully busy, dedicating one is defensible; below that they are renting idle silicon. If you cannot say what a given tenant costs you, our guide to per-tenant cost attribution is the prerequisite for every decision on this page.

On hardware: adapter artefacts are tied to a base model and a rank, not to a vendor's accelerator, but kernel support for batched multi-adapter serving is not uniform across vendors and generations — verify rather than assume when pricing an alternative in a mixed-vendor GPU fleet, and read prefill-decode disaggregation alongside it if your traffic mixes long prompts with short generations.

The honest summary is that multi-adapter serving is not clever, it is arithmetic. One base model is a fixed cost, adapters are nearly free, the KV cache is the real constraint — and the difficulty lies not in the mechanism but in routing correctly, watching per-tenant quality, and knowing which tenants do not belong in the pool. Get those three right and twenty customers on one GPU is unremarkable.