GraphRAG vs Vector RAG: When Knowledge Graphs Actually Win

What you need to know

The honest answer to "GraphRAG or vector RAG?" is that they fail in different places, and the skill is knowing which failure your questions will hit. Vector retrieval is the workhorse of production retrieval-augmented generation: you split a corpus into chunks, embed each one, and at query time pull back the chunks whose embeddings sit nearest to your question. It is fast, cheap, easy to build and genuinely excellent at one thing — finding the passage that already contains the answer. For "what is the refund window in our policy?" it is hard to beat, and you should reach for it first.

It breaks on a specific shape of question. When the answer is not sitting inside any single chunk but is distributed across the relationships between many chunks — which of our suppliers also supply our competitor, through which subsidiaries — vector similarity has nothing to grab. There is no one passage that says it; the answer has to be assembled by following a chain. A benchmark using Diffbot's KG-LM evaluation found this dramatically: vector-only retrieval accuracy degrades toward 0% as the number of entities per query climbs past about five, while graph retrieval stays stable with ten or more entities per query. That is not a marginal gap; it is a cliff.

GraphRAG is the answer to that cliff, and it pays for it. Instead of only embedding text, it uses a language model during indexing to read every chunk and extract the entities and the relationships between them, building a knowledge graph it can later traverse. Microsoft's GraphRAG adds a second trick — community summaries — that lets it answer broad, global questions about an entire corpus that no chunk contains. The cost is real: indexing is far dearer because it is many model calls rather than one cheap embedding, you now run a graph store, and you have to keep the graph fresh. This guide sets out exactly when that trade is worth it, gives you a decision matrix and a cost comparison, and lands on the architecture most teams should actually ship — a hybrid that uses vectors for breadth and the graph for depth. Whether you are building over an Indian lending corpus or a UK clinical knowledge base, the maths and the mechanics are the same.

How vector RAG works

The mental model for vector RAG is similarity search over meaning. During indexing you chunk your documents — typically a few hundred tokens each, with some overlap — and pass every chunk through an embedding model that turns it into a fixed-length vector of floats. Semantically similar text lands near in that vector space, so a chunk about "loan default" sits close to one about "missed repayments" even though they share no words. You store those vectors in a vector database with an approximate nearest-neighbour (ANN) index, which is what makes retrieval fast at scale.

At query time you embed the user's question with the same model, then ask the ANN index for the k nearest chunks — usually the top five to twenty. Those chunks become the context you stuff into the prompt, and the language model writes its answer from them. The whole appeal is in that simplicity: there is no reasoning during indexing, just one embedding call per chunk, and retrieval is a single vector lookup. It scales to millions of chunks, it is cheap to run, and a competent engineer can stand up a usable pipeline in an afternoon. How you chunk and embed is the lever that most affects quality here; our guide to RAG chunking and embedding strategies for production goes deep on getting that right.

The limitation is baked into the mechanism. Vector retrieval returns chunks that are individually similar to the query, but it has no notion of how chunks connect to one another. If the answer requires joining a fact in chunk A to a fact in chunk C through an entity mentioned in both, similarity search will not reliably surface both — and it certainly will not surface the bridge that links them, because the bridge is a relationship, not a passage. Add a reranker (covered in our piece on RAG reranking with cross-encoders and hosted rerankers) and you improve which chunks rise to the top, but you do not change the fundamental fact that you are retrieving isolated chunks rather than connected structure.

How GraphRAG works

GraphRAG replaces "find similar chunks" with "build a graph of what the corpus is about, then traverse it". The indexing phase does the heavy lifting. For each chunk, a language model performs entity and relationship extraction: it reads the text and emits structured triples — subject, relation, object — such as (Acme Ltd, supplies, Globex) or (Priya Sharma, reportsTo, Arun Mehta). Those triples are merged into a single knowledge graph where entities are nodes and relationships are typed, directed edges. The same entity mentioned across many chunks collapses to one node, so the graph stitches together facts that were scattered across the corpus.

At query time, GraphRAG identifies the entities in your question, finds their nodes, and traverses the graph outward along edges to gather connected context — not just the chunk that mentions an entity, but the neighbours, and the neighbours' neighbours, up to whatever hop depth you allow. This is why it survives multi-entity questions: following Alice → workedOn → Project ← workedOn → Bob is an explicit graph walk, not a guess at which chunks happened to be similar. The relationships are stored as data, so the model is reasoning over structure rather than hoping similarity surfaced the right passages.

Microsoft's GraphRAG adds the piece that handles global questions. After building the graph it runs community detection to cluster tightly connected entities, then uses a language model to write a hierarchical community summary for each cluster — a short description of what that group of entities and relationships is collectively about. When you ask a corpus-wide question such as "what are the main themes across these reports?", GraphRAG generates a partial answer from each relevant community summary and then combines the partials into a final response. In Microsoft's evaluation on global sense-making questions over datasets in the million-token range, this approach beat a conventional RAG baseline on the comprehensiveness and diversity of answers roughly 70 to 80% of the time. That is a class of question vector RAG essentially cannot serve, because there is no single chunk that summarises a whole corpus.

Where vector RAG wins

Vector RAG is not a compromise you tolerate until you can afford a graph — for a large share of real questions it is simply the correct tool, and reaching for a graph would be over-engineering. It wins decisively on single-hop factual lookup: any question whose answer lives in one passage. "What is our parental-leave policy?", "What did clause 7.2 of this contract say?", "Which dosage is recommended for adults?" — each of these is answered by retrieving the one chunk that contains it, and similarity search does that beautifully. There is no chain to follow, so the graph's traversal buys you nothing.

It also wins on the qualities that matter most in early production: it is fast, cheap and scalable. Indexing is a single embedding call per chunk with no model reasoning, so you can index millions of chunks for a fraction of what graph extraction would cost, and retrieval is a sub-100-millisecond ANN lookup. It is easy to build and operate — the moving parts are an embedding model and a vector store, both of which are mature, well-documented and available as managed services in both Indian and UK cloud regions. And because the index is just embeddings, updates are trivial: a new document means embedding its chunks and upserting them, with no re-reasoning over the corpus. When your questions are predominantly lookups and your priorities are latency, cost and shipping speed, vector RAG is not just acceptable — it is the right call. Our production RAG hybrid retrieval guide covers how to get the most from this path, including pairing dense vectors with keyword search.

Where GraphRAG wins

GraphRAG earns its complexity on the questions vector RAG structurally cannot answer well, and they fall into two families. The first is multi-hop questions — anything that requires joining facts across several entities. "Which of our active suppliers also appear in our competitor's filings?" or "Trace the ownership chain from this fund to the operating company" demand following relationships, and as the entity count climbs the vector approach falls off the cliff described earlier: accuracy degrading toward 0% past roughly five entities per query, while graph retrieval holds steady past ten. The reason is mechanical — the answer is the path, and only a graph stores paths.

The second is global sense-making — questions about a whole corpus rather than a point in it. "What are the recurring risk themes across these 4,000 board minutes?" or "How are these research groups connected by shared authors and topics?" cannot be served by any single chunk, because the answer is a property of the structure. This is exactly what community summaries were built for, and it is where Microsoft reported the 70-to-80% comprehensiveness-and-diversity win over conventional RAG. A clean way to say it: GraphRAG wins when the answer lives in the relationships between chunks, not inside any single chunk.

Two grounded examples make the pattern concrete across our markets. An Indian fintech building a fraud and compliance assistant over transaction records, KYC documents and corporate registries needs to ask "which accounts connect to this flagged entity through shared directors or addresses" — a multi-hop relational query where vector similarity is close to useless and a graph is close to essential. A UK life-sciences team building a research assistant over a decade of internal trial reports needs "what mechanisms have been hypothesised across all our studies for this side effect, and which teams worked on each" — a global sense-making query that only community summaries over a graph can answer comprehensively. In both, the value is in connections the corpus never states in one place.

The honest cost of GraphRAG

It would be dishonest to sell the graph without the bill, because the bill is the reason most teams should not make everything a graph. The dominant cost is indexing. Vector indexing is one cheap embedding per chunk; GraphRAG indexing runs a language model over the entire corpus to extract entities and relationships, then runs it again to summarise every community. That is potentially thousands of reasoning calls where vectors needed none. As of June 2026, the LightRAG paper illustrated the gap starkly: a document set that cost roughly $4 to $7 to index under a traditional GraphRAG pipeline could be processed for about $0.15 with a lighter graph approach — the headline being that classic GraphRAG indexing can run an order of magnitude or more dearer than vector indexing, before you even serve a query.

Beyond indexing there are standing costs. You now operate a graph store — Neo4j or a comparable engine — alongside your vector database, which is another system to provision, secure and pay for in your Indian or UK cloud region. Build complexity rises: extraction prompts need tuning so the schema stays consistent, entity resolution has to merge "Acme Ltd" and "Acme Limited" into one node, and traversal logic has to be written and tested. And there is staleness: when documents change, a naive design re-indexes the whole corpus at full extraction cost, so you need an incremental-update strategy — the kind LightRAG popularised, where new data is graphed locally and merged into the existing graph rather than triggering a global rebuild — or your freshness and your bill will fight each other.

Cost figures as of June 2026. The ~$4–7 versus ~$0.15 indexing comparison is from the LightRAG paper's reported figures for a traditional GraphRAG pipeline versus a lighter graph approach on a given document set; absolute numbers depend on corpus size, model tier and token counts, so treat them as an order-of-magnitude signal rather than a quote for your workload.

The decision matrix

Most of the choice collapses to two questions: how many entities does a typical query touch, and is it asking about a point or the whole corpus? The matrix below maps the common shapes to a recommended method. Read it as a default, not a law — your real query distribution, audited from logs, should be what decides, and many teams land on hybrid precisely because their traffic is mixed.

The 2026 recommendation: go hybrid

For most production systems in 2026 the right answer is neither pure approach but a hybrid: vectors for breadth, graph for depth. The reason is that real traffic is mixed — a stream of cheap single-hop lookups with a minority of expensive relational and global questions — and you want to serve each at the cost it deserves rather than overpaying for traversal on every query or failing the cliff on the hard ones.

The architecture is straightforward in shape. At query time, run a fast, cheap vector recall to shortlist the most relevant chunks and, with them, the entities those chunks mention. Then use a graph traversal to assemble connected context: take the shortlisted entities as entry points into the knowledge graph and walk outward to gather their neighbours and relationships, so the final context contains not just similar passages but the structure linking them. Vectors give you a good, broad starting set quickly; the graph turns that set into connected, multi-hop context the model can reason over. Neo4j makes a similar case for combining the implicit relationships uncovered by vectors with the explicit, factual relationships made visible by graphs — and that combination is the whole point.

The refinement that controls cost is a router that picks per query. A lightweight classifier — even a small, cheap model or a few heuristics — labels each incoming question as single-hop-factual, multi-hop, or global, and routes accordingly: vector-only for the lookups, vector-then-graph for the multi-hop, and the community-summary global path for corpus-wide questions. Now you only pay for traversal when the question actually needs it, which keeps the common case fast and cheap while still serving the hard minority correctly. That routing pattern is worth building deliberately; our guide to adaptive RAG query routing — naive, hybrid and agentic walks through how to classify queries and wire the branches, and once it is live you will want the observability to see which path each query took and where it failed, covered in hybrid retrieval and agent observability for production RAG.

The code: extract triples, then retrieve hybrid

Two snippets carry the whole idea. The first is the indexing primitive that makes a graph possible: using a language model to extract entities and relationships from a chunk as structured triples. The second is the query-time hybrid retrieve — a vector shortlist that seeds a graph query to expand neighbours. Both are deliberately framework-light so the mechanics are visible; in production you would use your graph driver and vector client of choice. Code stays in US English, as is conventional.

Snippet 1 — LLM-based entity and relationship extraction

During indexing, you pass each chunk to a model and ask it to return triples in a fixed JSON schema. A meaningful, consistent schema is what lets you merge the same entity across chunks later, so be strict about it. The example uses a small, cheap model tier, which is sufficient for extraction and keeps the per-chunk indexing cost down.

import json
from anthropic import Anthropic

client = Anthropic()

EXTRACT_SYSTEM = """You extract a knowledge graph from text.
Return ONLY JSON: a list of triples. Each triple is an object with
keys "subject", "relation", "object". Use canonical entity names
(e.g. "Acme Ltd", never "the company"). Relations are lowercase
verbs or short phrases (e.g. "supplies", "reports_to", "owns").
If no relationships are present, return []."""

def extract_triples(chunk_text: str) -> list[dict]:
    msg = client.messages.create(
        model="claude-haiku",          # cheapest sufficient tier for extraction
        max_tokens=1024,
        system=EXTRACT_SYSTEM,
        messages=[{"role": "user", "content": chunk_text}],
    )
    raw = msg.content[0].text.strip()
    try:
        triples = json.loads(raw)
    except json.JSONDecodeError:
        return []                      # skip un-parseable output; log for review
    # Keep only well-formed triples.
    return [
        t for t in triples
        if isinstance(t, dict)
        and {"subject", "relation", "object"} <= set(t)
    ]

# Example: build the graph by extracting from every chunk, then
# merging triples into nodes/edges (entity resolution collapses
# "Acme Ltd" and "Acme Limited" to one node before insert).
for chunk in all_chunks:
    for t in extract_triples(chunk.text):
        upsert_edge(
            subject=normalize(t["subject"]),
            relation=t["relation"],
            obj=normalize(t["object"]),
            source_chunk_id=chunk.id,   # keep provenance back to the text
        )

The load-bearing parts: the system prompt pins a strict triple schema so output is mergeable; normalize() (your entity-resolution step) is what stops the same real-world entity becoming three nodes; and keeping source_chunk_id on every edge preserves provenance, so when the graph leads you to an entity you can still pull the original passage as evidence. Treat extraction output as untrusted — a malformed line should be skipped and logged, never allowed to break the whole index.

Snippet 2 — hybrid retrieve: vector shortlist, then graph expand

At query time you first get a cheap vector shortlist of relevant chunks and the entities they mention, then run a Cypher (Neo4j-style) traversal from those entities to pull in their neighbours and the relationships connecting them. The assembled context is the union of the similar passages and the connected structure around them.

def hybrid_retrieve(question: str, k: int = 8, hops: int = 2) -> dict:
    # 1. Vector shortlist — fast, cheap breadth.
    q_vec = embed(question)
    top_chunks = vector_store.search(q_vec, k=k)          # ANN nearest chunks
    seed_entities = entities_mentioned_in(top_chunks)     # e.g. ["Acme Ltd", ...]

    # 2. Graph expand — depth. Walk out from the seed entities to
    #    gather neighbours and the relationships linking them.
    cypher = """
    MATCH (e:Entity)
    WHERE e.name IN $seeds
    MATCH path = (e)-[r*1..$hops]-(neighbor:Entity)
    RETURN e.name      AS source,
           neighbor.name AS target,
           [rel IN relationships(path) | type(rel)] AS chain
    LIMIT 200
    """
    graph_rows = graph_db.run(
        cypher, seeds=seed_entities, hops=hops
    )

    # 3. Assemble context: similar passages + connected structure.
    return {
        "chunks": [c.text for c in top_chunks],   # breadth (vector)
        "relations": [                            # depth (graph)
            f'{row["source"]} -{">".join(row["chain"])}-> {row["target"]}'
            for row in graph_rows
        ],
    }

# The prompt then gets BOTH: the retrieved passages for grounding
# and the relation chains so the model can reason over connections
# no single chunk states.

Conclusion and next steps

The decision is less a religious war than a routing problem. Vector RAG is the correct, cheap default for the single-hop lookups that make up most real traffic, and you should not apologise for using it where it fits. GraphRAG is the answer to a specific, well-defined failure of vector retrieval — multi-hop questions that span many entities, and global questions about a whole corpus — and the evidence for the gap is concrete: accuracy collapsing past roughly five entities per query for vectors while graphs hold steady past ten, and a 70-to-80% comprehensiveness win for community summaries on global sense-making. The catch is the cost, which is real and front-loaded in indexing, so you reach for the graph only where your audited queries prove you need it.

The pragmatic 2026 answer is hybrid: vectors for breadth, graph for depth, and a router that pays for traversal only when the question demands it. Start this week by labelling a sample of your real queries by entity count and locality, measure how big the relational-and-global slice actually is, and let that number decide whether you build the graph at all. If you are the engineer who diagnosed that your retrieval was failing on multi-hop questions and shipped a hybrid that fixed it without inflating the bill, that is exactly the measurable, hard-won work the people hiring in AI want to see — and a Verified Builder profile on AI Tech Connect is where you put it in front of them.

Sources

• Microsoft Research — From Local to Global: A GraphRAG Approach to Query-Focused Summarization (arXiv 2404.16130)
• Microsoft Research — GraphRAG: New tool for complex data discovery (community summaries)
• FalkorDB — GraphRAG vs Vector RAG accuracy benchmark (Diffbot KG-LM evaluation)
• LightRAG: Simple and Fast Retrieval-Augmented Generation (arXiv 2410.05779) — indexing cost and incremental updates
• Neo4j — Knowledge graph vs vector RAG: benchmarking and the hybrid case