Graph RAG augments retrieval-augmented generation with structured knowledge graph traversal. The promise: questions whose answers span multiple documents can be answered by following graph relationships, not just by vector similarity.
This page covers what graph RAG is, when it helps, and the costs.
Works well when:
Fails when:
Build a knowledge graph from your corpus:
Graph queries can:
For multi-hop questions, graph traversal can find answers vector retrieval misses.
Extract entities from query; traverse graph; pass results to LLM.
Issues: brittle to entity extraction errors; misses passages without entities.
Vector retrieval for breadth; graph for relationships.
Combine results before LLM call.
LLM agent uses graph queries as a tool. Decides when to traverse vs read.
Most flexible; highest cost.
Build hierarchical community summaries from graph. Use community summaries for global questions.
Specifically targets "what are the major themes" type queries.
Identify entities mentioned in documents.
Tools:
Identify relationships between entities.
LLMs are good at this with appropriate prompts.
Two extremes:
Most production systems land in the middle.
Choice depends on query patterns and existing infrastructure.
Cypher (Neo4j), SPARQL (RDF), Gremlin (TinkerPop).
Powerful but require schema knowledge.
LLM translates question to graph query language.
Quality depends on schema clarity and example coverage.
"Find paths from X to Y." Useful for knowledge exploration.
PageRank for importance, community detection for clusters, shortest path for relationships.
"Who collaborates with researchers at company X?" "What are the consequences of decision Y?"
Vector retrieval misses these.
Multiple entities with same name. Graph context resolves.
"How many papers cite this work in the last year?"
"Where did this claim originate?"
Connecting facts across documents.
Most question-answering doesn't need graphs:
For these, graph RAG adds complexity without quality.
Building a knowledge graph from documents:
Often as much work as the rest of the system.
Graph queries can be slow for complex traversals.
Now you have two retrieval systems to maintain.
Build standard RAG. Measure quality.
If quality is good enough, you don't need graph.
Identify question types where standard RAG fails.
Build graph features specifically for those.
For each query, do both vector retrieval and graph queries; combine.
Often the best balance.
Schema documentation matters. LLMs querying the graph need it.
Anthropic's open-source GraphRAG builds:
For "global" questions about a corpus:
For "local" questions:
Effective for "summarize the main themes in this corpus" use cases.
Hard. Standard RAG evaluation (precision/recall on retrieved passages) doesn't capture graph value.
Curate eval sets that test relational reasoning.
Bad entity extraction → useless graph.
Schema doesn't match evolving content.
Graph RAG when standard RAG would suffice.
Query returns empty; agent makes up answer.
Graph stale relative to documents.
LLM extraction creates entities that aren't in source.
Build graph RAG when:
Skip graph RAG when:
Graph RAG is evolving:
It's promising for complex domains; not always needed.