Cypher-X

Gwen Shapira's argument is not that every AI workload belongs in PostgreSQL. It is that teams should first test how far a mature relational system can take them before adding separate vector, graph, document, and coordination systems. Postgres can combine SQL, text search, JSONB, geospatial extensions, vectors, transactions, and access control around data the application already owns.

Shapira is the co-founder and CPO of Nile, an Apache Kafka committer, and the former head of Confluent's Cloud Native Kafka engineering organization. She presented this 47-minute, 32-second talk at QCon AI New York 2025; InfoQ published the recording and transcript on July 15, 2026.

These notes first report what Shapira presents. A later section labeled Synthesis separates my engineering conclusions from her claims.

What You Will Learn

Why retrieval-augmented generation (RAG) includes more than vector search.
How relational queries provide deterministic context before semantic search adds approximate results.
How pgvector's HNSW behavior changes filtering and index design.
Why vector recall must be measured against exact-search ground truth.
How high-level database tools, transactions, and row locks support agents.
Where read-only credentials, row-level access, and narrow write permissions reduce risk.

Start With the Data a Human Would Need

Shapira uses an imaginary AI-assisted issue tracker throughout the talk. Its LLM assigns priorities to issues, a task for which the issue text alone is insufficient. A human might also consider:

The project's priority.
The number of affected customers.
How many tickets were created that day and the count and priority mix of currently open issues, including P0s.
Whether complaints are increasing relative to the previous week.
The issue's age percentile.
Whether a dependency chain connects it to a critical blocked issue.
How similar issues were prioritized previously.

Her sequence matters: define how a person would solve the problem, retrieve data with deterministic queries, add semantic search, manually inspect what the model receives, and only then package the successful operations as agent tools. She argues that a developer should understand a task before delegating its control loop to an agent.

RAG Is Broader Than Vector Retrieval

In Shapira's terminology, RAG is any application-controlled process that finds relevant information and places it in the model's prompt. That includes an SQL join or aggregate just as much as nearest-neighbor search.

Relational context has a useful debugging property: the same query over the same database state returns the same rows, subject to an explicit ordering when row order matters. SQL can join issue, project, and customer records; compute counts and percentiles; compare time windows; and traverse recursive dependencies. Shapira's practical advice is not to reject a graph database when it is genuinely required, but to account for the extra database, query language, data pipeline, and operations before introducing one.

Postgres expands this relational base with several data forms:

Large text values and full-text search.
JSONB storage, operators, and specialized indexes.
Geospatial data through extensions such as PostGIS.
Vector embeddings and approximate indexes through pgvector.

Postgres can also construct nested JSON in the query. Shapira recommends doing that in the database when JSON is the model-facing contract, avoiding an application-side loop that materializes and reshapes the same rows. She notes that JSON repeats keys and punctuation as tokens and that a complete JSON document cannot be parsed as complete until its closing characters arrive, which complicates response streaming. Incremental JSON parsers can process partial input, so this is a trade-off rather than a literal inability to stream. She also emphasizes that JSON is established and works well with current models. She briefly mentions TOON, a more compact tabular serialization, as something to experiment with rather than a proven replacement in her own systems.

The system prompt contains business policy and the required output contract; the user prompt contains the retrieved issue data and request. The issue text itself is untrusted and may contain instructions intended to alter the model's behavior. Shapira therefore recommends keeping policy in the system prompt and data in the user prompt. This separation improves the instruction hierarchy, but the presentation does not claim it eliminates prompt injection.

Add Semantic Context Deliberately

The issue-prioritization example uses embeddings to retrieve previously prioritized issues with similar meaning. The application embeds the new issue, computes its distance from existing issue embeddings, applies a maximum-distance cutoff, orders by distance, and selects the nearest results.

The cutoff is important. A top-k query always has a nearest item even when the entire corpus is irrelevant; without a threshold, a sparse project could return an unrelated issue simply because nothing else is closer. Shapira also advises using an embedding model suited to the domain and selecting a coherent unit to embed. In Q&A, she says a whole source file may work when it contains one closely related object, while a file containing many unrelated objects should be split into smaller logical units. She treats that boundary as empirical, not universal.

Exact distance calculations become expensive as the corpus grows. Shapira says she has seen the transition become relevant at roughly 3,000 to 10,000 embeddings, but this is an experience-based range, not a general capacity limit. Dimensions, hardware, distance function, latency target, and workload all affect the crossover. At that point pgvector's approximate nearest-neighbor indexes trade perfect retrieval for speed.

Understand HNSW Recall and Filtering

Shapira focuses on Hierarchical Navigable Small World (HNSW) rather than IVFFlat, citing a benchmark by PostgreSQL contributor Jonathan Katz in which HNSW delivered higher query throughput at comparable recall levels. HNSW links vectors into a navigable graph and creates layers: upper layers cover longer distances, while lower layers refine the local neighborhood. She advises keeping the upper layer in memory because repeatedly fetching it undermines throughput.

For approximate search, recall measures how many of the true nearest neighbors the index returns. If exact search says ten items are the nearest and the indexed query returns nine of those, recall is 90%. The benchmark Shapira discusses actually used 100 nearest neighbors per query; ten is her simplified explanation of the metric. Recall is different from request throughput and from an LLM answer's subjective quality.

Filtering has a counterintuitive cost. In the pgvector behavior Shapira describes, an HNSW scan seeking the top ten neighbors initially considers 200 candidates by default, then applies predicates such as project and status. If too few candidates survive, iterative scanning seeks more. A selective filter can therefore make the vector query slower rather than faster. The exact candidate count and scanning behavior are pgvector settings and version-dependent, not properties fixed by the HNSW algorithm itself.

Her Q&A adds two constraints:

PostgreSQL does not combine an HNSW index with a conventional index on the same table in the way it can combine B-tree indexes through bitmap scans. The planner chooses the HNSW path or another index path.
For many large tenants, partitioning can give each partition a separate HNSW index and avoid searching vectors belonging to other tenants. Shapira notes that Nile provides such partitioning behind the scenes; she does not present it as a zero-cost choice for every Postgres deployment.

Quantize, Then Verify the Trade-off

Shapira recommends testing vector quantization. Replacing 32-bit components with 16- or 8-bit representations can reduce memory and storage, reduce the data processed by distance calculations, and reduce network traffic if an application unnecessarily transfers vectors. She illustrates the 8-bit case as one quarter of the size and says this can make everything four times faster, while also giving a hypothetical recall change from 99.99% to 98.5%.

Those figures are an illustration from the presentation, not a benchmark guarantee. Her actual decision rule is more useful: measure whether the recall loss is acceptable for the business before adopting the smaller representation.

Shapira recommends inspecting retrieval before the result reaches an LLM. A fluent generated answer can hide a broken distance operator, metric, or embedding configuration. Her suggested validation loop is:

Choose queries whose expected neighbors are understood by a person.
Check the retrieved context directly, without an LLM rewriting it.
Periodically run exact search without the HNSW index to establish ground truth.
Compare indexed results with that ground truth and track recall as the data distribution changes.

The exact query can run on a read-only replica without the approximate index or during an appropriate low-load period. Shapira says HNSW should generally maintain quality as data is added without rebuilding, but skew may change the result; her posture is "trust and verify."

Postgres also permits hybrid retrieval without another data service. A query can combine vector and text search, use one for candidate generation and the other for ranking, or join vector results to relational tables for enrichment.

Move From Retrieval Code to Agent Tools

Shapira distinguishes the two control loops:

In conventional RAG, application code decides which context to retrieve and puts it into the prompt.
In an agentic workflow, the application describes tools, the model chooses tool calls, the application executes them, and the loop continues until the model returns an answer.

A tool is ultimately a function plus a description and input schema. Model Context Protocol (MCP) servers make tool discovery and invocation reusable, and agent SDKs can manage the call-result loop. This does not remove the need to design the database operation behind a tool.

Shapira prefers higher-level tools such as find_similar_issues and open_issue_count over unrestricted database access. They preserve a stable contract while the SQL and storage design change, and they constrain the space of possible actions. She acknowledges that choosing high- or low-level tools also reflects how much freedom and surprise a team accepts.

Her direct safety rule is unambiguous: if an agent receives database access, make it read-only rather than granting read-write access to production.

Use Transactions for Shared Agent Memory

Some agents need durable notes rather than retrieval alone. In the issue tracker, one agent can record why it selected a priority so that another agent or a person can review the decision later. Concurrent workers must not overwrite one another or process the same issue simultaneously.

Shapira uses Postgres's ACID transactions, multi-version concurrency control, and row locks as a simple work queue. A worker begins a transaction and locks an issue; SKIP LOCKED allows another worker to bypass that row and choose other work. After processing, the first worker writes the priority or notes and commits.

She also identifies the cost: keeping a user-facing issue row locked during a slow model call can block a person trying to edit it. A more mature design can put agent work ownership and notes in a separate table, reducing interference with the main application record. The talk demonstrates that relational coordination primitives are available; it does not claim long transactions are always the right queue architecture.

Constrain What Each Agent Can See and Change

Postgres provides row-level security (RLS) and fine-grained privileges. Shapira says an agent can be restricted to particular rows and, when a write is necessary, to a single column in one table. This complements high-level tools: the function limits intended behavior while database permissions limit the effect of mistakes or unwanted calls.

The presentation's concrete controls are therefore:

Default agent database credentials to read-only.
Prefer narrow, task-level tools to general SQL execution.
Use row-level access where agents or tenants have different data scopes.
Grant only the table and column writes a workflow actually requires.
Keep deterministic retrieval and raw results observable for validation.

Synthesis: An Incremental Production Path

The following is my synthesis of the presentation, not a production case study reported by Shapira.

Prove the context contract first. Write and test the relational query, including tenant and status constraints, before exposing it as a tool. Save representative inputs and raw retrieval outputs.
Establish an exact baseline. Start with exact vector search while the corpus is small. Introduce HNSW only after latency measurements justify the recall trade-off.
Evaluate retrieval separately from generation. Measure neighbor recall and inspect retrieved records before scoring the final LLM response. This separates a retrieval regression from a model regression.
Treat partitioning and quantization as measured optimizations. Tenant size, skew, query filters, memory, and acceptable recall determine whether either helps; neither presentation example is a universal threshold.
Wrap proven retrieval in least-privilege tools. A named operation with a narrow schema is easier to test and authorize than arbitrary SQL. Retain database permissions as a second boundary.
Separate coordination state from user data. If model calls make transactions long-lived, claim work in a dedicated table and keep locks off records people need to edit.
Add infrastructure only when a measured limit appears. Postgres may not remain the best system for every scale or access pattern, but using its existing capabilities first makes the eventual reason to split systems concrete.

Key Takeaways

RAG is a context-selection technique, not a synonym for vector search.
Deterministic relational context is easier to inspect and test than semantic retrieval, so build it first.
Semantic search needs both a distance cutoff and a domain-appropriate embedding strategy; top-k alone does not ensure relevance.
HNSW performance must be considered together with post-filtering, tenant layout, memory, and measured recall.
Quantization can materially reduce resource use, but the presentation's 4x figure is illustrative and must be validated on the actual workload.
Inspect retrieved data before an LLM turns it into a convincing answer.
Agents do not require unrestricted SQL. High-level tools and read-only or narrowly scoped credentials provide safer contracts.
Postgres transactions and locks can coordinate agent work, though slow model calls make lock placement an application-design concern.

Shapira's strongest lesson is methodological: understand and test how the data solves the problem before handing control to an agent. She presents Postgres as a mature, open-source, extensible, multimodal foundation whose SQL, ACID transactions, approximate retrieval, and authorization can be evaluated in one system rather than hidden behind a stack of new services. Her comparison of its reliability with newer vector databases is based on experience, not a benchmark in this presentation.

Canonical reference: Gwen Shapira, Postgres for Production Agents: Your Relational Foundation for Enterprise AI, QCon AI New York 2025, published by InfoQ on July 15, 2026. InfoQ provides the video, transcript, and downloadable slide deck.