MEntA: membership inference on RAG corpora in five entailment queries

What is this?

On May 23, 2026 (revised May 31, accepted to USENIX Security 2026), Nguyen Linh Bao Nguyen, Wanlun Ma, Viet Vo, Alsharif Abuadbba, Minghong Fang, Jun Zhang and Yang Xiang published “Five Queries Are Enough: Query-Efficient and Surrogate-Free Membership Inference Attacks on RAG via Entailment” (arXiv:2605.24312, cs.CR).

The target is membership inference against Retrieval-Augmented Generation (RAG): not stealing a document’s contents, but answering the prior question — is this specific document in the retrieval corpus at all? For an enterprise assistant grounded on internal data, that yes/no is itself sensitive. Confirming that a particular contract, patient record, résumé, or unpublished report is “in the index” leaks who a company works with, who its customers are, or what it holds, before a single line of the document is exfiltrated.

Membership inference on RAG is not new — earlier work such as “Generating Is Believing” (arXiv:2406.19234) and “Is My Data in Your Retrieval Database?” (arXiv:2405.20446) established the threat in 2024. What this paper adds is practicality: the attack is cheap, quiet, and defense-agnostic.

How it works

The method, MEntA (Membership Entailment Attack), drops two assumptions that made older attacks easy to spot or expensive to run.

Older RAG MIAs                         MEntA
-------------------------------------  -------------------------------------
Templated probes ("Is the following    Broad, natural information-seeking
document in your data? ...")           questions that read as normal traffic
Shadow / surrogate models to           No surrogate model needed
calibrate a score                      (surrogate-free)
Many repeated queries per target       ~5 queries per candidate document
Detectable by query filters            Detectors miss it, or flag benign
                                       users at high rates

Instead of asking the system about a document directly, the attacker asks ordinary, broad questions and then uses natural-language inference (NLI) to measure how strongly the model’s answers entail the candidate document. If the document was retrieved and used to ground the response, the answer contains claims that follow from it; entailment runs high. If it was not in the corpus, the answer and the document diverge. Membership is read off the entailment signal, maximizing information per query rather than brute-forcing many probes.

The reported numbers are the headline. Across the NFCorpus, SCIDOCS and TREC-COVID retrieval sets, MEntA reaches up to 0.991 AUC with only 5 queries, beating prior methods by up to 0.42 AUC under matched conditions, and cuts total attack cost by up to 65×. Crucially, it stays effective under state-of-the-art RAG defenses, while existing detectors either miss it or raise so many false positives on legitimate users that they are impractical to deploy. No payloads or attack code are reproduced here — this is a summary of a published, peer-reviewed method.

Why it matters

RAG is now the default way to ground an LLM on private data, which is exactly why this result lands. The privacy boundary most teams reason about is “can someone read the document?” — protected by access control on the source store. Membership inference attacks a different boundary: the model’s behaviour leaks corpus composition even when no content is returned verbatim.

Three properties make MEntA operationally relevant rather than academic. It is low-budget (five queries is within any normal usage quota), stealthy (non-templated questions look like ordinary use), and defense-agnostic (it held up against the defenses the authors tested). That combination means rate limits and naive prompt filters — the usual first line — do not reliably stop it. The caveat: this is benchmark research on public retrieval sets, not a reported in-the-wild incident, and the attacker still needs query access to the RAG endpoint and a list of candidate documents to test.

Defenses

1. Treat corpus membership as sensitive metadata. Decide explicitly which collections are so sensitive that even confirming a document’s presence is a disclosure, and segregate those behind stricter controls or separate, authenticated endpoints rather than a shared assistant.

2. Add calibrated noise at the right layer. Differential-privacy-style RAG (DP-RAG) and answer-level perturbation degrade the entailment signal the attack reads. The paper shows current defenses are not sufficient on their own, so treat noise as one layer, not the fix — and measure the privacy/utility trade-off on your own data.

3. Limit and monitor per-principal query patterns. Because the attack needs only a handful of broad questions per target, hard volume limits help little. Watch instead for systematic enumeration — many distinct, document-shaped probes from one principal — and require authentication so queries are attributable.

4. Minimize and partition the corpus. Do not index documents the assistant does not need. Scope retrieval to the requesting user’s authorization so a query can only ever match documents that principal is allowed to see, shrinking the set an attacker can probe.

5. Constrain grounded answers. Abstaining when retrieval confidence is low, summarizing rather than quoting, and avoiding answers that closely track a single source all reduce how much a response entails any one document.

6. Red-team for membership leakage, not just extraction. Add membership-inference tests (entailment-based, low-query) to your RAG evaluation alongside content-exfiltration and poisoning tests. A pipeline that blocks verbatim leakage can still leak membership.

Status

Item	Reference	Date	Notes
MEntA paper	arXiv:2605.24312	2026-05-23 (rev 05-31)	Accepted, USENIX Security 2026
Result	up to 0.991 AUC / 5 queries	—	NFCorpus, SCIDOCS, TREC-COVID; surrogate-free
Cost	up to 65× cheaper	—	vs. prior SOTA MIAs, matched setting
Prior art	arXiv:2406.19234, arXiv:2405.20446	2024	Established MIA-on-RAG feasibility
Real-world status	—	—	Benchmark research; no in-the-wild incident reported

The takeaway is not that RAG is unsafe to use. It is that grounding a model on private data creates a privacy channel separate from document access — and membership, not just content, has to be in your threat model.