Hybrid BM25 + vector retrieval cut gradient-guided RAG poisoning from 38% to 0%

What is this?

On March 10, 2026, an independent researcher published Semantic Chameleon: Corpus-Dependent Poisoning Attacks and Defenses in RAG Systems (arXiv:2603.18034). The headline is a defensive one: a simple hybrid retriever that combines classic BM25 keyword search with dense vector search reduced gradient-guided RAG poisoning from 38% to 0% co-retrieval across an n = 50 evaluation, without modifying the LLM, the system prompt, or the retrieval ranking logic.

RAG poisoning sits in the OWASP LLM Top 10 as a vector-and-embedding weakness. An attacker who can inject documents into a knowledge base — through user-generated content, API submissions, or a compromised data pipeline — can plant text that gets retrieved for specific target queries and steers the model’s answer, all without ever touching model weights. The paper’s contribution is to show that the retrieval architecture itself is a powerful, cheap, immediately deployable control against the gradient-optimized version of this attack.

How it works

The attack the paper defends against is a dual-document (sleeper–trigger) poisoning optimized with Greedy Coordinate Gradient (GCG) — the same family of coordinate search that produces universal jailbreak suffixes. The optimization maximizes cosine similarity in embedding space so the poisoned pair gets co-retrieved for a chosen query while staying dormant for everything else. On a pure vector retriever over Security Stack Exchange (67,941 documents), this reached 38.0% co-retrieval.

The defense is to stop relying on embedding similarity alone:

Retrieval mode                        Gradient-guided attack success
------------------------------------  ------------------------------------------
Pure dense (vector only)              38.0%  co-retrieval
Hybrid BM25 + vector (alpha 0.3-0.7)  0%     — attack class eliminated
Hybrid, attacker also optimizes BM25  20-44% — partially circumvented, bar raised

The reason it works: a GCG-optimized embedding vector is tuned for the dense channel. BM25 scores lexical overlap, which the optimized payload does not control, so the poisoned document fails to surface once sparse signal is fused in. The paper recommends a mixing weight of alpha ≤ 0.5 (favoring or balancing the sparse channel) as a first-line default. It is honest about the limit: when the attacker jointly optimizes for both sparse and dense channels, hybrid retrieval is partially circumvented (20–44% success). Hybrid retrieval raises the bar a lot; it is not impenetrable.

Two further findings matter for defenders. First, susceptibility is model-dependent: across five LLMs the end-to-end attack ranged from 46.7% (GPT-5.3) to 93.3% (Llama 4), with safety-violation rates from 6.7% (Claude Sonnet 4.6) to 93.3% (Llama 4) — model safety training helps but does not close the gap. Second, the attack is corpus-dependent: a technical corpus absorbed attack vocabulary and enabled stealth, while a general corpus (FEVER Wikipedia) exposed the same payloads as vocabulary anomalies, with detection difficulty varying 13–62× between corpora. The same payload does not transfer across knowledge bases.

Why it matters

Most RAG defenses proposed to date are detection-and-filtering layers bolted on after retrieval, and prior surveys show they leave residual risk. This result points the other way: a change you probably already have the components for — sparse + dense fusion is standard for retrieval quality — doubles as a security control against the gradient-guided attack class. It is the rare mitigation that costs little and is deployable today.

It also reframes how to reason about RAG risk. Susceptibility is not a single number for your stack; it is the product of your retriever architecture, your model’s safety training, and your corpus composition. A defense validated on one corpus may not hold on another, so “we tested it on benchmark X” is not evidence it holds on your production knowledge base. This is the same lesson as SilentRetrieval’s demonstration that fluent poisons evade perplexity filters: retrieval-layer threats demand retrieval-layer and corpus-aware thinking.

Defenses

The paper’s results map onto a layered, immediately actionable posture.

1. Deploy hybrid retrieval as a default. Fuse BM25 with your dense retriever (e.g. via reciprocal rank fusion) and bias toward the sparse channel — the paper recommends mixing weight alpha ≤ 0.5. This eliminated the single-channel gradient-guided attack entirely in testing.

2. Do not treat hybrid retrieval as the perimeter. Joint sparse + dense optimization still reached 20–44%. Layer corpus monitoring and query-pattern detection on top; assume a determined attacker who knows your retriever can still get through.

3. Evaluate susceptibility per model. A 47-point spread between model families means model choice is a security decision for RAG, not just a quality one. Measure your model’s behavior under retrieved-but-poisoned context before shipping.

4. Make ingestion a trust boundary. Gate user-generated content, API submissions, and third-party feeds before they enter the corpus. Persistent poison that activates only on target queries is precisely the case manual review misses.

5. Use corpus-aware detection. Because detectability varies 13–62× by corpus, tune anomaly detection to your domain. Vocabulary-anomaly scoring works on general corpora but is weak on technical ones, where stealth is higher.

6. Add provenance and traceback. Track which documents drove an answer so a poisoned record can be located and purged — complementary to graph-based defenses like Argus.

Status

Item	Reference	Date	Notes
Semantic Chameleon preprint	arXiv:2603.18034	2026-03-10	Corpus-dependent RAG poisoning attacks and defenses
Primary result	§6.2	2026-03-10	Hybrid BM25+vector: 38% → 0% on n=50 (Security Stack Exchange, 67,941 docs)
Defense bypass	§6.2.2	2026-03-10	Joint sparse+dense optimization: 20–44% success
Multi-model end-to-end	§6.3	2026-03-10	46.7% (GPT-5.3) to 93.3% (Llama 4) attack success
Artifacts (defense code + scripts)	GitHub	2026	scthornton/semantic-chameleon
OWASP context	LLM Top 10 (2025)	2025	Vector and embedding weaknesses are a top-10 risk

The finding is not that RAG poisoning is solved — joint-channel attacks remain viable and model-level susceptibility stays high. It is that a standard retrieval-quality technique is also a cheap, deployable-today defense against the gradient-guided poisoning class, and that any honest RAG threat model has to account for retriever architecture, model choice, and corpus composition together.