CodeSpear: when grammar-constrained decoding becomes a jailbreak surface

What is this?

On June 10, 2026, Yitong Zhang, Shiteng Lu and Jia Li published Grammar-Constrained Decoding Can Jailbreak LLMs into Generating Malicious Code on arXiv (cs.CR, 2606.11817). The finding is deliberately counterintuitive: a technique adopted to make LLM code generation more reliable can be turned into a jailbreak.

Grammar-Constrained Decoding (GCD) is the mechanism behind structured-output and “valid code only” features. At each step the model would normally sample from its full token distribution; GCD applies a per-token mask that zeroes out any token which would break a supplied grammar, so the output is guaranteed to parse. The paper names the attack CodeSpear and shows that simply applying a benign-looking code grammar constraint can coerce a model into completing a malicious request it would otherwise refuse. It builds on the broader control-plane attack surface described in When Grammar Guides the Attack (arXiv:2503.24191, March 31, 2025), which first framed structured output as an attack plane orthogonal to prompt-based attacks.

How it works

Safety alignment is overwhelmingly trained in the natural-language plane: the model learns to emit a refusal sentence (“I can’t help with that”) when a request is harmful. The refusal is a sequence of tokens. GCD operates one layer down, on which tokens are even permitted to be sampled.

The structural problem is this: when a grammar constraint requires the next tokens to be valid code, the tokens that make up a natural-language refusal are no longer in the allowed set. The model cannot say “I won’t do this” because that string does not satisfy the grammar. Forced to emit something that parses as code, the path of least resistance becomes completing the requested program token by token. The refusal behavior the model learned in the language plane is simply unreachable in the code plane. Reported across 10 LLMs and 4 benchmarks, CodeSpear raises attack success rate by more than 30 percentage points on average over representative jailbreak baselines — without any adversarial prompt wording, the manipulation lives entirely in the decoding constraint.

We are deliberately describing the mechanism, not a runnable grammar: the durable lesson is that a control surface below the prompt can override prompt-level safety.

Why it matters

This is a different shape of risk from input-side attacks like mathematical-encoding jailbreaks or positional slot attacks. Those manipulate what you say to the model. CodeSpear manipulates the decoding machinery, which means it can slip past defenses that only inspect the user prompt. Any product exposing structured-output or “constrained code” features over an API hands callers partial control of that machinery, so the threat is most acute for coding assistants and code-generation endpoints where grammar constraints are a normal, advertised capability. The deeper point is architectural: safety trained in one modality (natural language) does not automatically transfer to another (constrained code), and an attacker who can choose the output grammar can choose the modality.

Defenses

The same paper ships a defense, CodeShield, and its design is the actionable takeaway:

• Align safety in the code modality, not just in language. CodeShield teaches the model to emit honeypot code under a malicious GCD request — output that is semantically harmless (it does not implement the harmful behavior) yet structurally diverse (so it cannot be trivially squeezed out by tightening the grammar). The model stays safe even when the attacker controls the grammar.
• Preserve natural-language refusals where they are still reachable. When no grammar forces code-only output, CodeShield keeps the normal refusal behavior, so benign utility is not sacrificed.
• Treat the decoding constraint as untrusted input. If your platform lets callers supply grammars or JSON schemas, log and constrain them, and do not assume prompt-level guardrails cover this path.
• Run output-side safety classifiers on generated code, independent of the decoding constraint, so a parse-valid-but-malicious completion is still caught after the fact.

CodeShield is reported to restore safety under CodeSpear across the 10 models tested while preserving benign code-generation utility.

Status

CodeSpear and CodeShield were published as an arXiv preprint on June 10, 2026; the underlying control-plane attack surface was first documented in March 2025. The authors frame GCD’s safety implications as an open, under-examined risk rather than a single patched bug — structured-output features are widespread, and the defensive answer is a modality-aware alignment change, not a one-line fix.