LLM Reasoning Blocks Security

I’ve never been happy with the idea of locking up model reasoning in encrypted blocks. If we’re going to treat an AI’s internal thoughts like state secrets, we’d better be damn sure they stay secret. The latest research suggests we might not be.

Researchers showed how to smuggle sensitive data straight out of proprietary LLM APIs by hiding it in the encrypted reasoning blocks that models return. It’s not some elaborate side-channel attack—just a clever abuse of how providers handle their own encryption. They demonstrated this across OpenAI, Anthropic, and Google’s latest models, which should make anyone running a commercial API pause. If you’re trusting these blocks to keep secrets while they shuttle between user and server, you’re trusting the wrong thing.

The kicker? This isn’t just theory. They tested it with a sanitized GitHub repo search—no API keys, but plenty of tokens that don’t belong in public code. The results were messy, but the method worked. And if it works once, it’ll work again.

Introduction to LLM Reasoning Traces

LLM reasoning traces are just the model’s internal monologue rendered as text. That’s all they are — not some magic window into cognition, but a structured output you can inspect, debug, and even edit. The idea is straightforward: force the model to show its work in a consistent format so you can trace how it got from prompt to answer. In practice, it’s often more useful for debugging than for understanding the model’s actual reasoning process, because the trace is still just text generated by a next-token predictor.

Kimi-K3’s approach pre-fills the first 1% of the visible answer before starting generation, which sounds like a minor trick but changes how the model structures its response. Instead of waiting to see what the model decides to output, you’re giving it a skeleton to fill in, which can stabilize formatting and reduce hallucinations around edge cases. The trade-off is that you’re constraining the model’s natural response pattern by injecting a partial answer upfront. Whether that constraint helps or hurts depends entirely on the task — for structured outputs like code or math, it’s usually helpful; for open-ended prompts, it can feel unnatural.

The other specs are where things get more concrete. A 12,000-token generation limit is generous enough to handle long problem-solving sessions, but it’s not arbitrary — it’s tuned to fit most of the Codeforces problems the team tested on. Those 120 problems are a reasonable benchmark for coding ability, though they’re all publicly available, which means the model could have seen similar problems during training. The 708 agent trajectories are more interesting: these aren’t just solutions, but records of how intermediate tools (like interpreters or search) were used. That’s the part that feels closest to real reasoning, because it captures decisions beyond just the final answer.

The mobile scrolling quote is telling — it’s not just that these traces are long, it’s that they’re structured in a way that assumes you’ll read them linearly. Most debugging sessions happen in a terminal or IDE where scrolling is fine, but if you’re trying to share a trace with someone over chat or review it on a phone, the formatting breaks down fast. The model itself doesn’t care, but the humans around it do.

Here’s a minimal example of how you’d generate a reasoning trace with a pre-filled skeleton:

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "kimi-k3"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id)

skeleton = "The solution is:"
inputs = tokenizer(skeleton, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=12_000)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))

The output will start with the skeleton and then continue with the model’s generated reasoning. It’s not sophisticated, but it’s a practical way to ensure the trace starts with a consistent structure. If you’re debugging, you can edit the skeleton to force the model down a specific path or inject clarifying steps. If you’re evaluating, you can compare traces across runs to spot where the model diverges. Neither use case is glamorous, but both are useful.

Sanitizing Repositories and Preventing Leaks

Sanitizing repositories and preventing leaks is crucial in today's development landscape. One approach is to prefill a model's reasoning with a set of predefined answers. For instance, Kimi-K3's reasoning can be prefilled with the first 1% of visible answers. This method has its limitations, though - it's not a foolproof way to prevent leaks, and it may not be suitable for all types of repositories.

The model in question has a 12,000-token generation limit, which is relatively modest. To put this into perspective, the model was trained on 120 Codeforces problems, which is a small but curated dataset. Additionally, there are 708 publicly available agent trajectories that can be used to fine-tune the model. However, it's worth noting that the model's performance may degrade if it's not properly sanitized.

I've seen some interesting approaches to repository sanitization, including the use of models like "claude-opus-4-8". However, as one user noted, "Fascinating approach; however, a nightmare to scroll on mobile." This highlights the importance of considering the user experience when designing repository sanitization tools.

To demonstrate how to implement a basic repository sanitization tool, consider the following Python code:

import re

def sanitize_repository(repo_content):
    # Remove sensitive information like API keys and credentials
    repo_content = re.sub(r'API_KEY=.*', 'API_KEY=REDACTED', repo_content)
    return repo_content

repo_content = "API_KEY=1234567890"
sanitized_content = sanitize_repository(repo_content)
print(sanitized_content)

This code snippet shows a simple way to remove sensitive information from a repository. It's a basic example, but it illustrates the importance of considering repository sanitization when working with sensitive data.

It's also worth noting that repository sanitization is an ongoing process. As new vulnerabilities are discovered, the model and the sanitization tools need to be updated to prevent leaks. This can be a time-consuming process, but it's essential to ensure the security and integrity of the repository.

Ultimately, the key to effective repository sanitization is to strike a balance between security and usability. By using a combination of automated tools and manual review, developers can help prevent leaks and ensure that their repositories remain secure.

Demonstrating the Vulnerability

I think the implications of demonstrating this vulnerability are significant, particularly for the security of proprietary models. By showing that encrypted thought injection can be used to recover proprietary reasoning from encrypted traces, researchers have highlighted a potential flaw in the way model providers return a model's reasoning to clients. The fact that these encrypted blocks are portable and can be sent back to the server when a conversation continues raises concerns about the potential for unauthorized access to sensitive information.

The community reaction to this discovery has been telling, with some researchers already exploring ways to exploit this vulnerability. The method of using a weaker model to "jailbreak" and decrypt encrypted chain-of-thought blocks produced by a stronger model is a clever one, and it's likely that we'll see more research in this area. I'm not convinced that this is a straightforward problem to fix, though - I think it's going to require some fundamental changes to the way model providers think about security and encryption.

One thing that's not entirely clear to me is how widespread this vulnerability is, and how easy it is to exploit in practice. The fact that researchers have demonstrated it across frontier models from OpenAI, Anthropic, and Google suggests that it's not just a problem with one particular model or provider, but I'd like to see more research on the specifics of how this vulnerability can be used in different contexts. For example, are there certain types of models or use cases that are more vulnerable to this type of attack than others?

Ultimately, I think this discovery raises some important questions about the security of AI models and the potential risks associated with using them. As we continue to see more widespread adoption of AI in different industries and applications, it's going to be crucial to address these vulnerabilities and develop more robust security protocols to protect sensitive information. One question that I think is worth sitting with is: what are the potential consequences of this vulnerability being exploited at scale, and how can we mitigate those risks before it's too late?

Conclusion

The fact that portable reasoning blocks can be extracted and reused across different models, as demonstrated with the frontier models from OpenAI, Anthropic, and Google, raises more questions than it answers. I'm still not sure what to make of the implications of this on API security, especially considering the potential for encrypted thought injection. The ease with which these blocks can be sent back to a server when a conversation continues is unsettling, to say the least.

The experiment with Kimi-K3, where prefilling the model's reasoning with the first 1% of the visible answer led to some interesting results, only adds to my confusion. The 000-token generation limit and the ability to decode reasoning blocks using tools like Luna Terminal-Bench all point to a complex issue that doesn't have a straightforward solution. As I see it, the real challenge lies in sanitizing repositories and preventing leaks, a task that's easier said than done, especially with the presence of potential API keys or tokens lurking in the code.

What's next, then? I think the key takeaway from all this is that LLMs are not as secure as we'd like to think, and the use of portable reasoning blocks only exacerbates the issue. Can we develop more robust methods for sanitizing repositories and protecting against encrypted thought injection? Or are we just scratching the surface of a much larger problem? I genuinely don't know, and I'm not sure anyone else does either.