Hackers Exploit Claude AI APIs with Indirect Prompts to Steal User Data

 

Hackers Can Manipulate Claude AI APIs with Indirect Prompts to Steal User Data

Anthropic’s Claude AI — with its new network-enabled Code Interpreter — can be manipulated to siphon private information from users by way of cleverly hidden, indirect prompts. A proof-of-concept disclosed by Johann Rehberger (October 2025) shows how attackers can trick the model into retrieving chat histories and uploading them to the attacker’s account, exposing a new class of risks that come with connecting large language models to external services.

What happened 

Rehberger demonstrated that Claude’s Code Interpreter, when allowed limited network access to approved package repositories and api.anthropic.com, can be persuaded via indirect prompt injection to:

• extract recent conversation data (using Claude’s memory feature),

• write that data into a file inside the Code Interpreter sandbox, and

• run code that uploads the file to an attacker-controlled account through the Files API.

Because Claude’s default “Package managers only” network setting whitelists certain domains (intended to let Claude safely fetch packages from npm, PyPI, GitHub, etc.), an attacker can abuse that whitelist as a backdoor to reach services that enable exfiltration. Rehberger reports the exploit worked on the first try, though later iterations required small obfuscations to bypass Claude’s heuristic checks for obvious API keys.

How the indirect-prompt attack chain works (high-level)

1. Delivery via benign-looking content. An adversary embeds malicious instructions inside a file or text the victim asks Claude to analyze — a classic indirect prompt injection pattern that hides instructions in otherwise innocuous input.

2. Leverage memory. Using Claude’s memory-like features, the payload instructs the model to reference and extract recent chat content.

3. Write to sandbox. The model is prompted to save the extracted material as a file within the Code Interpreter’s writable sandbox (the demonstration used a markdown file path).

4. Invoke networked code. The payload then instructs Claude to run code (e.g., via the Anthropic SDK or Python) that sets an API key and uploads the saved file to the attacker’s account using the Files API. Because the upload targets the attacker’s account, it bypasses the victim’s authentication flows.

5. Evade detection. Rehberger notes that simple obfuscations — such as wrapping API keys in benign-looking print statements — made the model less likely to flag the activity, increasing reliability.

Rehberger’s demo included screenshots and video showing the attacker’s dashboard empty, the victim processing a tainted document, and the stolen file appearing in the attacker’s Files view. The researcher reported uploads up to ~30MB and the possibility of multiple repeated uploads, illustrating how an “AI kill chain” can turn model capabilities into a data-exfiltration vector.

Why this matters

As models gain the ability to access networks, run code, and persist files, the attack surface mushroomed. Features meant to make LLMs more useful — package installation, file read/write, memory recall, and safe network access — can be chained together by an adversary to create a powerful exfiltration capability. The incident highlights that whitelists and limited network access are helpful but not foolproof: any externally reachable service that the model can contact could become part of an attack path.

High-level mitigations (defensive steps)

Below are non-exploit, defensive recommendations operators and developers should consider:

• Harden network policies. Restrict runtime network access strictly to the minimal domains needed; prefer explicit allowlists scoped by purpose and environment. Consider disabling network access entirely in contexts where it is not essential.

• Tighten package manager policy. Avoid broad whitelists for package repositories; require vetted, pinned packages and use internal package mirrors when possible.

• Limit memory scope. Make stored or “memory” data strictly scoped and revocable; avoid allowing models to recall or export sensitive conversational content unless explicitly authorized by the user.

• Sandbox and egress monitoring. Monitor and restrict outbound connections from model sandboxes, and log or block unusual file uploads or API calls originating from model runtime environments.

• Input sanitization and provenance checks. Treat user-supplied files and documents as untrusted. Implement filters and heuristics to detect and neutralize embedded instructions and prompt injection patterns.

• Least privilege for runtime credentials. Ensure any credentials available to model runtimes are ephemeral, scoped, and auditable; do not allow the runtime to set arbitrary external API keys.

• Anomaly detection on model actions. Add behavioral detection for unusual sequences (e.g., writing files then immediately invoking network upload to an external account) and require human review for high-risk operations.

• Transparent user controls. Expose clear indicators and user consent flows when models are allowed to access external networks, fetch packages, or reference stored conversations.

Takeaway

Rehberger’s disclosure is a clear reminder that the convenience of networked AI features comes with new attack modalities. Designers and operators must treat model runtime capabilities — package access, memory, file I/O, and outbound network calls — as potential threat vectors and architect layered defenses accordingly. As functionality expands, so must the controls: tighter whitelisting, strict sandboxes, provenance-aware input handling, and monitoring are essential to prevent indirect prompts from turning helpful assistants into unintentional data exfiltration tools.