The recent security incident at NEAR AI serves as a stark reminder of a dangerous misconception in modern cloud architecture: treating hardware-level isolation as a silver bullet for application security. When an attacker leveraged a remote code execution (RCE) vulnerability in the SGLang inference engine to breach NEAR AI servers, they didn’t just exploit a software bug. They exposed a fundamental gap in how we think about the boundary between runtime safety and hardware encryption.

For platform engineers and AI security teams, this event forces a hard look at the infrastructure stacks powering high-performance language models. SGLang runs inside a Trusted Execution Environment (TEE)—an environment designed to shield sensitive workloads from host-level tampering. Yet, the attacker successfully bypassed these boundaries, compromising data across 18 AI agents, exposing deprecated Private Chat accounts for roughly 100 users, and leaking cloud API keys.

Let’s break down how this happened, why the TEE illusion failed to stop it, and what it means for the future of AI infrastructure security.

Anatomy of the SGLang RCE Vulnerability

To understand the breach, we first need to look at the engine at the center of it. SGLang is a high-performance inference serving framework designed to accelerate Large Language Model (LLM) execution through advanced optimizations like RadixAttention and structured generation. Like many modern inference engines, it bridges the gap between raw hardware compute and user-facing APIs, handling complex concurrent requests, KV-cache management, and dynamic token generation at blistering speeds.

As the tech industry moves aggressively toward efficient AI architectures to cope with surging demand—a trend we’ve analyzed previously in our coverage on how the tech industry moves towards efficient ai—speed often takes precedence over hardening.

The core of the incident involved an RCE flaw within the SGLang inference engine running on NEAR AI servers. While the exact mechanics of an RCE in an LLM runtime often involve unsafe deserialization, improper handling of complex prompt payloads, or memory corruption bugs in underlying parsing logic, the end result is devastatingly simple: it allows an unauthenticated or low-privilege remote entity to execute arbitrary code within the context of the running application.

Vector Type Traditional Web App RCE AI Inference Engine RCE (SGLang)
Primary Input HTTP JSON payloads, form data, URL parameters Prompt text, structured generation tokens, API config flags
Execution Context Application server (Node.js, Python/Django, Go) GPU/CPU compute runtime managing heavy tensor operations
Privilege Level Varies (often restricted via containerization) High, often co-located with high-bandwidth memory and model weights
Target Data User sessions, databases, environment secrets Active inference streams, decrypted prompts, API keys in memory

Unlike traditional web applications where RCE might give an attacker access to a web worker node, an RCE inside an AI inference engine places the adversary directly inside the memory space where high-value tensor operations, active user prompts, and execution contexts live.

Piercing the Shield: Why TEEs Failed to Stop the Breach

One of the most eye-opening aspects of the NEAR AI incident is that the vulnerable SGLang instance was running inside a Trusted Execution Environment (TEE). For many architects, “running in a TEE” (such as AMD SEV or Intel SGX) is treated as the ultimate defensive posture. It implies that even if the cloud provider is compromised, or the hypervisor is malicious, the workload’s memory remains encrypted and completely isolated.

To understand why this security layer failed to prevent the breach, we must distinguish between confidentiality at rest/transit and application-level logic integrity:

TEEs protect memory from external inspection and unauthorized physical or hypervisor-level access. They do not validate the security posture of the software running inside them.

When an application contains a remote code execution vulnerability, the security guarantees of a TEE crumble from the inside out. Consider the execution flow:

  1. Encrypted Execution: The TEE hardware decrypts instructions and data inside the secure enclave CPU boundary just-in-time for processing.
  2. The Vulnerable Entrypoint: An incoming inference request hits the SGLang server carrying a payload engineered to trigger the RCE flaw.
  3. Internal Execution: Because the code is executing legitimately within the application runtime process, the CPU treats the attacker’s injected instructions with the full privileges of the SGLang process.
  4. Boundary Bypass: The attacker doesn’t need to break the TEE encryption from the outside. By achieving code execution inside the enclave, they inherit access to the decrypted memory space, the processing pipeline, and any connected local resources.

This architectural reality mirrors broader infrastructure challenges where hardware constraints dictate software behavior, a dynamic we frequently observe when examining how engineering teams optimize around compute limitations, such as those discussed in our analysis of DeepSeek’s strategy for engineering AI compute constraints. When systems are pushed to their limits for performance, security boundaries often get conflated with hardware features.

The Blast Radius: What Was Exposed at NEAR AI

When the SGLang instance was compromised, the attacker gained a foothold within the NEAR AI inference server. The resulting blast radius highlights how tightly coupled modern AI runtimes are with sensitive application data and credentials.

The incident report confirmed three distinct categories of exposure:

  • AI Agent Data: Data associated with 18 AI agents operating on an older deployment was compromised. These agents relied on the inference infrastructure to process state and generate responses, exposing historical or active operational data.
  • Deprecated User Accounts: Account details and logs for roughly 100 users of a deprecated Private Chat product were accessed. The fact that this was an older, legacy feature underscores how forgotten endpoints often become the weakest link in an infrastructure footprint.
  • Leaked API Keys: Two customers’ NEAR AI Cloud API keys were inadvertently exposed in server logs and subsequently leveraged or dumped during the attack. NEAR AI acted swiftly to revoke these keys, but the incident underscores how easily operational secrets bleed into standard application logging pipelines.

The severity of this impact demonstrates that securing the model weights is no longer enough. The infrastructure serving the model is a fully functional computing node that handles state, authentication tokens, and user inputs—making it an attractive target for full-scale lateral movement.

Securing the AI Inference Pipeline: Best Practices for Developers

Preventing similar breaches requires shifting our defensive posture away from hardware-as-a-cure-all and toward a rigorous, defense-in-depth engineering approach for AI runtimes. Here is how development and security teams can harden their inference stacks:

1. Strict Input Validation and Sanitization

Inference engines traditionally treat incoming prompts as raw strings meant to be tokenized and vectorized. However, as these engines incorporate complex parsing logic, structured generation constraints (like JSON schema enforcement), and tool-calling capabilities, they act as complex parsers.

  • Implement strict schema validation on all incoming API requests before they reach the core inference loop.
  • Reject payloads that exceed expected token complexity or utilize unexpected formatting flags.

2. Dependency Pinning and Software Supply Chain Audits

High-performance inference stacks like SGLang rely on rapidly evolving dependency trees, including custom CUDA kernels, tensor libraries, and fast HTTP servers.

  • Pin every dependency down to specific cryptographic hashes.
  • Establish automated Software Bill of Materials (SBOM) generation and vulnerability scanning pipelines tailored to Python and C++ extensions used in ML runtimes.

3. Network Segmentation and Zero-Trust Inference Nodes

The SGLang server should never have direct access to administrative APIs, cloud control planes, or unrotated customer keys.

  • Principle of Least Privilege: Run inference containers with minimal user privileges, dropping all unnecessary Linux capabilities (CAP_SYS_ADMIN, etc.).
  • Isolate Logging: Ensure that API keys, authorization tokens, and raw prompt payloads are strictly filtered out of application logs and stdout streams to prevent credential leakage.
# Example: Hardened Kubernetes Pod Security Context for Inference Runtimes
apiVersion: v1
kind: Pod
metadata:
  name: secured-sglang-inference
spec:
  securityContext:
    runAsNonRoot: true
    runAsUser: 10001
    fsGroup: 10001
  containers:
  - name: sglang-engine
    image: ai-registry.internal/sglang:v0.4.x-hardened
    securityContext:
      allowPrivilegeEscalation: false
      readOnlyRootFilesystem: true
      capabilities:
        drop:
          - ALL
    resources:
      limits:
        nvidia.com/gpu: "8"

Future Outlook: The Evolution of AI Runtime Security

The SGLang breach at NEAR AI is a watershed moment for AI inference security. As artificial intelligence systems transition from experimental demos to critical enterprise infrastructure, the attack surface of the serving layer will face intense scrutiny.

We can expect several structural shifts in the industry over the coming quarters:

  • Stricter Compliance Frameworks: Security auditors will no longer accept TEE deployment as a proxy for application safety. Compliance frameworks will mandate rigorous fuzzing and vulnerability assessments specifically for inference runtimes.
  • Memory-Safe Infrastructure Push: The heavy reliance on memory-unsafe languages (C/C++) in the core data-processing layers of high-performance engines will drive a renewed push toward memory-safe alternatives or strict sandboxing of parsing modules.
  • Decoupled Architecture: Future AI serving architectures will likely decouple the request-handling and token-parsing logic from the raw model execution environment, ensuring that an RCE in an API parser cannot trivially escalate into memory access of the broader runtime.

Hardware isolation is a powerful tool, but it is only as secure as the software running inside it. By acknowledging the limits of the TEE illusion and adopting rigorous, defense-in-depth engineering practices, developers can build AI inference pipelines that are fast, efficient, and resilient against modern remote exploitation.