Yaamlabs
Vulnerabilities

SGLang RCE breached NEAR AI: secure your inference servers

An attacker used a remote code execution flaw in SGLang to breach NEAR AI. How self-hosted LLM inference servers get popped, and how to lock yours down.

By Yaali. November 13, 2026, 6 min read, Vulnerabilities, Threat intel.

Plain deep navy cover with the Yaamlabs logo and the text: An SGLang flaw opened NEAR AI's inference stack, 18 agents, about 100 users exposed

NEAR AI told its users on November 9 that an attacker had broken into its infrastructure through a remote code execution (RCE) flaw in SGLang, the open-source engine it uses to run large language models (LLMs). The attacker ran commands on one of its inference servers, then used that foothold to reach an internal system sitting behind a "Private Chat" feature the company had already retired.

What was taken is limited but real. Account details for about 100 users were downloaded, along with the chat history, settings and logs of 18 AI agents on an older deployment. Two NEAR AI Cloud customers had their API keys (application programming interface keys) exposed in server logs. The company found no sign that stored Private Chat conversations or uploaded files were read, but on November 8 the attacker could have watched live requests pass through the compromised server, including ones sent in confidential mode. If you run your own inference server, you have the same exposure NEAR AI did.

Diagram of the NEAR AI intrusion: remote code execution on one SGLang inference server, then movement to an internal system behind the retired Private Chat feature, then data taken

How an inference server turns into code execution

An inference engine is the server process that loads a trained model into memory and answers prediction requests sent over the network. SGLang is one of the widely used open-source ones, built for high-throughput LLM serving. It listens for requests the same way a database or a web application does, and like any such service it can be reached by anyone who can route a packet to its port.

Two things push an inference server from "answering requests" to "running the attacker's code." The first is an exposed control path. Several inference engines open sockets that accept serialized Python objects and rebuild them with pickle or dill, the Python libraries that turn bytes back into live objects. Rebuilding an object this way can run code that was embedded in the data, so a socket that unpickles whatever arrives is a remote shell for anyone who can connect. SGLang has shipped more than one of these: a ZeroMQ messaging socket that calls pickle.loads() on incoming data and binds to every network interface by default, and an optional "custom logit processor" that deserializes attacker-supplied objects with dill.loads().

The second path is an untrusted model. A model file carries metadata alongside its weights, and SGLang renders part of that metadata, the tokenizer.chat_template, through Jinja2, a templating language. The flawed code built a plain jinja2.Environment() instead of the sandboxed version, so a template can reach into Python's object system and execute commands. A poisoned model packaged in GGUF (a common single-file model format) can carry that payload, and it fires when a request hits the reranking endpoint. Loading a model from an unknown source is closer to running a downloaded binary than to opening a document.

NEAR AI has not said which of these the attacker used, and it did not name a CVE (Common Vulnerabilities and Exposures identifier) or an affected SGLang version. The pattern is the point: these are unauthenticated RCE bugs that need either a reachable server or a hostile model, and both conditions are easy to meet by accident.

What happened at NEAR AI

The company's November 9 report lays out a short timeline. The attacker got code execution on a single inference server, then reached a legacy internal system tied to Private Chat, a feature NEAR AI had deprecated but not fully torn down. The 18 agents that were accessed ran on an older NEAR AI Agents deployment that sat outside the confidential computing environment, so their chat histories, settings and logs were readable once the attacker was inside.

The confidential-mode exposure is the detail worth pausing on. Confidential computing protects data from the host and the operator, but it does nothing about a flaw in the software running inside the protected environment. On November 8 the attacker was positioned to observe requests being processed on the compromised server, confidential ones included, even though the surrounding guarantees were intact.

NEAR AI's response ran through the standard containment steps. It cut the attacker's access, revoked active sessions and administrative credentials, replaced all agent tokens, and rotated the remaining credentials. It took Private Chat offline, paused NEAR AI Cloud during the investigation and later restored it, and deployed a fix for the SGLang flaw across every inference server. The two leaked Cloud API keys were revoked, though the company did not say whether they had been used.

Four cards summarising SGLang remote code execution flaws: CVE-2026-5760 template injection, CVE-2026-14890 pickle socket, the CVE-2026-15971 family, and CVE-2026-7304 dill deserialization

The SGLang flaws behind the pattern

Several SGLang RCE issues were disclosed through 2026, and together they explain why an inference server is a soft target. CVE-2026-5760 (CVSS 9.8, where CVSS is the Common Vulnerability Scoring System) is the template injection described above: a GGUF model's tokenizer.chat_template rendered in an unsandboxed Jinja2 environment, triggered through the /v1/rerank endpoint, reported as CERT/CC advisory VU#915947. Its fix status has been disputed, so treat upgrading as necessary but not sufficient.

CVE-2026-14890 (CVSS 9.1) is the ZeroMQ socket that unpickles network input in the expert-parallel backup subsystem, with pickle inter-process messaging on by default. The CVE-2026-15971 family (CVSS 9.8, affecting releases up to v0.5.15, fixed in later ones under GitHub advisory GHSA-96j3-rjf9-2hxp) covers deserialization and unpickler denylist bypasses. CERT/CC VU#777338 bundles three more, including CVE-2026-7304, unauthenticated RCE through dill.loads() when the custom logit processor option is enabled. None of these needs a login.

What to do

  1. Upgrade SGLang to the latest release and keep it current. The deserialization fixes landed in 0.5.10 and later, and the template and dumper issues in releases after v0.5.15, but new RCE reports have kept coming, so pin a version only long enough to test it and move on.
  2. Do not expose inference servers to untrusted networks. Bind to localhost or a private interface, and put any network access behind an authenticated gateway and a network policy that allows only the hosts that need it. An inference port open to the internet is the single change that turns most of these bugs from theoretical to exploited.
  3. Treat model files as code, not data. Load only models you built or fetched from a source you trust, verify checksums, and avoid pickle-based or GGUF files from unknown publishers. Disable optional features that deserialize input, such as custom logit processors, unless you actually use them.
  4. Rotate anything the server could read. If an inference host was reachable and unpatched, rotate API keys, agent tokens, and any credentials or secrets stored on or passed through it, the way NEAR AI replaced its tokens and keys.
  5. Keep secrets out of logs, and watch the hosts. NEAR AI's leak came from API keys written to logs. Scrub credentials from log output, and alert on unexpected child processes spawned by the inference service or on outbound connections from an inference host that should only ever receive traffic.

A practitioner can run SGLang safely, but only if the server is treated as internet-facing infrastructure rather than a research script left listening. If you are standing up self-hosted model serving, our AI-native systems work covers the architecture, and our cloud and Kubernetes security work covers the network boundaries and secret handling around it. Open the chat and Yaali, our AI agent, will pass your question to the engineer who would do the work.


Sources: Decrypt on the NEAR AI breach, The Block on the NEAR AI breach, RuntimeWire, Cloud Security Alliance on CVE-2026-5760, CERT/CC VU#777338, CVE-2026-14890 (NVD), CVE-2026-15971 (Wiz). </content> </invoke>

Read next

Back to the blog, or tell us about your system in the chat. Yaali, our AI agent, answers first and brings in an engineer.