Yaamlabs

Vulnerabilities

NVIDIA DCGM Exporter flaw lets anyone crash GPU monitoring

CVE-2026-47483 lets an unauthenticated attacker exhaust NVIDIA DCGM Exporter through Go's profiler. Lava found 2,100 exposed GPU hosts. Fixes and checks.

NVIDIA DCGM Exporter, the service that publishes GPU temperature, utilization, memory and error counters for Prometheus to collect, has a high severity flaw, CVE-2026-47483, rated CVSS 8.2. Builds that serve Go's /debug/pprof/ profiling endpoints let anyone who can reach the exporter, with no account, flood it with requests until it runs out of memory and crashes. NVIDIA lists every platform as affected. The fix is in the container release tagged 4.5.3-4.8.2, which turns profiling off by default.

The reason this matters now is exposure. Researchers at Lava, who reported the bug, published scan data on October 8 showing about 2,100 hosts serving DCGM Exporter metrics to the internet on its usual port, TCP 9400, with no authentication. About a quarter of them also served the profiling endpoints, which puts hundreds of GPU servers within reach of the crash. If you run GPU nodes in a cloud, a colocation rack or a Kubernetes cluster, check today whether port 9400 answers from outside.

How it works

DCGM Exporter is a small Go program. It reads counters from NVIDIA's Data Center GPU Manager (DCGM), the library that talks to the GPU driver, and serves them as plain text at http://<host>:9400/metrics for a Prometheus server to scrape. That endpoint has no authentication or TLS by default, which is normal for Prometheus exporters and safe only when the port stays on a private network.

Go ships a profiler, net/http/pprof, that adds diagnostic pages under /debug/pprof/ to a program's web server: heap and goroutine dumps, the process command line, and CPU and execution traces. Some of these handlers take a duration from the caller (in Go's standard package, the seconds parameter on /debug/pprof/profile and /debug/pprof/trace) and keep the request open, collecting data, for that long. Lava has not said which handlers it used, only that some hold a request open for a caller-chosen time.

On vulnerable builds those pages were served on the same port as /metrics, and many concurrent requests to them make the exporter's memory climb until it crashes. NVIDIA classes it as uncontrolled resource consumption (CWE-770), with denial of service and information disclosure as the impact; the disclosure is whatever pprof reveals about the running process.

When the exporter dies, alerts for GPU overheating, Xid errors (the error codes NVIDIA's driver reports) and memory faults stop firing, and Lava notes that the CPU and RAM pressure lands on the same host that runs training or inference jobs, which can slow them down.

What attackers are doing

There are no reports of exploitation in the wild. Lava tested the resource exhaustion against NVIDIA's unmodified official container in its own environment, and not against anyone's public deployment.

The exposure figures come from four Shodan scans between March and May 2026. Lava counted about 2,100 hosts (it also gives "more than 2,000") answering on the metrics endpoint, reporting more than 12,000 unique GPU IDs that it values at about $100 million. The hosts belonged to nearly 300 organizations. By GPU count, the United States had 5,274 (44%), Romania 2,054 (17%) and China 1,967 (16%).

Consumer and mining rigs made up 35% of exposed GPUs, neoclouds 25%, hosting and colocation 19%, telecoms and ISPs 10%, hyperscalers 6% and universities 5%. About half were data center parts such as H100 and H200, including 312 B200 and 32 B300 cards, and the rest mostly RTX 4090 and 5090. In every scan about 60% of the GPUs showed 0% utilization, which suggests a lot of idle capacity sitting on the open internet.

Even without the crash, the metrics page gives away GPU model, UUID, utilization, power and error events. Lava also found 12,096 public Node Exporter hosts reporting metrics for NVIDIA (Mellanox) InfiniBand and RoCE adapters, revealing server models, firmware and network hardware. At Voltage Park it counted 71 public DCGM Exporter hosts and 672 Node Exporter hosts; the provider said all the DCGM instances were customer-deployed and helped contact those customers.

What to do

1. Upgrade to a fixed image

NVIDIA's bulletin, published July 28, 2026, names DCGM 4.5.3 and DCGM Exporter 4.8.2 as the fixed versions. DCGM Exporter images are tagged with both numbers, so the first fixed image is 4.5.3-4.8.2, released in May, whose release notes say it made the /debug/pprof endpoints opt-in through the --enable-pprof flag or the DCGM_EXPORTER_ENABLE_PPROF environment variable. The bulletin lists 4.8.2 in both the affected and fixed columns, and Mend's database gives 4.6.0-4.8.3 as the fixed release. That July release "strengthened" the pprof opt-in and socket validation, so move to 4.6.0-4.8.3 or a newer tag where you can.

To find what you run in Kubernetes:

kubectl get pods -A -o jsonpath='{range .items[*]}{.metadata.namespace}{" "}{.metadata.name}{" "}{.spec.containers[*].image}{"\n"}{end}' | grep dcgm-exporter

On plain Docker hosts, docker ps --format '{{.Names}} {{.Image}}' | grep dcgm-exporter does the same. If you upgrade through NVIDIA's GPU Operator or the dcgm-exporter Helm chart, check that the image tag it pins is a fixed one.

After upgrading, make sure nothing turns profiling back on: search your manifests and values files for enable-pprof and DCGM_EXPORTER_ENABLE_PPROF. NVIDIA's current command reference says --enable-pprof now only works together with --web-config-file, a Prometheus exporter-toolkit file that puts TLS or authentication in front of the root, health, metrics and pprof endpoints. Use it with profiling off too, so the metrics need a login.

2. If you cannot upgrade today

Take port 9400 off the internet. In cloud security groups or the host firewall, allow TCP 9400 only from your Prometheus servers, and apply the same rule to Node Exporter (TCP 9100) and Prometheus itself (TCP 9090). In Kubernetes, a NetworkPolicy on the exporter pods that admits only the monitoring namespace does the job; also check for hostNetwork: true or a hostPort, which publish the port on the node's own address. DCGM Exporter can also bind to one address with -a or --address (environment variable DCGM_EXPORTER_LISTEN, default :9400, which means every interface), so set it to the node's private address.

3. Check whether you were exposed or hit

From outside your network, run curl -s -o /dev/null -w '%{http_code}\n' http://<public-ip>:9400/debug/pprof/ against each GPU host's public address. A 200 means the profiler is reachable; a timeout means the port is filtered. Lava's write-up gives no indicators of compromise, so look for the crash itself:

  • In Kubernetes, look at the restart count of dcgm-exporter pods, then kubectl describe pod for Last State: Terminated with Reason: OOMKilled.
  • On Docker, docker inspect --format '{{.State.OOMKilled}} {{.RestartCount}}' <container>.
  • In Prometheus, gaps where up was 0 for the dcgm-exporter job that no maintenance explains.
  • In load balancer, reverse proxy or VPC flow logs, requests to /debug/pprof/ or bursts of connections to port 9400 from addresses outside your monitoring range.

Nothing published about this flaw involves persistence or credential theft, so cleanup is short: upgrade, close the port and review what the exposed /debug/pprof/cmdline and heap data could have shown about your environment.

The wider lesson

Prometheus exporters are built for a trusted network: plain HTTP, no login, everything on one port. Lava's numbers show how often GPU fleets end up on public addresses with those defaults intact, and a profiler riding along on the same port turned an information leak into a way to knock monitoring over. Put 9100, 9400 and 9090 in the same external scan and firewall review as SSH and the Kubernetes API, and alert when one of them answers from the internet.

Our cloud and Kubernetes security reviews look at how GPU and monitoring workloads are exposed inside and outside the cluster, and our attack surface management work flags ports like 9400 when they appear on the internet. Open the chat and Yaali, our AI agent, will pass your question to an engineer.


Sources: NVIDIA security bulletin 5857, Lava research, NVIDIA dcgm-exporter releases, Help Net Security, The Register, Mend vulnerability database, CERT.UG, NVD.

Back to the blog, or read this post on the full site.