Threat intel
Fireworks AI breached twice: what downstream teams check
Fireworks AI says a second intrusion through its cloud host reached billing data, datasets and partial model weights. How to check your fourth-party exposure.
On November 9, Fireworks AI, a company that runs artificial intelligence (AI) models for other businesses so they do not have to host the hardware themselves, disclosed a second and wider breach. The update revises a more hopeful account the company gave a month earlier. On October 6 it had detected an intruder using internal credentials to read environment variables in a limited number of customer accounts, and said at the time it saw no sign that models, datasets or personal information had been reached. The second disclosure says that is no longer true for some customers.
You are affected by this even if you have never heard of Fireworks AI, because one of your software-as-a-service tools may send your data through it. Notion confirmed that some chat sessions with its AI assistant may have been exposed, since Notion used Fireworks to run classification and inference jobs. That makes Fireworks a fourth party: not your vendor, but your vendor's vendor, holding your data at one remove where your own contracts and reviews rarely reach.
How the second breach happened
The path this time ran through infrastructure Fireworks does not own. A cloud provider that hosts the company's research-and-development machines had a security incident on October 23. The next day, October 24, the attacker used that access to reach Fireworks' own development systems and run an unauthorized workload, a program the attacker chose to execute inside the environment. Fireworks has not named the cloud provider and has not described the specific weakness that was used.
The two incidents expose the two things most worth protecting in an AI company. The first breach went after environment variables, the key-and-value settings a running process reads at startup, which is where teams commonly stash secrets such as application programming interface (API) keys, the long strings that let one service call another on your behalf. A single read of those variables hands over every secret stored there at once. The second breach reached deeper, into incomplete model weights and datasets. Model weights are the trained numbers that are the model itself, and the datasets are the material it was trained or evaluated on. For a model company, those are the product.
What makes a breach at the inference layer wide is that it sits under everyone. Fireworks' direct customers pass it prompts, keys and sometimes training data. Those customers have customers of their own. A compromise there can spill business records, secrets and proprietary models in a single event, and it reaches people several steps removed from the company that was actually hit.
What was exposed, and how the account changed
The November update lists the second incident's exposed data as resource and account metadata (identifiers, display names, descriptions and creation times), billing details including billing and shipping addresses and invoice and payment history, account balances and credit usage, and, for a limited set of accounts, secrets along with a subset of incomplete model weights and datasets. Payment card numbers were not affected, because Fireworks does not store them. The company also says its production inference service, its Training SDK (software development kit, the code libraries customers use to fine-tune models) and its Nexus product show no impact, because none of them keep customer data by default.
Fireworks' chief executive, Lin Qiao, said the attackers did not appear to be after the weights themselves and that most downloads were partial and stopped before completing. The company ties the activity to a financially motivated group involved in LLMjacking, short for large language model jacking, which means stealing access to AI models and reselling it through unofficial gateways so others can run inference on someone else's bill. That framing fits the first breach, where the prize was customer API keys that still work against the real model provider until they are revoked.
The downstream picture is where the fourth-party risk becomes concrete. Notion told affected users that fewer than one percent of its workspaces may have been caught up, through chat transcripts its AI assistant produced for an internal feature. Notion said the transcripts were mostly not linked to a specific user, company or workspace, that its own systems were not breached, and that it has turned off the feature involved and stopped routing open-weight models through Fireworks. It also advised treating anything typed into those chats, including any passwords or API keys pasted in during the window, as exposed.
Fireworks says it rotated credentials, isolated the affected systems from the network, tightened access to its data storage and stepped up monitoring, and brought in Palo Alto Networks' Unit 42, an incident-response team, to work the forensics.
Dates in the coverage vary slightly. The first incident was disclosed around October 9, with customer notices going out over October 7 to 9; some write-ups say October 10. The second update is dated November 9, which some reports refer to simply as Monday. The ranges above hold across sources.
What to do about your AI vendors
The useful response here is not about Fireworks. It is about treating any AI or inference provider the way you already treat a payment processor or a hosting company.
- Inventory what each AI vendor holds. List every AI or inference service your organisation or your software-as-a-service tools use, and for each one write down what data and secrets it receives, including anything passed inside prompts. Prompts are a data flow, not a conversation, and they often carry customer records or internal text.
- Rotate any API keys you have shared with an AI vendor. If you hold a Fireworks key, revoke and reissue it now; deleting a secret from a vendor's console does not invalidate the underlying credential at the provider that issued it. Do the same for any key that has sat unrotated for a long time with any AI service.
- Stop putting long-lived secrets in environment variables. A single read of the environment exposes every secret in it, which is exactly what the first breach did. Prefer short-lived, scoped credentials fetched at run time from a secrets manager, so a leaked value expires on its own and covers only one job.
- Ask vendors two specific questions. How do you keep one tenant's data and models isolated from another's, and how is your own cloud provider, your sub-processor, secured and monitored. The second breach came through a sub-processor, so a vendor that cannot answer the second question has a gap you will inherit.
- Map your fourth-party exposure. Work out which of your SaaS tools send your data to an AI provider behind the scenes. Vendor security pages and sub-processor lists name these; the Notion case shows the chain is often invisible until a provider several steps away is breached.
Your vendor's cloud is part of yours
The hard part of this breach is not the intrusion technique, which Fireworks has not even detailed. It is that the exposed data belonged to companies that had no direct relationship with the system that was compromised. When you adopt an AI feature, you adopt its whole supply chain, including the cloud account where its research machines run.
We build and test AI features with permission boundaries and tenant isolation treated as part of the design in our AI-native systems work, and the over-broad roles and leaked tokens that let an attacker move from one cloud account into the data behind it are the first thing we look at in a cloud and Kubernetes security assessment. If you are not sure which of your tools quietly route data to an AI provider, open the chat and Yaali, our AI agent, will pass the question to an engineer.
Sources: The Register: Fireworks AI gets burned twice as customer data exposure widens, MLQ.ai: Fireworks AI told customers of a second, more serious hack, Lavx: Fireworks AI discloses second breach as customer data exposure widens.