Ransomware
IDCF Cloud ransomware: what 495 customers should do now
Ransomware stopped four zones of IDC Frontier's IDCF Cloud on October 7. What is confirmed, what is not, and what cloud customers should do next.
At about 3:40 a.m. Japan time on October 7, virtual servers in IDC Frontier's IDCF Cloud stopped working. IDC Frontier, a SoftBank group company, first called it unauthorized access by a third party, then confirmed the same evening, in its second report and to ITmedia, that it was a ransomware attack. Its third report on October 8 named the damage: every virtual server in the tesla, henry, pascal and joule zones of East Japan Region 1, hosted at its Shirakawa data center in Fukushima, is stopped and cannot be restarted. The company counts 495 affected companies and local governments and is contacting each one individually.
The line customers need to read twice is that data in those four zones is "expected to be difficult to retrieve or restore", and that for now the only recovery path IDC Frontier offers is the customer's own backups. Downstream, websites of Ibaraki Prefecture, Kodaira City in Tokyo, Tobu Zoo and Jiji Press went offline, and JR East and the card issuer ViewCard have warned that email data held by an outside mail delivery service on IDCF Cloud may have been exposed. If you run anything on IDCF Cloud, or buy a service from a vendor that does, this affects you now.
What is confirmed and what is not
On the record, IDC Frontier says the cause is ransomware deployed by a third party, and the damage is limited, as far as it knows, to four of the six zones in East Japan Region 1. The radian and newton zones, East Japan Regions 2 and 3 and West Japan Region 1 have no confirmed intrusion. IDCF Cloud TypeS (formerly White Cloud ASPIRE) and IDCF Private Cloud are outside the incident.
Three points are still open:
- How the attacker got in. IDC Frontier says it is still identifying and blocking the intrusion route with an outside security firm reviewing networks, servers and storage in both East and West Japan.
- Whether data was stolen. The company has not said either way. It cut the region off the network to prevent "secondary damage and data leakage", which is a precaution, not a finding.
- Who did it. Images claiming that data had been encrypted circulated on social media on October 7. IDC Frontier said it was aware of them and was checking whether they were genuine. No group has been confirmed.
The JR East and ViewCard notices are also worded as possibilities. JR East said it cannot rule out that email logs were viewed or taken: up to about 1.67 million Eki-net email addresses and up to about 390,000 Otona no Kyujitsu Club records, which also include membership numbers, card expiry dates and birth dates. ViewCard said about 4.03 million email addresses may be involved. JR East said names, addresses, phone numbers and card numbers were not involved.
How a provider-level attack differs
On a public cloud the provider runs the hypervisor (the layer that runs every customer's virtual machines), the storage arrays behind the virtual disks and the management console. Customers harden what runs inside their own machines. Ransomware that reaches the provider's layer can stop or encrypt virtual machines from underneath, where patching and endpoint agents inside the guest do not reach. IDC Frontier has not said which layer was hit; whole zones going down at once points below the individual customer, but that is our inference.
Three consequences follow for customers. First, the console is gone: IDC Frontier switched off the customer management console in every region, and its staff start and stop virtual servers on request instead. Second, copies kept inside the same platform share the platform's fate. The company's statement that only customer-held backups can be relied on suggests that provider-side copies in the affected zones are not a recovery route for now. Third, customers inherit the provider's incident: their own data may have been on the affected servers, and their own notification duties start whether or not the provider has finished investigating.
What to do if you use IDCF Cloud
1. Work out your exposure, including through suppliers
List every virtual machine and its zone. If any sit in tesla, henry, pascal or joule, treat their data as unavailable and possibly accessed. Then ask your SaaS and outsourcing vendors directly whether they host on IDCF Cloud: mail delivery, CMS hosting and e-commerce platforms were among the services that went down, and many vendors posted outage notices without naming the provider.
2. Rebuild elsewhere, from your own backups
IDC Frontier advises customers in the four zones to rebuild in a separate environment rather than wait for the old one, and SoftBank's corporate division is now the customer contact for backup and migration help. Restore from a backup taken before 3:40 a.m. on October 7, onto freshly built servers from clean images or infrastructure-as-code templates. Do not boot old disk images you cannot vouch for. Before cutting traffic over, lower the time to live (TTL) on your DNS records so you can move again quickly if needed.
3. If you are in an unaffected zone, take a copy out today
IDC Frontier has asked customers in radian, newton and the other regions to back up their own data and says it will announce how. With the console offline, use what you control from inside the server: database dumps, file sync or agent-based backups to storage at a different provider.
4. Rotate every secret those servers held
Assume anything stored on an affected virtual machine is known to someone else. That covers SSH keys and the matching authorized_keys entries elsewhere, database and application passwords, API keys for payment, mail and SMS providers, TLS private keys (reissue the certificates), cloud access keys for other platforms, and any service account those machines used to reach your office network or identity provider. Regenerate your IDCF Cloud API keys once the console is back.
5. Look for misuse where the logs survived
Logs on the affected servers may never come back, so check the other end: sign-in logs of your identity provider and SaaS tools for use of credentials those servers held, mail provider logs for sending you did not do, and database or API audit logs for access from unfamiliar addresses since early October. If your service sends mail to customers through a provider on IDCF Cloud, warn customers to expect phishing that uses real order or membership details.
6. Ask IDC Frontier for specifics in writing
Request the time window of attacker activity, whether your virtual disks or volumes were accessed or copied, any indicators of compromise (IP addresses, file hashes, tool names) found by its investigators, and when the console and other regions will be declared safe. You need these answers for your own report to Japan's Personal Information Protection Commission and to your customers, so agree with your legal team now on what you will notify and when.
The wider lesson
A cloud provider can be as unavailable as an on-premises server room, and when it is, the only backups that count are the ones the provider cannot reach. Keep at least one copy of important data and the build scripts for your servers at a second provider, under credentials the first provider never sees, and rehearse a full restore there at least once a year. Keep a list of which suppliers host where, so a provider incident tells you within an hour who is affected.
Our platform and cloud engineering team builds environments that can be rebuilt on a second provider from code and offsite backups, and our security operations work covers incident response when a supplier is hit. Open the chat and Yaali, our AI agent, will pass your question to an engineer.
Sources: IDC Frontier third report, IDC Frontier fourth report, IDC Frontier second report, Jiji Press via Nippon.com, ITmedia NEWS, ASCII.jp, Mynavi TECH+, Impress Watch, piyolog, TRAICY Global on ViewCard, TRAICY Global on JR East.