Multi-cloud didn't save Railway. One provider's automated suspension still took everything down.
On May 19, Google Cloud's automated systems suspended Railway's production account, and the entire platform went dark for about eight hours for roughly three million users. The part that should make every infrastructure team sit up is not the suspension itself. It is that Railway runs across Google Cloud, AWS, and its own bare-metal servers, and the workloads on AWS and metal went down too, despite still running the whole time. Multi-cloud was supposed to be the insurance policy. It paid out nothing.
I want to walk through why, because the failure here is architectural and quietly common, and most teams who think they are protected have the same hole.
How a billing flag took down three clouds
Google did not delete Railway's servers. It suspended the account as part of an automated action that hit multiple customers at once, with no advance notice and, per Railway, no public explanation of why. The disks and instances were intact. What vanished was access to the control plane that lived on GCP.
That control plane is the brain. It is what tells the edge proxies where every workload lives and how to route a request to it. When it went offline, the proxies kept serving for a short while on cached routing tables. Then the cache expired. After that the proxies had no way to resolve where anything was, so requests to workloads on AWS and on bare metal failed exactly the same as requests to workloads on GCP. The servers were healthy and reachable in principle. Nothing knew how to find them.
Railway said it plainly in the postmortem: "We take full responsibility for the architectural decisions that allowed a single upstream provider action to cascade into a platform-wide outage." That is the right framing. Spreading compute across three providers does not buy you resilience if the one thing that coordinates all three sits inside a single account on a single provider. You did not build three independent systems. You built one system with its compute scattered and its brain in one basket.
The recovery made a second problem visible
Outages rarely fail in one clean line. When Railway started bringing things back, the flood of retried requests tripped GitHub into rate-limiting Railway's OAuth and webhook integrations. So even as the platform came up, users could not log in and builds would not trigger, because the integration layer was now being throttled by a different vendor reacting to the recovery traffic.
This is the webhook-storm pattern, and it is worth naming on its own. A dependency comes back, every client retries at once, the upstream sees a spike that looks like abuse, and it clamps down right when you need it most. If you have ever watched a queue drain into a thundering herd, you know this shape. The first outage was the control plane. The second was self-inflicted by the recovery, and it was invisible until it happened. Disk access came back around 23:54 UTC; full network restoration did not land until 01:30 UTC the next day.
What Railway is changing, and what it tells you
Railway's fixes are the tell. They are demoting GCP to backup-only, extending their database shards across AWS and metal, and redesigning the mesh so a single provider's action cannot cascade. Read that list backwards and you get the diagnosis: the control plane was single-homed, the data had a single-provider center of gravity, and the mesh assumed the coordinating layer would always be reachable. None of those are exotic mistakes. They are the defaults you end up with when you grow on one cloud and bolt the others on later.
If you operate anything with users, the question this incident asks you is not "am I multi-cloud." It is narrower and more useful: if the account holding my control plane gets frozen, what still works? For most teams the honest answer is nothing, and they have never tested it, because the failure mode is not a server dying. It is a provider's fraud-detection or billing system flipping a bit with no human in the loop.
The part nobody owns: who tells you it broke
Here is the piece I keep coming back to, because it is the one I have direct skin in. When your platform's control plane is down, your own dashboards are very likely down with it. The monitoring that lives inside your infrastructure goes dark at the same moment the infrastructure does. You find out your service is unreachable the way Railway's customers did, from users, not from your own tooling.
The only monitoring that survives this class of failure is monitoring that runs somewhere your incident cannot reach. An external check that hits your real endpoints from outside your account, on a schedule, and screams the second they stop answering. A cron job that is supposed to run every hour and reports when it silently does not. These are unglamorous and they are exactly what is still standing when an automated suspension takes your control plane and your status page in one move. I build a small product in this space precisely because I kept watching teams discover outages from their customers, and the fix is almost always the same: put the watcher outside the thing it is watching.
That is the same principle Railway is now applying to its own architecture, one layer down. Do not let the system that is supposed to detect a failure depend on the system that is failing.
A short checklist I would run this week
- Find your single point of coordination. Not your single point of compute, your single point of control. The scheduler, the service mesh control plane, the thing that knows where everything is. Ask where it runs and whose account it runs under.
- Test the account-freeze scenario, not the server-death one. Most resilience drills kill an instance. Almost none simulate the provider revoking access to the whole account. That is the one that happened here.
- Plan for the recovery thundering herd. When you come back, every client retries at once. Add jitter and backoff to your own clients, and assume your integrations may throttle you exactly when you are trying to recover.
- Put one monitor outside everything. If a single external check cannot tell you within a minute that your front door stopped answering, you are relying on your customers to be your alerting system.
Railway handled this well in the end. They were transparent, they took responsibility, and their remediation list is the correct one. But the reason it is worth your seven minutes is that the trap they fell into is the default trap. Compute is easy to spread. Control is the thing you forget is centralized until the day it is the only thing that matters.
Sources
- InfoQ: Google Cloud Suspends Railway's Production Account, Causing Eight-Hour Platform-Wide Outage
- Railway incident postmortem, May 2026 (status.railway.com)