Nördnytt! 🤓Security Disclosure
- vintagedave - 7798 sekunder sedanIt's 5:20 AM here in Europe and I just rotated a bunch of keys.
Only to read,
> Because deployments are paused, you will not be able to redeploy your services to pick up the new credentials until builds are restored. We will post here when builds resume with any additional details.
So now my service does not work. Which is worse: a stolen key or a broken product? I'm genuinely regretting following their advice on rotating keys, because logs showed no sign of incorrect usage, and our service would at least have remained online.
I hope that they automatically redeploy their customer services. I just worry that I won't have tested it. I mean: 5AM. Keys changed. Nothing more I can do. Should go back to sleep. No timeline to wake up to check deployments, so... I... wait?
- vldszn - 19324 sekunder sedanCrazy day for cybersecurity: PHP RCE, eMudhra certificates, and now Railway.
And it’s all just getting started imo.
- theturtletalks - 22366 sekunder sedanPostiz founder posted on X that his service is being brought down, his keys were already used and Railway was quiet about this until he spoke up:
- piazz - 9157 sekunder sedanYup, got my OpenRouter key. They managed to run up a whole $2 on Opus 5.5 and Fable. Well, time for a new hosting provider I guess...
Incidentally, I'm testing Muse as an email triage agent today - it buried "oh also, railway and anthropic say yours keys have been leaked and you need to rotate asap" under 50 routine deploy / admin emails notifications. Oh to live through such an AI powered software renaissance!
- fbrncci - 20957 sekunder sedanRight just as I take my first day off (for the entire year!) from work to fly out for a vacation. The mother of all security bombs starts dropping all around stacks and services that I am responsible for. How else could have taken time off go for me…
- MattIPv4 - 22795 sekunder sedanSeen multiple folks in the Railway Discord confirming that their secrets stored in Railway have been used by third parties, so it definitely sounds like the attackers were able to read decrypted environment variables.
- martinald - 21988 sekunder sedanI have a bad gut feeling that every day is going to look a bit like today with endless urgent patches and breaches.
Fwiw it sounds similar to the hack over the weekend that opencode suffered.
- alpineman - 21734 sekunder sedanTrain wreck. People were writing on discord 3 hours ago, 10 hours after the incident started, that keys created just for railway were running up usage. Meanwhile builds have been down for more than 12 hours. Only NOW did they just send out an email to all customers. I wouldn't have found out until then if I wasn't checking the discord periodically.
- nosefrog - 1311 sekunder sedanRailway was not honest about the reason for their last big outage.
- averrous - 7660 sekunder sedanAnyone can help?
I changed he api key in the env var as openrouter revoke the previous one based on this incident. However, as the redeploy cannot be done in my region, the env cannot be updated and now the product cannot be used. Any command in railway to fix this?
- fyzix - 20917 sekunder sedanHow is it possible rotate keys if you disable deployments? You're basically telling us to take our services offline...
- DrawTR - 22345 sekunder sedanJeez, what a rough past 24 hours for all security folk. Hang in there
- AJRF - 21732 sekunder sedanDo cloud providers have a lock-down mode where you can press 1 button and stop all spend?
- - 19489 sekunder sedan
- jasona123 - 5627 sekunder sedansighs
Thankfully I'm only using Railway for a hobby project (aptly, for a real-time transit dashboard) but still
- minraws - 22113 sekunder sedanhow are the secrets decrypted, I would like some explanation so that I can have some understanding here, I am starting to feel like all these companies are just meme-ing with their security posture.
Like did someone vibe code too hard, how do env secrets leak for everyone...
- 1-6 - 18012 sekunder sedanIt's time for competent programmers to get rehired to help patch massive exploits that are about to unfold before us.
- paxy - 22162 sekunder sedanIt’s funny that every security breach disclosure now has a mandatory “the attacker used AI!” as if it absolves you from having to maintain a secure system. If anything it makes the situation more damning. If it was possible to find exploits using AI then your own red teaming should have found them before the attacker did.
- cowlby - 9823 sekunder sedanThis is so fucked up. I was reviewing my account and I just saw this in the logs... Claude says database credentials and bucket credentials go in via env vars so likely they had access to database and bucket contents.
Audit Logs Oct 21, 2026 05:54:13 PM CDT BucketInstanceCredential.regenerated in my-project production by Railway Staff Oct 21, 2026 05:54:13 PM CDT BucketInstanceCredential.regenerated in my-project production by Railway Staff Oct 21, 2026 05:54:13 PM CDT BucketInstanceCredential.regenerated in my-project production by Railway Staff - tstrebitzer - 5431 sekunder sedanmeanwhile on railway.com:
Ship software peacefully
- dylanz - 16048 sekunder sedanI'm rotating production secrets all night... great. I confirmed that they've actually used many of my secret keys already.
- arbayi - 18357 sekunder sedani revoked my api keys, changed with new ones but guess what? railway does not allow deployments at the moment, so my app is down.
- gbnwl - 22106 sekunder sedanThe past two days have been awful as someone who uses Railway.
- eecks - 21143 sekunder sedanHuh, I've been using Railway tonight and this was the first I've seen of this. Just got an email about it now too at 23:36
- MattIPv4 - 22691 sekunder sedan
- sergiotapia - 21132 sekunder sedanIf you're on Railway do not wait, ROTATE every API key at the source, and do not set new values because Railway is still compromised and not secured.
- ksajadi - 13778 sekunder sedansomeone managed to get the environment variables AND the encryption keys for those? i feel for the secops engineers over there right now. brutal
- wateralien - 22159 sekunder sedanIs this our new AI daily reality?
- shinycode - 19512 sekunder sedanI’m lucky I moved away from it to host my own db with coolify
- rpmisms - 6118 sekunder sedanAw shit. I just got done patching our PHP servers, now I gotta do the Railway too? I swear I'm gonna wake up to a Vercel SNAFU.
- AndrewKemendo - 8930 sekunder sedanYeah I’m not sure what happened but there was a huge bot swarm the last few days
I had an open gitea host pulled into some botnet
Luckily digitaloceam sent an alert.
- sergiotapia - 18347 sekunder sedanActually I will move my projects away from Railway. With AI what it is today, no reason to shy away from hosting on actual VPS.
What is the best VPS to use today? Is it still hetzner or ovh?
- barelysapient - 14905 sekunder sedanUgh. Love Railway.
- lijok - 21968 sekunder sedanDamn I guess all those l33t hiring practices were for nothing
- Keloran - 23007 sekunder sedanWell this is certainly interesting, I wonder how their secrets were being stored, I assumed they were stored in vault and then injected with a sidecar, but I guess not
- goforbg - 18545 sekunder sedanthe worst service I've signed up for has to be railway. What a terrible product.
- esseph - 22137 sekunder sedanFuck, I thought this was about trains
- dayyan - 17465 sekunder sedan[flagged]
- jon128 - 14799 sekunder sedan[dead]
- torben-friis - 18857 sekunder sedan>We believe this is part of a series of attacks recently targeting platform companies with the aim of compromising AI inference providers.
Is that an accusation? As in, major AI players taking out competitors ?
- visch - 13888 sekunder sedanThis is exactly why we should own our deployment stacks. Going to push up the launch here as pdt is a railway alternative thttps://github.com/autoidm/pdt for scheduled jobs
AI makes this so much easier to do now, we should all own our own code, and deployment infrastructure (use a cloud instead of someone wrapping aws for you)