Hetzner under large ddos attack
In this technical retrospective (2013), @alternize shares practical architectural lessons learned from running, scaling, or migrating production software under lean constraints.
HN engineers debated the tradeoffs between cloud convenience and raw server economics, database indexing strategies, and resisting premature architectural complexity.
We have several clusters in DC19 and have had stable uplinks for most of the year, but now and then DDOS happens. This also happened with our other servers on other providers too, like UK2 and OVH over the years. Unfortunately, DDOS is not easy to protect from and cost a lot to manage. I'd rather have these outages on rare situations than pay 200% more for the same performance. No client's have complained either, because they know we offer more for less.
But if you run HA nodes, cloud or premium services on cheap providers like Hetzner, UK2 or OVH (excluding their cloud offering that has DDOS protection) you only have yourself to blame.
The company I work in has some servers in RZ19 and we have intermittent problem of similar kind all the time over the last month or two. Most of the time the servers just stop being reachable from the Internet or packet loss occurs when trying to reach them, at random times during the day for brief periods like 1 minute, and Hetzner explained it with DoS attempts on other servers in the same data center. Has anyone else had similar issues? Yesterday evening and today they have of course much bigger issues.
I hate this - especially on a sunday afternoon when I don't want to think about work :( My phone keeps ringing because some of our servers appear to offline....
I wonder how many admins around the globe are currently annoyed by monitoring alerts :-(
I have had bad experiences with Hetzner in past, there are many better providers out there I do not understand why people use them and their cheap desktop based servers and minimal support (even if hardware breaks and you can prove it they drag their legs)
yes, 144.76.0.0/24 looks down for me too, unfortunately even the robot administration is unavailable
it seems large parts of their address space within the 144.76.0.0/24 netblock (and not just the mentioned 144.76.36.0/27) are currently shut down. ipv6 traffic is still fine.