Saving on egress switching from AWS to Hetzner
In this technical retrospective (2022), @zwass shares practical architectural lessons learned from running, scaling, or migrating production software under lean constraints.
Core Principles for Independent Builders:
Defaulting to simpler architectures (such as monolithic frameworks, SQLite, or managed VPS) prevents early engineering bankruptcy.
Evaluating true compute and egress costs reveals that high cloud bills can often be slashed by 80%+ through judicious re-platforming.
Code you write at 2 AM as a solo engineer must be easy to debug six months later without complex microservice dependencies.
Community Debate Themes:
HN engineers debated the tradeoffs between cloud convenience and raw server economics, database indexing strategies, and resisting premature architectural complexity.
CATEGORY Frugal Engineering →
HISTORICAL ERA Pandemic Bubble (2020–2022)
COMMUNITY SCORE 326 points
TOTAL DISCUSSIONS 207 comments
Beware that Hetzner is quick at blocking IPs, with zero recourse or explanation.
If they receive any kind of abuse report, they instantly put you on a 24h timer, but they won't show you contents of that message. If you do not respond in time, they will simply block the IP. What's strange is that I was using Cloudflare in front of Hetzner, but it seems that Cloudflare is ratting their users out and forwarding any abuse mail to the owners of "cloud protected" IPs, all without any notification or warning. Hetzner then sends you an email like "Please remove from our network within the next 24 hours. This site violates 6.2 of Hetzner's terms and conditions.", where site in question is an obvious mirror/search engine of another social network.
What's perhaps more irritating is that Hetzner Cloud has a tiny LIFO queue of ipv4 addresses unique per customer, so you will get the same IP as soon as you release one. You can imagine what happens to a Rancher-managed k8s cluster with Hetzner node driver when one of those IPs gets a no-route: it will get stuck trying to re-deploy a failed node. Mine eventually fell apart though because I did not attend to it for a couple of years.
I'm now hosting from a proverbial garage (NixOS and cron) but proxying traffic through a cheap cloud VM, which saves me at least $1500/mo.
I work at Microsoft (not on Azure) and run Tor relays. My Tor Exit hosts are Psychz Networks, Terrahost, and OVH, along with a middle relay on CenturyLink Fiber.
Why? Azure egress bandwidth is too expensive for a Tor relay, even for a well-paid software engineer like me. I have ~1.5 Gbps Tor bandwidth. If I were to put that on Azure, I would be paying a mortgage worth of bandwidth fees. Same can apply to AWS and Google Cloud.
However, Big Clouds charge more for bandwidth because (a) they're a leader and (b) they have paid peering deals with Big Telecom (e.g. AT&T, Deutsche Telekom, Telefonica, etc.). Smaller hosts like Hetzner don't peer with those ISPs and rely on "transit", but that also means slightly worse performance to Big Telecom.
In short, big clouds can use their market power and advanced technology to command a premium, whereas smaller clouds have to win over customers with lower prices to make up.
Tor relays are ill-suited for Big Clouds, but then your 5000-node Kubernetes cluster is ill-suited for OVH or Hetzner.
I do use the free Azure credit for Tor Bridges, however.
A couple of talking points come to mind:
The linked blog post mentions October 2021, while in December 2021 AWS lowered pricing for CloudFront where the first terabyte out to internet is now completely free:
https://aws.amazon.com/about-aws/whats-new/2021/11/aws-price-reduction-data-transfers-internet/ ↗
For larger use cases most cloud providers have custom or private pricing options available. For example in CloudFront pricing page AWS publicly states that "Custom discounted pricing is available for customers willing to commit to a minimum of 10 TB of data transfer per month for 12 months or longer."
I have noted that most of the blog posts like the one linked above talk about migrating workloads around for savings of tens or hundreds of dollars while focusing on low level IaaS (VMs), and completely ignore the benefits of leveraging cloud services, APIs and related software ecosystems instead of DIY.
[Disclaimer: I'm Co-Founder and CEO of https://www.vantage.sh ↗ - a cloud cost platform]
On this topic as its relevant, at Vantage we launched an automated cost recommendation that keeps an eye on egress out of Cloudfront and S3 and recommends when it makes financial sense to switch to another provider (in the linked blog post, Cloudflare) https://www.vantage.sh/blog/cloudflare-specific-cost-recommendations-for-amazon-cloudfront-and-amazon-s3-egress ↗ -- this has been very popular with current customers/users.
We're actually in process of adding a bunch of new infrastructure and service providers and should include Hetzner in our list of recommended providers but thought I'd mention for anyone wanting an automated approach to knowing when you hit the financial tipping point, this is one option.
It's the same story every day, the same pattern. And it's the same because it works.
Startup: use because it's quick to get started. It's easy to scale it up 100X in 5 minutes for when you make it big. And it provides a long list of easy to use services that are overpriced per use, but which free you to focus on your business rather than complex DevOps.
Then once you scale to the point that you're comparing your salaries and your ops cost, you find smaller services to handle the expensive part of your business. Ones that do exactly what you need, not much else, and charge accordingly. Maybe they're self-operated, or maybe they're other companies with a narrow focus.
Then you write a blog post about how everyone is crazy to use since it's so expensive, and you don't know why you ever thought they were a good idea.