Mega Benchmark: Cost of AWS vs. GCE vs. IBM vs. Hetzner for Machine Learning
In this technical retrospective (2017), @Radim shares practical architectural lessons learned from running, scaling, or migrating production software under lean constraints.
HN engineers debated the tradeoffs between cloud convenience and raw server economics, database indexing strategies, and resisting premature architectural complexity.
We found IBM’s Softlayer to be highly “new-user” friendly, which greatly expedited the process of configuring and ordering an instance
That's surprising. Usually the complaint is that IBM's stuff if complicated to figure out and Google and others are "user-friendly".
Softlayer is also the only platform that allows the provisioning of bare metal servers amongst the 3 cloud service providers,
Wonder if there is an opportunity there to deploy Kubernetes on it and manage it on your own. Before cloud orchestration was the super-secret-money-making sauce, but lately with some solutions being open source maybe that's changing and bare metal servers would become popular again. Wonder in general if Google talking about Borg and sponsoring Kubernetes was aimed at weakening Amazon's AWS's position.
Looks like the instances they use are not GPU instances, so I’m not sure what the point of this is. Also they do not take into account EC2 spot pricing, which saves a ton of money and are the main reason I use AWS over alternate clouds.
People are always talking about spot pricing when GPU compute comes up which is valid but not for the ML use-case (which this article is discussing). Spot is 100% incompatible with ML training. You need to be guaranteed that your many hour/multiple day job is going to complete which is antithetical to spot. At Paperspace, our goal is to provide a steady state low price that you can depend on because our primary audience is ML. Spot would be more ideal from an efficiency perspective but it’s just not viable in this context.
With regards to stability, I had no issues with either AWS, Softlayer or Hetzner. However, when running a long task (the benchmark with Tensorflow-GPU) on GCE, the job abruptly stopped after a few days without showing any errors, and I couldn’t figure out why. A repeated attempt also resulted in the same failure.
I encountered same error with google cloud ML engine. No errors on warning when shutting down. Any idea why?
As a matter of curiosity, is anyone out there using dedicated hosts in a big way for ML? The price/performance is pretty good based on these stats.
(I've used Hetzner in the past--in my experience their hosts are not very reliable but are super cheap.)
Hi guys, sorry for the unresponsive site. I had to add crap to the end of the URL so I could resubmit here. But now without caching, the HN traffic is melting our server :( EDIT: fixed.
__________
Answering the comments here:
GPUs: makes no sense for word2vec (and many other ML algos).
Azure: no support from Microsoft for our program. The Docker image and all code is 100% public though, so please run yourself & report the results. We'll be happy to update.
AWS spot: might still try that.
Thanks for the feedback!