Chapter 3: Leaving The Cloud - The Small Business Version

In Chapter 2: What we've tried, we talked about the journey from very simplistic PythonAnywhere PaaS hosting to coordinating 1, then 3, and eventually up to 8 small servers in DigitalOcean. At each step of that journey, those were the right choices for Talk Python and for me managing them.

Now, we'll cover how Talk Python of today (2024) is hosted. It's a massive improvement on many fronts and has opened a bunch of new opportunities for us while reducing the complexity.

First, we need to talk about leaving the cloud.

The title of this chapter is inspired by David Heinemeier Hansson (DHH). He's the creator of Ruby on Rails and co-founder of Basecamp. A couple of years ago, they got entirely fed up with the complexity and the cost of hyperscaler cloud platforms (AWS in their case). They decided that all the complexity of S3, lambdas, managed DBs, etc., was just too much. The cost was entirely out of control. So, they moved all their apps onto a single huge physical system at a colocation-based data center.

DHH wrote extensively about this on his blog. The entire series is well worth a read. Start with these two: "We have left the cloud" and "The Big Cloud Exit FAQ," then follow the links for more within those articles.

Now, this probably doesn't apply apples to apples to you. It definitely does not to Talk Python. But it offers powerful lessons that apply to Talk Python and to you, too.

Here are the key points from Basecamp's journey:

1. AWS, Azure, and similar hyperscale clouds are very complex, with 50-100 interconnected (and billed separately) services. Examples include S3 and AWS Lambda.
2. Modern hardware has gotten much faster since AWS and others were created. However, the savings have not been passed along to consumers.
3. Lock-in at these clouds is a big risk.
4. Containerization offers much of the isolation that the cloud promises.
5. "Let the cloud handle the operations" has not really paid off. You have to be an expert in AWS rather than Linux, but also Linux.

These bullet points don't do justice to the whole idea, so read the two articles linked above.

What pertinent lessons can we take from this?

1. We can't justify an expensive physical server and the monthly hosting costs. However, we can get our hands on a single big, powerful server for very cheap. We get a single cloud VM at DigitalOcean or Hetzner. In this sense, we can step out of the cloud service lock-in and just run our server while still leveraging all the redundancy and management from their world-class data centers.
2. If we only have one server, we'll need isolation. Containers work perfectly for this.
3. Managing containers and managing VMs require nearly the same effort and skill set.
4. With the setup above, we are perfectly portable across cloud providers (Hetzner, DigitalOcean, and even AWS EC2, if you must ;) ). This keeps us isolated from cloud vendor lock-in.
5. Every piece of software needed to operate this setup (Linux, Docker, SSH, NGINX/Caddy, Python, and more) is free and open-source.

The Current Talk Python Infrastructure

With those principles laid out, you can guess our setup today. I shut down all eight servers one by one. They were migrated to a single large cloud VM running Ubuntu Linux. Large is relative, of course. We are currently using an 8 vCPU + 16GB RAM machine.

When I wrote this chapter, that previous paragraph ended with "running at DigitalOcean." But, part way through this book and largely because of this book, I decided to move to Hetzner. There you'll get faster CPUs and faster networks for much lower prices. I wrote this whole migration up to be included here. You'll find it in Chapter 15: Moving to Hetzner (a Retrospective).

On this central server, we're no longer running our Python apps directly. Instead, everything is containerized using Docker. These different apps and their dependencies are managed via Docker Compose. I intentionally avoided Kubernetes for the time being. It feels like Kubernetes adds significant operational complexity and makes monitoring our running apps harder than pure Docker. Your mileage may vary.

So what does this cost? That depends! But for any operating business, it's truly negligible. Here are some prices based on the hosting company:

• Azure: $350/mo
• AWS: $202/mo
• DigitalOcean: $112/mo <-- Talk Python 2023
• Hetzner: $29/mo <-- Talk Python today

We've been using DigitalOcean for many years, and they have been great. However, the German company Hetzner recently opened a US-based data center. Look at the pricing. It's 6x cheaper than DO and a whopping 20x cheaper than Azure. We could double the server to a whopping 16 vCPU and 32 GB RAM for only $50/mo. We'll go way deeper into this in Chapter 15: Moving to Hetzner (a Retrospective).

Bandwidth costs are another massive consideration. Recall that we're doing around 15 TB/mo of traffic. AWS and Azure charge $0.09/GB. That might seem fine. But for 15 TB, that's 15 x 1,024 x $0.09 = $1,382.40. Wow! In practice, we only have about 1 TB/mo running through our web servers directly. The included bandwidth at Hetzner covers this entirely, so it's $0 more. The rest is through a CDN, for which we pay roughly $40/mo. We have a whole chapter on CDNs later.

The beautiful thing about this setup is that when I wanted to try hosting at Hetzner, it was literally just a data export and restore migration. I didn't have to rewrite our apps for their equivalent of S3 or lambda or whatever. I don't think they have them anyway. Nor do I want them.

With this infrastructure, we typically handle at least 9M requests to the backend Python code and associated database each month. This could go much higher. The typical server load is 5%, with spikes to 20-30%.

Plus, we do zero caching. None. The web app is so fast it's just not needed. A typical request through NGINX -> Python -> Database -> Python -> NGINX response is 30-50ms. Even if we pushed the response time to 0ms, no one would notice. But adding some caching would allow an even higher load on the same server.

Be cloud agnostic

Buying dedicated servers and hosting them in a co-location data center like Deft makes little sense for Talk Python. We can still mirror these benefits by getting a single, large, affordable VM on dedicated hardware at a state-of-the-art data center. This might as well be a dedicated server on real hardware we colocated somewhere else. We saw the price is incredible with Hetzner and DigitalOcean.