0

Preface


So I'm not one to jump aboard Kubernetes just because it's become really popular. I don't have a business, it's literally just me, so I'm not interested in making things as complicated as possible without any real need.

Now I don't mind learning Kubernetes, but my research has told me it's only for very large companies and that many people are jumping aboard because it's the new hype but I've also read about many people quickly leaving because they realize it's designed for a team of dedicated people and not just for any person.

My issue


I want to start a digital ocean cluster of 5 computers each about 2GB of memory until I can bring in revenue from it. I want to roll out about 10 - 15 websites. Some are dead simple with just a static html page served from NGINX, others are full web apps requiring Redis, MySQL or Mongo, static asset storage, etc... I want to just develop an app and throw it out there to run on the cluster and be able to expand the cluster as needed and not have to worry about extra work and potential downtime for everything as a whole for just deploying a new app or site.

I do know that not everything in the computer world is that simple and I know that there's several moving wheels in a cluster so that's perfectly fine. What I mean is I don't want is to needlessly complicate it if I don't have to.

It's important to note that all of these web sites will be in their own domains. It's not going to be under the same domain or sub-domain or a path in the same domain. I've actually bought domains for all these websites ahead of time and they're already linked up to Digital Ocean waiting for me to sort this out.

Tried Solutions


Docker Compose

My first attempt was to avoid a cluster to keep things simple on one machine and use Docker Compose. But I quickly ran into all kinds of issues that made it clear it was not a good solution. I had to juggle multiple compose files running at the same time and access each others networks which got problematic if a very certain startup and shutdown order wasn't used and I had the complication of having to have a separate router compose file that would reverse proxy to the different compose files since I only had one IP and one port for all the websites.

Docker Swarm

I switched to Docker Swarm so that each stack could be managed separately and this was ok for the most part, a bit of hacking around to get data storage right given I run as non-root but I just bind mounted to folders and told swarm to keep it tied to that machine.

The biggest issue I had was still the routing issue and swarm amplified this. The entire cluster only had one IP and one port (Unless I were to spin-up individual nodes for each website and tie them to a host network which defeats the point of even doing all this). So I still had to create a separate image that would configure a reverse proxy to listen to arbitrary ports and reverse proxy to the container. This got even more complicated with SSL and I ran into all kinds of proxying issues.

So I would have to deploy a webapp with a webserver then I would have to go and modify my router image for the new update and deploy that. There was a high chance I'd have to go through several iterations of uploading to a private Docker Hub since swarm doesn't build images which is understandable and then download it to the swarm and see if it works or not.

If I wasn't careful odds are high even when deploying to production the router could fail and there goes every single website and webapp I have running across the whole cluster since it all has to go through the Router. Testing locally first before deploying was incredibly hard for the router because changing a few environment variables wasn't enough. The router almost had to be completely different for a local dev because it's specific to the cluster it's deployed to which feels counter productive to the philosophy behind a container which depends on portability and self-containment.

I'm aware of NGINX router images that work with Docker Sockets

These are images people have created that auto-provision a router based on communication from the Docker process. But not only are these either dead or have many hundreds of open issues but it feels very hacky and doesn't feel like it'd be ok for production especially with such a high issue count. Then it also seems like it'd give me more complications over the reverse proxy details and if SSL certificates were involved and more-so over multiple domains and sub-domains.

I'm seriously considering Kubernetes


I've been avoiding it like the plague!!! But it's looking more and more like the only option for me. I've heard Kubernetes will assign each pod it's own IP meaning I don't have to deal with the BS from a router that reverse proxies to my whole entire cluster and that's so specific to my cluster I have to try and figure out inaccurate ways to creatively test it outside of my cluster before deploying to my cluster where it still may fail in which case everything on my cluster becomes inaccessible externally. It's own IP address will make things so much incredibly better.

But more so it seems like Kubernetes is really integrated deep into Digital Ocean and that could be another huge saver rather than trying to install Docker on a regular linux container and setup a swarm and setup everything myself manually from scratch and then have to deal with creating a cluster specific router.

0

Here's what I decided to do:

About Kubernetes Descision


Kubernetes is just too complicated for me. I don't have any doubt I could learn it, but the thing I'm worried about is getting caught in the Kubernete internals for hours and days instead of writing code and doing actual development. I feel like Kubernetes is a giant cruise ship and I'd be the only one navigating it (Basically doing all jobs across the entire ship by myself that is supposed to require teams of people) ---- I feel it would be counter-productive and a mistake.

Also with the multiple address thing, yea i'ts $10 per address. It's really amazing I get unique addresses rather than sharing a single address, but If I'm looking for a cost effective way to host 5 or 10 websites then having up to $100 extra just in addresses for only 10 websites that little me is needing to pay for is also not very smart.

My Solution


I'm using Docker Swarm but I'm separating out the router to it's own classical server. Here's why:

The router needed to route to docker swarm so that all websites can share one address is not really part of the cluster. It makes no sense to be part of the cluster it's routing to just as it makes no sense to be a container. A container is a self-contained environment that can be transported to anyone and thrown into any cluster. A cluster router for a specific cluster is counter-intuitive to that philosophy.

Setting up a classical server or a traditional server that routes internally to the cluster makes much more sense especially because I'm not a business so this solution will work really well for me, I don't have extra needs a business would have.

Doing this simplifies management and SSL certificates and allows me to issue reload commands ensuring no downtime just because of a bad config causing the entire thing not to start which means downtime for literally all of my websites.

It's not perfect


This solution isn't the best, I still have all the requests going through one server which is bad. Using swarm with just one address also means I can't load balance as load balancers forward a port, they don't forward domain names on the port. So I can't load balance either. But I know NGINX can scale vertically and it's really good at maximizing the CPU and RAM on a server, so it's at least something.

Having a separate router server also means it has all the resources to itself and not shared by containerization.

NGINX can also be difficult to reverse proxy some websites but I think this may be the best decision.

So that's the decision I made on this.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.