diff --git a/content/posts/2019-04-04-kubernetes-setup.md b/content/posts/2019-04-04-kubernetes-setup.md new file mode 100644 index 0000000..56199e4 --- /dev/null +++ b/content/posts/2019-04-04-kubernetes-setup.md @@ -0,0 +1,313 @@ +--- +title: Single node Kubernetes setup with kubeadm and cert-manager +date: 2019-04-04 +author: James McDonald +type: post +categories: + - Tech +--- + +These are essentially my notes on setting up a single-node Kubernetes cluster +at home. Every time I set up an instance I have to dig through lots of posts, +articles and documentation, much of it contradictory or out-of-date. Hopefully +this distilled and much-abridged version will be helpful to someone else. + +This follows +https://kubernetes.io/docs/setup/independent/create-cluster-kubeadm/ for the +initial cluster setup. You should look there for more detail. I strongly +recommend that you read about the various projects and verify that this +information is still valid. Things change pretty quickly. I've deliberately not +included a lot of detail in the post beyond what I did and the commands to +use. + +# Requirements +You can work around these, but it's recommended to have at least: + +* 4GB RAM +* 2 cores + +# Decisions +Kubernetes is an open ended platform which lets you make a lot of decisions yourself. Unfortunately it also requires you to make them (see projects like k3s to avoid some of that). I decided to use: + +* Debian stable (stretch), because why would you ever not use Debian? +* Docker as the container runtime. Next time I'll try something else (CRI-O), but Docker is familiar so it's easy to diagnose any issues. +* Flannel for networking. No particular reason; it has a nice name. +* No Helm. Helm is great, but it hides away details and I'm interested in details. +* nginx for ingress. There are a bunch of possibilities, but nginx is the common choice and integrates nicely with lots of other things. +* Let's Encrypt certificates validated by Cloudflare DNS. I use Cloudflare already, so this was an easy choice. DNS validation allows for wildcards and for internal hosts. +* Single node. There are a lot of cool things about Kubernetes that you don't get with a single node, but what I'm setting up here is for home. You can easily add more nodes by following the instructions `kubeadm` gives you when it runs. + +# Enable net.bridge.bridge-nf-call-iptables + +This is required by Flannel and possibly other networking options. You can read +more at +https://kubernetes.io/docs/setup/independent/create-cluster-kubeadm/#pod-network + +This was actually already set on my machine, but it doesn't hurt to be explicit. + +``` +cat > /etc/sysctl.d/20-bridge-nf.conf < /etc/docker/daemon.json < /etc/apt/sources.list.d/docker.list +apt-get update +apt-get install -y --no-install-recommends docker-ce +``` + +The `--no-install-recommends` will avoid pulling in stuff you don't need, +including the aufs DKMS package. + +# Install Kubernetes components +``` +curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add - +cat </etc/apt/sources.list.d/kubernetes.list +deb https://apt.kubernetes.io/ kubernetes-xenial main +EOF +apt-get update +apt-get install -y kubelet kubeadm kubectl +``` + +The `xenial` in the APT source is correct. That's the repo they seem to +update, and these are Go binaries anyway so they're self-contained. + +# Run `kubeadm` to set up the cluster + +The `--pod-network-cidr` setting is required by Flannel, which I chose to use +for pod networking. + +``` +kubeadm init --pod-network-cidr=10.244.0.0/16 +``` + +That's it. Neat, huh? + +There is still a bunch of work to do to make the cluster actually useful. You +can do most of the rest of this as a non-root user. Follow the instructions +`kubeadm` gave you to copy the credential **as your regular user**. + +``` +mkdir -p $HOME/.kube +sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config +sudo chown $(id -u):$(id -g) $HOME/.kube/config +``` + +And as a handy extra tip, you'll want completion: + +``` +source <(kubectl completion bash) +``` + +# Install Flannel for pod networking +``` +kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/a70459be0084506e4ec919aa1c114638878db11b/Documentation/kube-flannel.yml +kubectl get pods --all-namespaces +``` + +You should see coredns pods come to life if all is well. + +# Untaint the master so you can run pods +``` +kubectl taint nodes --all node-role.kubernetes.io/master- +``` + +At this point you can run pods and expose them with services. If that's all you +need, you're done! Next I set up an nginx-ingress and cert-manager to allow for +hostname-based HTTPS ingress with Let's Encrypt certificates. + +# Set up nginx-ingress + +I set up the nginx-ingress with host networking so I could expose my cluster's +services via ports 80/443 on the host. You can also expose it via a NodePort or +LoadBalancer, but this worked well for my simple setup. + +``` +kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/mandatory.yaml +cat > nginx-host-networking.yaml < external-dns.yaml < clusterissuer-cf.yaml < wildcard-cert.yaml < nginx-ingress.yaml <