Experimenting with flux2 on microk8s
In which I try to get flux2 working on microk8s...
![Experimenting with flux2 on microk8s](/content/images/size/w2000/2021/02/Rca08b77da6326b94cc7854d84bd71687.png)
Flux is a tool for configuring Kubernetes clusters using Git repositories. Changes in configuration pushed to a GitHub repo are then pushed to the cluster. This enables a GitOps-style approach to application infrastructure management.
microk8s is a minimal but complete Kubernetes distribution available from Canonical. It is fully containerized as a snap package which is easy to install and run.
Spun up a dedicated VPS on Vultr, SSH'd in, and ran OS updates:
apt update ; apt upgrade -y
![](https://boxofcables.dev/content/images/2021/02/image-1.png)
Installed microk8s:
snap install microk8s --classic
![](https://boxofcables.dev/content/images/2021/02/image-2.png)
Confirmed microk8s was up and running:
microk8s status --wait-ready
![](https://boxofcables.dev/content/images/2021/02/image-13.png)
The snap of flux is flux 1.x, so to get install flux2, it is necessary to use the install script:
Installed flux2:
curl -s https://toolkit.fluxcd.io/install.sh | sudo bash
![](https://boxofcables.dev/content/images/2021/02/image-3.png)
Ran the flux prerequisites check:
flux check --pre
![](https://boxofcables.dev/content/images/2021/02/image-8.png)
flux2 doesn't detect the kubectl alias.
Looked at the source of flux2 where error is being generated, it is looking for kubectl in the OS PATH variable.
Create a kubectl wrapper script:
nano /usr/local/bin/kubectl
Copy and paste the following wrapper script:
#!/bin/bash
microk8s kubectl "$@"
Made the wrapper executable:
chmod u+x /usr/local/bin/kubectl
Tested executable:
kubectl version
![](https://boxofcables.dev/content/images/2021/02/image-9.png)
Re-ran flux prerequisites check:
flux check --pre
![](https://boxofcables.dev/content/images/2021/02/image-14.png)
kubectl is detected, but kubernetes client initialization is now failing.
Troubleshooting attempted:
- Looked at the source of flux2 where error is being generated.
- Used kubectl config view to get the IP of our cluster and set the KUBERNETES_MASTER variable, did not work.
![](https://boxofcables.dev/content/images/2021/02/image-15.png)
- Checked ~/.kube for a config file, not there. Found the kubeconfig used by microk8s deep in the snap, linked it to ~/.kube/config, success!
![](https://boxofcables.dev/content/images/2021/02/image-17.png)
Set my GitHub PAT and username:
export GITHUB_TOKEN=<token>
export GITHUB_USER=sirredbeard
![](https://boxofcables.dev/content/images/2021/02/image-18.png)
Bootstrapped my repository:
flux bootstrap github \
--owner=$GITHUB_USER \
--repository=flux-sandbox \
--branch=main \
--path=./clusters/my-cluster \
--personal
![](https://boxofcables.dev/content/images/2021/02/image-19.png)
Received error:
unable to clone 'ssh://git@github.com/sirredbeard/flux-sandbox', error: dial tcp: lookup github.com: Try again
![](https://boxofcables.dev/content/images/2021/02/image-20.png)
Confirmed GitHub repository was created:
![](https://boxofcables.dev/content/images/2021/02/image-21.png)
Re-ran bootstrap command, no change.
Looked for logs. Probably in source-controller:
kubectl -n flux-system logs source-controller-65f5b6cb7c-ct7k8
![](https://boxofcables.dev/content/images/2021/02/image-22.png)
Looks like source-controller can't reach notification-controller or github.com, odd.
Confirm DNS is enabled:
microk8s enable dns
![](https://boxofcables.dev/content/images/2021/02/image-23.png)
There is some networking thing here I am not seeing. Taking a break to return to it later...
Continuing: I ran the issue by the Kubernetes team, something about it just did not seem right, and it turns out this is a DNS bug.
The temporary workaround stated is to convert to a non-HA cluster, which I did as followed:
microk8s disable ha-cluster
microk8s disable dns
microk8s stop
microk8s start
microk8s enable dns
![](https://boxofcables.dev/content/images/2021/02/image-24.png)
Re-ran the bootstrap script, and success!
![](https://boxofcables.dev/content/images/2021/02/image-25.png)
I then connected to the VPS over SSH using VS Code and git cloned my flux-sandbox repo to my home folder:
![](https://boxofcables.dev/content/images/2021/02/image-26.png)
![](https://boxofcables.dev/content/images/2021/02/image-27.png)
I then added the repository for a sample app, podinfo, from the flux2 Get Started guide to my deployment:
flux create source git podinfo \
--url=https://github.com/stefanprodan/podinfo \
--branch=master \
--interval=30s \
--export > ./clusters/my-cluster/podinfo-source.yaml
![](https://boxofcables.dev/content/images/2021/02/image-31.png)
This created the postinfo-source.yaml manifest (below), which I then committed and pushed to GitHub from VS Code:
![](https://boxofcables.dev/content/images/2021/02/image-32.png)
Next, I then deployed the podinfo app by creating a Flux Kustomization manifest for podinfo:
flux create kustomization podinfo \
--source=podinfo \
--path="./kustomize" \
--prune=true \
--validation=client \
--interval=5m \
--export > ./clusters/my-cluster/podinfo-kustomization.yaml
![](https://boxofcables.dev/content/images/2021/02/image-33.png)
Which I then committed and pushed.
![](https://boxofcables.dev/content/images/2021/02/image-34.png)
And confirmed running:
kubectl get pods --all-namespaces
![](https://boxofcables.dev/content/images/2021/02/image-35.png)
Pausing for now.