Cluster Setup

Metal LB
This section is to get metallb setup and working for a bare metal setup. 

 Let create the namespace 

 kubectl create namespace metallb-system

kubectl label namespace metallb-system pod-security.kubernetes.io/enforce=privileged 

 We will use helm for ease of upgrades and the initial install. First, we need to add the repo. 

 helm repo add metallb https://metallb.github.io/metallb 

 Now, install metallb with helm. 

 helm install metallb metallb/metallb -n metallb-system 

 We need to choose a pool of IP addresses that metal lb can hand out for the type LoadBalancer. In my case, I really just want this for nginx ingress. We need to create the following yaml file to apply to the API. 

 apiVersion: metallb.io/v1beta1

kind: IPAddressPool

metadata:

 name: main-pool

 namespace: metallb-system

spec:

 addresses:

 - 192.168.249.10-192.168.249.11

---

apiVersion: metallb.io/v1beta1

kind: L2Advertisement

metadata:

 name: l2-lb

 namespace: metallb-system 

 Lets verify everything is up and running. 

 kubectl get pods -n metallb-system

nginx Ingress Controller
After installing metallb we can move on to the ingress part of setup. My preferred choice is nginx. We will also get this ready for monitoring with Prometheus. 

 Create the namespace. 

 kubectl create namespace ingress-nginx

kubectl label namespace ingress-nginx pod-security.kubernetes.io/enforce=privileged 

 We will use helm to install nginx. This will help with upgrades in the future. Note the extra values for Prometheus. 

 helm upgrade --install ingress-nginx ingress-nginx --repo https://kubernetes.github.io/ingress-nginx --namespace ingress-nginx --set controller.metrics.enabled=true --set-string controller.podAnnotations."prometheus\.io/scrape"="true" --set-string controller.podAnnotations."prometheus\.io/port"="10254" 

 After a little bit of time, we can check the status of the namespace. The important part to note is the service/ingress-nginx-controller. It should have an External-IP provided by metallb. 

 kubectl get all -n ingress-nginx 

 Example Output: 

 NAME READY STATUS RESTARTS AGE

pod/ingress-nginx-controller-8b8b9f598-jqxcr 1/1 Running 0 23m

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE

service/ingress-nginx-controller LoadBalancer 10.109.19.205 192.168.249.10 80:32588/TCP,443:30617/TCP 23m

service/ingress-nginx-controller-admission ClusterIP 10.100.21.15 <none> 443/TCP 23m

service/ingress-nginx-controller-metrics ClusterIP 10.107.12.193 <none> 10254/TCP 23m

service/prometheus-server NodePort 10.97.78.93 <none> 9090:32631/TCP 5s

NAME READY UP-TO-DATE AVAILABLE AGE

deployment.apps/ingress-nginx-controller 1/1 1 1 23m