Kubernetes Part 4: Install Loadbalancer Metallb on Kubernetes


My previous post the Kubernetes installation ended with a nodeport configuration as a final step to access a pod (container) from outside the kubernetes network.

In Kubernetes there are several different port configurations for Kubernetes services:

  • Port exposes the Kubernetes service on the specified port within the cluster. Other pods within the cluster can communicate with this server on the specified port.
  • TargetPort is the port on which the service will send requests to, that your pod will be listening on. Your application in the container will need to be listening on this port also.
  • NodePort exposes a service externally to the cluster by means of the target nodes IP address and the NodePort. NodePort is the default setting if the port field is not specified.

Source: https://www.bmc.com/

Kubernetes installations at cloud providers like AWS, Azure  Linode  have all a loadbalancer included in their Kubernetes service., Users can access the website (hosted by multiple pod)s from one single IP, which is of course balanced, and your Rasberry Pi Kubernetes doesn't have that option. 

There is however a great solution to have  loadbalancing funtionality available with your newly build Rasberry Pi Kubernetes cluster, and that is the bare metal load balancer from MetalLB.

The link to their website is https://metallb.universe.tf  .

MetalLB supports the use of the flanel network addon we have installed during the installation procedure of installing Kubernetes on a Rasberry Pi Cluster. 

What you see in the post below are the excerpts form the installation procedure, which I have used to configure the MetalLB load balancer on my Pi Kubernetes cluster.

I have used the installation by manifest option. To start: type the following commands on the master node via an SSH session to the K8s-master :

kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.12.1/manifests/namespace.yaml
kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.12.1/manifests/metallb.yaml
# On first install only
kubectl create secret generic -n metallb-system memberlist --from-literal=secretkey="$(openssl rand -base64 128)"
As the  website describes:

This will deploy MetalLB to your cluster, under the metallb-system namespace.
  • The components in the manifest are: The metallb-system/controller deployment. This is the cluster-wide controller that handles IP address assignments. 
  • The metallb-system/speaker daemonset. This is the component that speaks the protocol(s) of your choice to make the services reachable. 
  • Service accounts for the controller and speaker, along with the RBAC permissions that the components need to function. 
The installation manifest does not include a configuration file. MetalLB’s components will still start, but will remain idle until you define and deploy a configmap. The memberlist secret contains the secretkey to encrypt the communication between speakers for the fast dead node detection.

As mentioned after the installation you need to configure of MetalLB. You can choose between a Layer2, BGP or Advanced configuration . I have chosen to use the Layer 2 configuration. It's the simplest to configure and suited my needs.

For the configuration we use a yaml file (example below). You need to change the example values (in red) with the values of your own netwerk. You need enter a range of availble IP addresses of your network. In the example below the MealLB loadbalancer can use ip addresses form the range 192.168-72.65 to 192.168.1.127 which is a total op 64 ip adresses. You can use a range of whatever you want, but don't make it too small (<10) otherwise you are very limited of the apps you can publish via the load balancer.
apiVersion: v1
kind: ConfigMap
metadata:
  namespace: metallb-system
  name: config
data:
  config: |
    address-pools:
    - name: default
      protocol: layer2
      addresses:
      - 192.168.1.65-192.168.1.127  # < example ip range you reserve for MetalLB


So open an SSH session to the k8s-master node.

Type in the following command:

nano metallb-config.yml
  • copy & past the text for the yaml file with your own address values
  • press Ctrl-V to past
  • press Ctrl-O to save
  • press Ctrl-X to exit
You now have created the file metallb-config.yml

Now to apply the yaml file in Kubernetes. Type the following command:
 kubectl create -f metallb-config.yml

Now type in the following command to check if the config has been created.
 kubectl get configmap -n metallb-system

You should see something similar like this







If that is the case, you can configure to use your deployment to use the metalLB load balancer with the following command, to "expose" your deployment to the "outside world"

 kubectl kubectl expose deployment nginx-1 --port 80 --type=LoadBalancer --name=nginx-1 

The red values are example values. You can fill in het your own values. A short explanantion of the parameters. You can download the example nginx-1 yaml file here

- deployment: refers to your deployment in Kubernetes
- port: is the portnumber the IP on the loadbalancer will use
- name: The above command will create a service. This parameter will give the service it's. I usually just name it same as my deployment.

If you didn't see any errors you see if the configuration works, by typing in the following command:
kubectl get service

You should see something like this


As you can see my deployment nginx-1 has now an external-ip via the load balancer (example: 192.168.72.65) which is reacheable via port 80.  So if you type in https://192.168.72.65 from your network you should see a webpage hosted on the nginx-1 deployment. MetalLB will load balance requests via ip 192.168.72.65 to all pods configured.

I hope the article helpfull and easy to understand in configuraton MetalLB for your Pi Cluster. If you have any questions, do not hesitate leave a comment. 

In my next blog I will explain how I have configured persistent volumes for Pi Kubernetes cluster via a NFS share on my Synology NAS.




Comments

  1. Thank you for your tutorial. I am lost at "kubectl expose deployment nginx-1 --port 80 --type=LoadBalancer --name=nginx-1" because in "Part 3" a "deployment" wasn't discussed as far as I see. 'Error from server (NotFound): deployments.apps "nginx-1" not found'. Same error when I replace "nginx-1" with "nginx-example" that was created in Part 3.

    ReplyDelete
    Replies
    1. Anonymous5:09 PM

      I have added the example for nginx-1. You can find it here

      https://github.com/erik-de-bont/blog/blob/main/kubernetes/part_4/nginx-1-example.yml

      Delete
    2. This works: kubectl expose service nginx-example --port 80 --type=LoadBalancer --name=nginx-example1

      Delete
    3. I just saw your post, thanks!

      Delete
  2. I set up my IPAddressPool wrong the first time around, I've since deleted and recreated it, but the service keeps getting an external IP from the first pool I created and not the active one. Is this cached somewhere that I can clear it out?

    ReplyDelete
    Replies
    1. You can type this command to check the log " kubetail -l app.kubernetes.io/component=speaker -n metallb-system". More info: https://metallb.universe.tf/configuration/troubleshooting/.

      If that doesn't help. I suggest to complety remove metallb via this command:

      kubectl delete -f https://raw.githubusercontent.com/metallb/metallb/v0.12.1/manifests/metallb.yaml

      Wait a few minutes so kubernetes can remove all metallb pods and re-apply the installation again via "kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.12.1/manifests/metallb.yaml" and redo the configuration. I had to change it also in the past, and this procedure worked.

      Delete
  3. There's a few new things with MetalLB that are important to note:

    1. The new way of making a config is to declare an IP address pool and an L2 advertisement (for layer 2 mode). This can be seen here in the docs: https://metallb.universe.tf/configuration/#layer-2-configuration

    2. If you're using a single node in your cluster with kubeadm, make sure you remove the label "exclude-from-external-load-balancers" on the control plane node. Not removing this makes it so your LB ip address isn't advertised to the network. I wasn't able to curl my load balancer because of this issue. The command to fix it is:
    ```
    kubectl label nodes node.kubernetes.io/exclude-from-external-load-balancers-
    ```
    This is a known issue and is being resolved: https://github.com/metallb/metallb/issues/2285

    ReplyDelete

Post a Comment