Cilium

Cilium eBPF Networking Plugin

Cilium is open source software for transparently securing the network connectivity between application services deployed using Linux container management platforms like Docker and Kubernetes.

At the foundation of Cilium is a new Linux kernel technology called eBPF, which enables the dynamic insertion of powerful security visibility and control logic within Linux itself. Because eBPF runs inside the Linux kernel, Cilium security policies can be applied and updated without any changes to the application code or container configuration.

Official Documentation

Configuration

Kube Proxy Replacement

The service host and port can be found in the default namespace.

kubectl -n default get svc kubernetes

NAME         TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGE
kubernetes   ClusterIP   10.43.0.1    <none>        443/TCP   23h

Cluster IP Address Management

  • Set pod ip pool to non conflicting value.
  • Enable IPv6 support.

Loadbalancer using BGP

BGP support is enabled by providing the BGP configuration via a ConfigMap and by setting a few Helm values.

Wireguard Encryption

When WireGuard is enabled in Cilium, the agent running on each cluster node will establish a secure WireGuard tunnel between it and all other known nodes in the cluster.

Hubble

Hubble is the observability layer of Cilium and can be used to obtain cluster-wide visibility into the network and security layer of your Kubernetes cluster.

Cluster Mesh

  • PodCIDR ranges in all clusters and all nodes must be non-conflicting and unique IP addresses.
  • Nodes in all clusters must have IP connectivity between each other. This requirement is typically met by establishing peering or VPN tunnels between the networks of the nodes of each cluster.

Cluster Mesh Documentation

Name and ID of the cluster. Only required for Cluster Mesh.

Tunnel protocol to use when connecting with external clusters or services.

Others

  • Disable layer 7 proxy.
  • Deploy Cilium operator on control plane nodes.
  • Use priority class system-cluster-critical.
l7Proxy: false

operator:
  nodeSelector:
    node-role.kubernetes.io/control-plane: "true"


priorityClassName: "system-cluster-critical"

Installation

To use the BGP functionality of cilium we first need to create a special config map object. The address range defined in the config map tells cilium in which range the services should be configured.

cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: ConfigMap
metadata:
  name: bgp-config
  namespace: kube-system
data:
  config.yaml: |
    peers:
      - peer-address: 10.10.0.1
        peer-asn: 64512
        my-asn: 64512
    address-pools:
      - name: default
        protocol: bgp
        addresses:
          - 10.10.10.0/24
EOF

Create a file named cilium.values.yaml with the following content.

kubeProxyReplacement: strict
k8sServiceHost: 10.43.0.1
k8sServicePort: 443

ipam:
  operator:
    clusterPoolIPv4PodCIDR: "10.42.0.0/16"
    clusterPoolIPv6PodCIDR: "fd00::/104"

ipv6:
  enabled: true

bgp:
  enabled: "true"
  announce:
    loadbalancerIP: "true"

encryption:
  enabled: true
  type: wireguard

hubble:
  relay:
    enabled: true
  ui:
    enabled: true

cluster:
  name: cute-kitten
  id: 7

tunnel: "geneve"

l7Proxy: false

operator:
  nodeSelector:
    node-role.kubernetes.io/control-plane: "true"


priorityClassName: "system-cluster-critical"

Install Cilium with this command.

helm install --namespace kube-system --values ./cilium.values.yaml cilium cilium/cilium