실험 환경

To do

Install Container Runtime

설치 가이드
위 linked page의 지침 中, CRI-O로 설치.

Short installation guide:

OS=CentOS_8 VERSION=1.25
curl -L -o /etc/yum.repos.d/devel:kubic:libcontainers:stable.repo https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable/$OS/devel:kubic:libcontainers:stable.repo
curl -L -o /etc/yum.repos.d/devel:kubic:libcontainers:stable:cri-o:$VERSION.repo https://download.opensuse.org/repositories/devel:kubic:libcontainers:stable:cri-o:$VERSION/$OS/devel:kubic:libcontainers:stable:cri-o:$VERSION.repo
yum install -y cri-o
systemctl enable crio & systemctl start crio

Installing K8s with kubeadm

Requirements & Prechecking

• 2 GB or more of RAM per machine

• 2 CPUs or more

• Unique hostname, MAC address, and product_uuid for every node.

  • You can find the following instructions.

    • Host name: hostname

    • MAC address: ip link or ip a or ifconfig -a

    • Product uuid: sudo cat /sys/class/dmi/id/product_uuid

• Full network connectivity between all machines in the cluster (public or private network is fine)

• Certain ports are open on your machines.

  • See here for more details.

• Swap disabled.

  • 일시적: swapoff -a

  • 영구적: /etc/fstab 파일의 swap mount 부분 제거.

• Forwarding IPv4 and letting iptables see bridged traffic

  • Setting:

    cat <<EOF | sudo tee /etc/modules-load.d/k8s.conf
    overlay
    br_netfilter
    EOF
    
    sudo modprobe overlay
    sudo modprobe br_netfilter
    
    # sysctl params required by setup, params persist across reboots
    cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
    net.bridge.bridge-nf-call-iptables  = 1
    net.bridge.bridge-nf-call-ip6tables = 1
    net.ipv4.ip_forward                 = 1
    EOF
    
    # Apply sysctl params without reboot
    sudo sysctl --system

  • Verify that the br_netfilter, overlay modules are loaded by running the following commands:

    lsmod | grep br_netfilter
    lsmod | grep overlay

    Verify that the net.bridge.bridge-nf-call-iptables, net.bridge.bridge-nf-call-ip6tables, and net.ipv4.ip_forward system variables are set to 1 in your sysctl config by running the following command:

    sysctl net.bridge.bridge-nf-call-iptables net.bridge.bridge-nf-call-ip6tables net.ipv4.ip_forward

Check and open required ports

기본으로 요구되는 port들은 다음과 같다.

Control-plane node(s)

Worker node(s)

Open ports via Firewalld

On every k8s nodes

/etc/firewalld/zones/public.xml
<?xml version="1.0" encoding="utf-8"?>
<zone>
  <short>Public</short>
  <description>For use in public areas. You do not trust the other computers on networks to not harm your computer. Only selected incoming connections are accepted.</description>
  <service name="https"/>
  <port protocol="tcp" port="6443"/>
  <port protocol="tcp" port="10250"/>
  <masquerade/>
  <rule family="ipv4">
    <source address="10.0.0.0/8"/>
    <accept/>
  </rule>
</zone>

$ firewall-cmd --permanent --add-port=6443/tcp
$ firewall-cmd --permanent --add-port=2379-2380/tcp
$ firewall-cmd --permanent --add-port=10250/tcp
$ firewall-cmd --permanent --add-port=10251/tcp
$ firewall-cmd --permanent --add-port=10252/tcp
$ firewall-cmd --permanent --add-port=10255/tcp
$ firewall-cmd --permanent --add-port=8472/udp
$ firewall-cmd --permanent --add-masquerade

# only if you want NodePorts exposed on control plane IP as well
$ firewall-cmd --permanent --add-port=30000-32767/tcp
systemctl restart firewalld

$ firewall-cmd --reload

$ firewall-cmd --permanent --add-port=10250/tcp
$ firewall-cmd --permanent --add-port=10255/tcp
$ firewall-cmd --permanent --add-port=8472/udp
$ firewall-cmd --permanent --add-port=30000-32767/tcp
$ firewall-cmd --permanent --add-masquerade

$ firewall-cmd --reload

Installing kubeadm, kubelet and kubectl

You will install these packages on all of your machines:

• kubeadm: the command to bootstrap the cluster.

• kubelet: the component that runs on all of the machines in your cluster and does things like starting pods and containers.

• kubectl: the command line util to talk to your cluster.

Installation instructions on CentOS, RHEL or Fedora

cat <<EOF | sudo tee /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-\$basearch
enabled=1
gpgcheck=1
gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
exclude=kubelet kubeadm kubectl
EOF

setenforce 0
sed -i 's/^SELINUX=enforcing$/SELINUX=permissive/' /etc/selinux/config

yum install -y kubelet kubeadm kubectl --disableexcludes=kubernetes

systemctl enable --now kubelet

Start kubelet

sudo systemctl start kubelet 명령으로 실행하고 나서 , journalctl -n1000 -f|grep -i kubelet 를 통해 로그를 보면 /var/lib/kubelet/config.yaml이 없다고 에러가 난다.
주기(10초)적으로 systemd에 의해 실행되며, 또한 설정 파일에 변경 사항이 발견되면 systemctl daemon-reload && systemctl restart kubelet이 실행된다.

추후, kuberadm init으로 자동 생성되면, /var/lib/kubelet/config.yaml이 생성되며, 정상적으로 데몬이 실행되어 있음을 확인할 수 있다.