统信国产UOS v20-1070E 系统环境安装 Kubernetes v1.26.9集群

前言为推进软硬件国产化工作我在统信UOS v20-1070E系统实验环境中部署了k8s v1.26.9集群。现将完整安装过程整理记录仅供自己后续参考使用注相关软件安装包下载链接未附请自行获取。一、基础环境准备所有节点UOS v20-1070E 系统环境优化在 UOS v20-1070E 上安装 Kubernetes v1.26.9集群时操作系统必须完成关闭防火墙 / SELinux、禁用 Swap、内核网络参数、加载网桥模块、文件句柄 / 进程数限制、主机名与 hosts、时间同步、内核版本建议。以下为可直接复制的操作步骤关闭防火墙systemctl stop firewalld systemctl disable firewalld关闭 SELinuxUOS 默认可能已关闭建议强制禁用 setenforce0sed-is/^SELINUX.*/SELINUXdisabled//etc/selinux/config[rootUoSv20-WorkerIP100 ~]# getenforceDisabled禁用 SwapK8s 强制要求# 临时关闭swapoff-a# 永久关闭注释 fstab 中 swap 行sed-i/swap/s/^/#//etc/fstab设置主机名与 hosts 解析hostnamectl set-hostname UoSv20-WorkerIP100--staticcat/etc/hostsEOF10.x.x.252 docker.harbor.com10.x.x.228 UoSv20-MasterIP22810.x.x.100 UoSv20-WorkerIP10010.x.x.227 UoSv20-WorkerIP22710.x.x.229 UoSv20-WorkerIP229 EOFvim /etc/hosts时间同步集群一致性必需# 安装 chronyUOS 官方源yum update yuminstall-ychrony systemctlenablechronyd systemctl start chronyd# 验证chronyc sources二、内核模块与网络参数所有节点关键1.加载内核模块网桥、IPVS# 一次性加载modprobe overlay modprobe br_netfilter modprobe ip_vs modprobe ip_vs_rr modprobe ip_vs_wrr modprobe ip_vs_sh modprobe nf_conntrack# 如何永久生效开机自动加载cat/etc/modules-load.d/k8s.confEOF overlay br_netfilter ip_vs ip_vs_rr ip_vs_wrr ip_vs_sh nf_conntrack EOF#立即重新加载并应用系统配置的内核模块列表 #使新添加或修改的内核模块在当前会话中生效而无需重启整个操作系统 sudo systemctl restart systemd-modules-load.service 验证模块是否已成功加载 使用 lsmod | grep ip_vs2.内核 sysctl 参数必须cat/etc/sysctl.d/k8s.confEOF# 网桥转发net.bridge.bridge-nf-call-iptables1net.bridge.bridge-nf-call-ip6tables1# IPv4 转发net.ipv4.ip_forward1# 关闭 IPv6可选环境无 IPv6 时建议net.ipv6.conf.all.disable_ipv61# 内存与 OOMvm.swappiness0vm.overcommit_memory1vm.panic_on_oom0# 文件句柄与 inotifyfs.file-max1048576fs.nr_open1048576fs.inotify.max_user_watches1048576fs.inotify.max_user_instances8192# 网络连接跟踪net.netfilter.nf_conntrack_max262144EOF# 生效sysctl-p/etc/sysctl.d/k8s.conf# 验证sysctlnet.bridge.bridge-nf-call-iptables net.ipv4.ip_forward三、系统资源限制所有节点cat/etc/security/limits.d/k8s.confEOF * soft nofile 1048576 * hard nofile 1048576 * soft nproc 65535 * hard nproc 65535 EOF立即生效当前会话ulimit-n1048576ulimit-u65535四、内核版本建议UOS v20-1070EUOS v20-1070E 默认内核4.19.x可运行 K8s 1.26但建议升级到 5.4 以支持更好的网络 / 存储特性如 Calico eBPF、CSI 驱动 1.查看当前内核uname -r 2.升级内核UOS 官方源yum install -y linux-image-5.4.0-125-generic reboot五、容器运行时前提所有节点K8s 1.26 默认推荐 containerd1.6.x需确保containerd 已安装并配置 cgroup 驱动为 systemd CNI 目录 /opt/cni/bin 存在并包含必要插件六、验证清单执行后检查# 1. Swap 已关闭 cat /etc/fstab | grep swap free -h # 2. 内核模块已加载 lsmod | grep br_netfilter lsmod | grep ip_vs # 3. 网络参数正确 sysctl net.bridge.bridge-nf-call-iptables net.ipv4.ip_forward # 4. 文件限制生效 ulimit -n ulimit -u七、安装 containerd 运行时和runc 到/usr/local/bin下1.安装containerd 运行时 到/usr/local/bin下ll /usr/local/bin 查看目录内文件情况 tar -zxvf containerd-1.6.22-linux-amd64.tar.gz mv bin/* /usr/local/bin/ 2.拷贝containerd.service 到/usr/lib/systemd/system/containerd.service cp containerd.service /usr/lib/systemd/system/ cp runc.amd64 /usr/local/bin/runc chmod x /usr/local/bin/runc ll /usr/lib/systemd/system/containerd.service -rw-r--r-- 1 root root 1308 4月 18 15:58 /usr/lib/systemd/system/containerd.service 3.创建/etc/containerd/certs.d目录config.toml、hosts.toml文件 mkdir -p /etc/containerd/certs.d mkdir -p /etc/containerd/certs.d/registry.k8s.io cat /etc/containerd/certs.d/registry.k8s.io/hosts.toml EOF server https://registry.k8s.io # 这里使用阿里云镜像加速源和你之前的配置保持一致 [host.https://registry.aliyuncs.com/google_containers] capabilities [pull, resolve] skip_verify false EOF sudo containerd config default /etc/containerd/config.toml cd /etc/containerd/ rsync -azuP config.toml certs.d/* 10.x.x.100:/etc/containerd/ vim config.toml 确保一下几项配置正确即可 root /DataDir/containerd #images 存储目录 state /run/containerd temp version 2 # --- 镜像与存储配置 --- [plugins.io.containerd.grpc.v1.cri.registry] config_path /etc/containerd/certs.d # --- 镜像与存储配置 --- [plugins.io.containerd.grpc.v1.cri.containerd.runtimes.runc.options] BinaryName /usr/local/bin/runc SystemdCgroup true sudo systemctl daemon-reload sudo systemctl enable containerd --now #确保/run/containerd/containerd.sock存在 ll /run/containerd/containerd.sock确保存在 4.安装 nerdctl wget https://github.com/containerd/nerdctl/releases/download/v2.2.2/nerdctl-2.2.2-linux-amd64.tar.gz tar -zxvf nerdctl-2.2.2-linux-amd64.tar.gz -C /usr/local/bin/ chmod x /usr/local/bin/nerdctl # 验证 nerdctl --version nerdctl images八、安装k8sv1.26.9 相关组件kubeadm kubelet kubectl cri-tools1.看k8s集群组件镜像版本 yum list kubelet --showduplicates|egrep 1.2[5-8].*|sort -r 2.安装k8s集群组件 K8S_VERSION1.26.9 yum install -y kubelet-${K8S_VERSION} kubeadm-${K8S_VERSION} kubectl-${K8S_VERSION} --disableexcludeskubernetes #配置kubelet开机自启在初始化前需要stop kubelet.service sudo systemctl enable kubelet.service 3.查看k8s集群组件镜像 kubeadm config images list --kubernetes-version v1.26.9 4.拉取k8s集群组件镜像 k8sImagesForNerdctl.sh kubeadm config images pull --kubernetes-version v1.26.9 #官方源拉取镜像cat k8sImagesForNerdctl.sh#!/bin/bashurlregistry.aliyuncs.com/google_containersversionv1.26.9kubeadm_images$(kubeadm config images list --kubernetes-version$version)forfull_imagein$kubeadm_images;do# 处理镜像名兼容 coredns 的三层路径image_name$(echo$full_image|awk-F/{if(NF3) print $3; else print $2})ali_image$url/$image_namek8s_imageregistry.k8s.io/$image_nameecho 拉取镜像$ali_imagenerdctl--namespacek8s.io pull$ali_imageecho 打标签$ali_image→$k8s_imagenerdctl--namespacek8s.io tag$ali_image$k8s_imageecho 删除阿里云镜像$ali_imagenerdctl--namespacek8s.io rmi$ali_imagedoneecho 所有镜像处理完成查看结果 nerdctl--namespacek8s.io images|grep-Ekube|etcd|pause|coredns5.创建crictl.yaml配置文件并写入 crictl.yaml cat /etc/crictl.yaml EOF runtime-endpoint: unix:///run/containerd/containerd.sock image-endpoint: unix:///run/containerd/containerd.sock timeout: 10 debug: false EOF 验证配置是否生效 crictl info 6.初始化k8s集群Master 节点初始化,worker节点不用初始化 kubeadm init --config /home/Resource/kubeadm-config.yaml --ignore-preflight-errorsallcatkubeadm-config.yaml apiVersion: kubeadm.k8s.io/v1beta3 kind: ClusterConfiguration kubernetesVersion: v1.26.9 imageRepository: registry.k8s.io advertiseAddress:10.x.x.100 networking: serviceSubnet:10.96.0.0/12 podSubnet:192.168.0.0/16 --- apiVersion: kubelet.config.k8s.io/v1beta1 kind: KubeletConfiguration cgroupDriver: systemd#查看k8s集群初始化状况kubectl get csWarning: v1 ComponentStatus is deprecated in v1.19NAME STATUS MESSAGE ERRORetcd-0 Healthycontroller-manager Healthy okscheduler Healthy okkubectl get pods -A7.控制平面安装calico CNI插件 导入images ctr -n k8s.io images import calico-kube-controllersv3.25.0.tar ctr -n k8s.io images import calico-cniv3.25.0.tar ctr -n k8s.io images import calico-nodev3.25.0.tar 创建Calico 网络插件POD kubectl apply -f calico.yaml 8.Worker节点加入k8s集群