SRE文档

深浅模式

1. nerdctl简介 ​

命令基本与docker兼容,nerdctl自带compose,nerdctl >= 0.8才有 生产推荐使用nerdctl

nerdctl 是用于 containerd 的与 Docker兼容的 CLI。主要适用于 Docker转到 Containerd 的用户，操作 Containerd 的命令行工具 ctr 和 crictl 不怎么好用，所以出现了 nerdctl工具.

nerdctl 操作的是 containerd 而非 docker，但它只是用法保持了 docker cli 的习惯，实质上操作的是 containerd

实际上nerdctl compose实现的是Compose Specification规范， 这个规范是从自Docker Compose file version 3 specification规范发展而来的。

2. 部署 ​

2.1 下载 ​

  • 基于Rocklinux9.x
bash
wget https://github.com/containerd/nerdctl/releases/download/v1.7.7/nerdctl-full-1.7.7-linux-amd64.tar.gz
wget https://github.com/containerd/nerdctl/releases/download/v1.7.7/nerdctl-full-1.7.7-linux-amd64.tar.gz

❌ 注意

安装 nerdctl-full 版本集成了 containerd 。如主机已安装 containerd 请选择 nerdctl简易版

2.2 安装 ​

由于安装包是基于二进制,直接解压即可使用

bash
tar zxvf nerdctl-full-1.7.7-linux-amd64.tar.gz -C /usr/local/bin
tar zxvf nerdctl-full-1.7.7-linux-amd64.tar.gz -C /usr/local/bin

2.2.1 配置内核参数 ​

bash
echo "net.bridge.bridge-nf-call-ip6tables = 1" >> /etc/sysctl.conf
echo "net.bridge.bridge-nf-call-iptables = 1" >> /etc/sysctl.conf
echo "net.bridge.bridge-nf-call-ip6tables = 1" >> /etc/sysctl.conf
echo "net.bridge.bridge-nf-call-iptables = 1" >> /etc/sysctl.conf
bash
modprobe br_netfilter

sysctl -p
modprobe br_netfilter

sysctl -p

2.2.2 命令补全 ​

bash
yum install bash-completion -y
yum install bash-completion -y
bash
vim /etc/profile
source <(nerdctl completion bash)

#执行生效
[root@kube-master-01 init_pack]#source /etc/profile


# 生成自动补全文件
nerdctl completion bash > /etc/bash_completion.d/nerdctl
nerdctl completion bash > /etc/bash_completion.d/docker
vim /etc/profile
source <(nerdctl completion bash)

#执行生效
[root@kube-master-01 init_pack]#source /etc/profile


# 生成自动补全文件
nerdctl completion bash > /etc/bash_completion.d/nerdctl
nerdctl completion bash > /etc/bash_completion.d/docker

2.2.3 nerdctl更名为 docker ​

看个人需要

bash
cat << 'EOF' > /usr/local/bin/docker
#!/bin/bash
/usr/local/bin/nerdctl $@
EOF

#t添加权限
chmod +x /usr/local/bin/docker
cat << 'EOF' > /usr/local/bin/docker
#!/bin/bash
/usr/local/bin/nerdctl $@
EOF

#t添加权限
chmod +x /usr/local/bin/docker

2.2.3 查看版本 ​

bash
[root@kube-master-01 init_pack]# nerdctl info
Client:
 Namespace:	default
 Debug Mode:	false

Server:
 Server Version: v1.7.22
 Storage Driver: overlayfs
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Log: fluentd journald json-file syslog
  Storage: native overlayfs
 Security Options:
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 5.14.0-427.33.1.el9_4.x86_64
 Operating System: Rocky Linux 9.4 (Blue Onyx)
 OSType: linux
 Architecture: x86_64
 CPUs: 2
 Total Memory: 3.543GiB
 Name: kube-master-01
 ID: e99af880-4220-489a-b0a0-6ae2fb786877
[root@kube-master-01 init_pack]# nerdctl info
Client:
 Namespace:	default
 Debug Mode:	false

Server:
 Server Version: v1.7.22
 Storage Driver: overlayfs
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Log: fluentd journald json-file syslog
  Storage: native overlayfs
 Security Options:
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 5.14.0-427.33.1.el9_4.x86_64
 Operating System: Rocky Linux 9.4 (Blue Onyx)
 OSType: linux
 Architecture: x86_64
 CPUs: 2
 Total Memory: 3.543GiB
 Name: kube-master-01
 ID: e99af880-4220-489a-b0a0-6ae2fb786877

❌ 注意

如果安装containerd不是完全安装,此处的安装网络插件

https://github.com/containernetworking/plugins/releases/tag/v1.5.1

2.2.4 部署buildkit服务 ​

bash
#看先前文章
cp  /usr/local/lib/systemd/system/buildkit.service /etc/systemd/system/buildkitd.service

# 重新加载Unit file
systemctl daemon-reload

#开机启动,并启动服务
systemctl enable --now containerd buildkit
#看先前文章
cp  /usr/local/lib/systemd/system/buildkit.service /etc/systemd/system/buildkitd.service

# 重新加载Unit file
systemctl daemon-reload

#开机启动,并启动服务
systemctl enable --now containerd buildkit

3. 基本命令 ​

https://github.com/containerd/nerdctl

3.1 创建容器 ​

bash
[root@kube-master-01 init_pack]# nerdctl run -d -p 80:80 --name=nginx --restart=always nginx
docker.io/library/nginx:latest:                                                   resolved       
....
|++++++++++++++++++++++++++++++++++++++|
layer-sha256:bbfaa25db775e54ec75dabe7986451cb99911b082d63bbf983ab20fc6f7faaf4:    done           |++++++++++++++++++++++++++++++++++++++|
layer-sha256:7bb6fb0cfb2b319dee79e476c11620e7fa47f22ecdedc999e207984f62a4554c:    done           |++++++++++++++++++++++++++++++++++++++|
layer-sha256:0723edc10c178df9245f49c9b8e503c4223a959ee5a072f043d71669132bc5e9:    done           |++++++++++++++++++++++++++++++++++++++|
layer-sha256:24b3fdc4d1e3b419643068364b3d4e1b7e280f5a8a3c1e3651e9e67363e6434b:    done           |++++++++++++++++++++++++++++++++++++++|
elapsed: 37.3s                                                                    total:  67.7 M (1.8 MiB/s)
380924631b3af4a82da1877891eaf5348a59116c7630e39fefffd766bf40f90b


#启动容器并指定特定网络(使用宿主机网络直接启动容器)
nerdctl run --name nginx --net host -d nginx:alpine
[root@kube-master-01 init_pack]# nerdctl run -d -p 80:80 --name=nginx --restart=always nginx
docker.io/library/nginx:latest:                                                   resolved       
....
|++++++++++++++++++++++++++++++++++++++|
layer-sha256:bbfaa25db775e54ec75dabe7986451cb99911b082d63bbf983ab20fc6f7faaf4:    done           |++++++++++++++++++++++++++++++++++++++|
layer-sha256:7bb6fb0cfb2b319dee79e476c11620e7fa47f22ecdedc999e207984f62a4554c:    done           |++++++++++++++++++++++++++++++++++++++|
layer-sha256:0723edc10c178df9245f49c9b8e503c4223a959ee5a072f043d71669132bc5e9:    done           |++++++++++++++++++++++++++++++++++++++|
layer-sha256:24b3fdc4d1e3b419643068364b3d4e1b7e280f5a8a3c1e3651e9e67363e6434b:    done           |++++++++++++++++++++++++++++++++++++++|
elapsed: 37.3s                                                                    total:  67.7 M (1.8 MiB/s)
380924631b3af4a82da1877891eaf5348a59116c7630e39fefffd766bf40f90b


#启动容器并指定特定网络(使用宿主机网络直接启动容器)
nerdctl run --name nginx --net host -d nginx:alpine

3.2 列出容器 ​

bash
[root@kube-master-01 init_pack]# nerdctl ps
CONTAINER ID    IMAGE                             COMMAND                   CREATED          STATUS    PORTS                 NAMES
380924631b3a    docker.io/library/nginx:latest    "/docker-entrypoint.…"    3 minutes ago    Up

#指定namespace
nerdctl -n k8s.io ps -a
[root@kube-master-01 init_pack]# nerdctl ps
CONTAINER ID    IMAGE                             COMMAND                   CREATED          STATUS    PORTS                 NAMES
380924631b3a    docker.io/library/nginx:latest    "/docker-entrypoint.…"    3 minutes ago    Up

#指定namespace
nerdctl -n k8s.io ps -a

3.3 进入容器 ​

bash
[root@kube-master-01 init_pack]# nerdctl ps
CONTAINER ID    IMAGE                             COMMAND                   CREATED          STATUS    PORTS                 NAMES
380924631b3a    docker.io/library/nginx:latest    "/docker-entrypoint.…"    3 minutes ago    Up        0.0.0.0:80->80/tcp    nginx
[root@kube-master-01 init_pack]# nerdctl exec -it 380924631b3a /bin/sh
[root@kube-master-01 init_pack]# nerdctl ps
CONTAINER ID    IMAGE                             COMMAND                   CREATED          STATUS    PORTS                 NAMES
380924631b3a    docker.io/library/nginx:latest    "/docker-entrypoint.…"    3 minutes ago    Up        0.0.0.0:80->80/tcp    nginx
[root@kube-master-01 init_pack]# nerdctl exec -it 380924631b3a /bin/sh

3.4 删除容器 ​

nerdctl rm -f nginx
nerdctl rm -f nginx

3.4 删除镜像 ​

nerdctl rmi -f <IMAGE ID>
nerdctl rmi -f <IMAGE ID>

3.5 列出镜像 ​

bash
nerdctl images
nerdctl -n=k8s.io images
nerdctl -n=k8s.io images | grep -v '<none>'
nerdctl images
nerdctl -n=k8s.io images
nerdctl -n=k8s.io images | grep -v '<none>'

3.6 拉取镜像 ​

bash
nerdctl pull nginx
nerdctl -n k8s.io pull nginx

#使用 nerdctl login --username xxx --password xxx 进行登录

#登陆
nerdctl login

#注销
nerdctl logout
nerdctl pull nginx
nerdctl -n k8s.io pull nginx

#使用 nerdctl login --username xxx --password xxx 进行登录

#登陆
nerdctl login

#注销
nerdctl logout

3.7 标签 ​

bash
#打标签，必须指定namespace 否则kubectl无法使用, 默认使用default命名空间下
nerdctl -n k8s.io tag old_image  new_image
#打标签，必须指定namespace 否则kubectl无法使用, 默认使用default命名空间下
nerdctl -n k8s.io tag old_image  new_image

3.8 镜像导出 ​

bash
nerdctl save -o busybox.tar.gz busybox:latest
nerdctl save -o busybox.tar.gz busybox:latest

3.9 镜像导入 ​

bash
nerdctl load -i busybox.tar.gz

或者
nerdctl load < busybox.tar.gz
nerdctl load -i busybox.tar.gz

或者
nerdctl load < busybox.tar.gz

4.0 镜像构建 ​

bash
#指定dockerfile文件，默认使用default命名空间下
nerdctl -n k8s.io build -t centos:v1.0 -f centos.dockerfile .

#默认
nerdctl build -t centos:v1.0 .
#指定dockerfile文件，默认使用default命名空间下
nerdctl -n k8s.io build -t centos:v1.0 -f centos.dockerfile .

#默认
nerdctl build -t centos:v1.0 .

❌ 注意

ctr和nerdctl命令需要指定名字空间，管理k8s创建的容器，需要使用k8s.io名字空间，即ctr/nerdctl -n k8s.io

4.1 推送镜像 ​

1.登录 ​

bash
echo Harbor12345 | nerdctl login --username "admin" --password-stdin  myharbor-minio.com:443

或者

nerdctl login --username "admin" --password Harbor12345 myharbor-minio.com:443

# 退出
nerdctl logout
echo Harbor12345 | nerdctl login --username "admin" --password-stdin  myharbor-minio.com:443

或者

nerdctl login --username "admin" --password Harbor12345 myharbor-minio.com:443

# 退出
nerdctl logout

2.推送 ​

bash
### 推送到Harbor
# --insecure-registry        skips verifying HTTPS certs, and allows falling back to plain HTTP
nerdctl --insecure-registry --namespace=k8s.io push xxx.com/bigdata/nginx:nerctl

ctr --namespace=k8s.io images push xxx.com/bigdata/nginx:nerctl --skip-verify --user admin:Harbor12345

# --namespace=k8s.io 指定命名空间，跟-n一样，不是必须，根据环境而定
# --skip-verify 跳过认证
# --user 指定harbor用户名及密码
### 推送到Harbor
# --insecure-registry        skips verifying HTTPS certs, and allows falling back to plain HTTP
nerdctl --insecure-registry --namespace=k8s.io push xxx.com/bigdata/nginx:nerctl

ctr --namespace=k8s.io images push xxx.com/bigdata/nginx:nerctl --skip-verify --user admin:Harbor12345

# --namespace=k8s.io 指定命名空间，跟-n一样，不是必须，根据环境而定
# --skip-verify 跳过认证
# --user 指定harbor用户名及密码

或者

bash
# 以下两个哪个都可以
# mkdir -p /etc/docker/certs.d/myharbor-minio.com:443
mkdir -p /etc/containerd/certs.d/domain.com:443

cat > /etc/containerd/certs.d/domain.com\:443/hosts.toml <<EOF
server = "https://domain.com"

[host."https://domain.com:443"]
  capabilities = ["pull", "resolve","push"]
  skip_verify = true
  #ca = "ca.crt"   #相对路径
  #ca = "/opt/auth/ca.crt"  #绝对路径
  #ca = ["/opt/auth/ca.crt"]
  ca = ["/etc/containerd/domain.com/ca.crt"]
  #client = [["/opt/auth/nginx.cclinux.cn.crt", "/opt/auth/nginx.cclinux.cn.key"]]
EOF
# 以下两个哪个都可以
# mkdir -p /etc/docker/certs.d/myharbor-minio.com:443
mkdir -p /etc/containerd/certs.d/domain.com:443

cat > /etc/containerd/certs.d/domain.com\:443/hosts.toml <<EOF
server = "https://domain.com"

[host."https://domain.com:443"]
  capabilities = ["pull", "resolve","push"]
  skip_verify = true
  #ca = "ca.crt"   #相对路径
  #ca = "/opt/auth/ca.crt"  #绝对路径
  #ca = ["/opt/auth/ca.crt"]
  ca = ["/etc/containerd/domain.com/ca.crt"]
  #client = [["/opt/auth/nginx.cclinux.cn.crt", "/opt/auth/nginx.cclinux.cn.key"]]
EOF