RKE2安装kubernetes（2）

RKE2安装kubernetes环境准备修改主机名hostnamectl set-hostname rke2-1 && bash系统版本[root@rke2-4 ~]# uname -aLinux rke2-4 3.10.0-693.el7.x86_64 #1 SMP Tue Aug 22 21:09:27 UTC 2017 x86_64 x86_64 x86_64 GNU/Lin

知我几分，

3786人浏览 · 2021-09-14 14:51:34

知我几分， · 2021-09-14 14:51:34 发布

RKE2安装kubernetes（2）

环境准备

修改主机名

hostnamectl set-hostname rke2-1 && bash

系统版本

[root@rke2-4 ~]# uname -a
Linux rke2-4 3.10.0-693.el7.x86_64 #1 SMP Tue Aug 22 21:09:27 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
[root@rke2-4 ~]# cat /etc/redhat-release 
CentOS Linux release 7.4.1708 (Core)

配置hosts解析

cat >> /etc/hosts << EOF
192.168.3.131  rke2-1
192.168.3.132  rke2-2
192.168.3.133  rke2-3
192.168.3.134  rke2-4
EOF

关闭防火墙与selinux

systemctl stop firewalld
systemctl disable firewalld
 
sed -i 's/SELINUX=enforcing/SELINUX=disabled/'  /etc/selinux/config 
setenforce 0

关闭 NetworlManager

systemctl  stop firewalld
systemctl  disable  firewalld

下载常用工具，修改yum源

yum install -y ntpdate vim wget tree httpd-tools telnet  lrzsz  net-tools bridge-utils  unzip

curl -o /etc/yum.repos.d/Centos-7.repo http://mirrors.aliyun.com/repo/Centos-7.repo
curl -o /etc/yum.repos.d/docker-ce.repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo

yum clean all && yum makecache

同步时间

ln -sf /usr/share/zoneinfo/Asia/Shanghai  /etc/localtime
ntpdate -u ntp.aliyun.com && date

修改内核参数

cat <<EOF >>  /etc/sysctl.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_forward=1
EOF

# 自动加载br_netfilter(网络过滤器) 模块 
modprobe br_netfilter

#sysctl命令动态的修改内核参数
sysctl -p /etc/sysctl.conf

# -p：从配置文件“/etc/sysctl.conf”加载内核参数设置
# modprobe 自动处理可载入模块

开放9345端口

TCP的端口，让master与master，master与work节点的通信

安装rke2

参考官方地址：https://docs.rke2.io/install/quickstart/

安装服务器节点

RKE2提供了一个安装脚本，这是一种基于systemd的系统上将其安装为服务的便捷方式。此脚本可以从 https://get.rke2.io 获得，要使用此方法安装RKE2 ,执行以下操作

1.运行安装程序，将rke2-server 服务和rke2 二进制文件安装到机器上
curl -sfL  https://get.rke2.io   | sh -

2.开启rke2-server 服务 
systemctl enable rke2-server

3.启动服务
systemctl start rke2-server

4.查看日志
journalctl -fu rk2-server.service

5.启动后会生成如下文件：
[root@rke2-1 ~]# ll  /var/lib/rancher/rke2/
total 4
drwxr-xr-x. 7 root root 4096 Sep  8 15:54 agent
lrwxrwxrwx  1 root root   58 Sep  8 16:06 bin -> /var/lib/rancher/rke2/data/v1.21.4-rke2r2-3a2840eb67e1/bin
drwxr-xr-x. 3 root root   41 Sep  8 15:54 data
drwx------. 7 root root   99 Sep  8 16:05 server

[root@rke2-1 ~]# cd  /var/lib/rancher/rke2/bin/
[root@rke2-1 bin]# ll
total 276740
-rwxr-xr-x. 1 root root  34902712 Sep  8 15:54 containerd  # 容器运行时
-rwxr-xr-x. 1 root root   6636544 Sep  8 15:54 containerd-shim
-rwxr-xr-x. 1 root root  11068832 Sep  8 15:54 containerd-shim-runc-v1
-rwxr-xr-x. 1 root root  11085408 Sep  8 15:54 containerd-shim-runc-v2
-rwxr-xr-x. 1 root root  23656944 Sep  8 15:54 crictl  # 操作containerdd 命令
-rwxr-xr-x. 1 root root  19651576 Sep  8 15:54 ctr   
-rwxr-xr-x. 1 root root  48239168 Sep  8 15:55 kubectl
-rwxr-xr-x. 1 root root 116760352 Sep  8 15:55 kubelet
-rwxr-xr-x. 1 root root  11044080 Sep  8 15:55 runc  # run容器的一个程序
-rwxr-xr-x. 1 root root    313680 Sep  8 15:55 socat  # 用来给containerd提供端口映射服务

# 生成了一个rke2.yaml 文件，完成kubernetes初始化后生产的admin.config，保存的是整个集群证书的一些信息，所以谁获得rke2.yaml文件就等于获得管理kubernetes集群的权限
[root@rke2-1 bin]# cd /etc/rancher/rke2/
[root@rke2-1 rke2]# ls -l
total 4
-rw-------. 1 root root 2977 Sep  8 16:06 rke2.yaml

[root@rke2-1 rke2]# export KUBECONFIG=/etc/rancher/rke2/rke2.yaml 
[root@rke2-1 rke2]# /var/lib/rancher/rke2/bin/kubectl get node
NAME     STATUS   ROLES                       AGE   VERSION
rke2-1   Ready    control-plane,etcd,master   29m   v1.21.4+rke2r2

[root@rke2-1 rke2]# /var/lib/rancher/rke2/bin/kubectl get pod -A
NAMESPACE     NAME                                                    READY   STATUS      RESTARTS   AGE
kube-system   cloud-controller-manager-rke2-1                         1/1     Running     0          29m
kube-system   etcd-rke2-1                                             1/1     Running     6          29m
kube-system   helm-install-rke2-canal-rtgsc                           0/1     Completed   0          29m
kube-system   helm-install-rke2-coredns-45w76                         0/1     Completed   0          29m
kube-system   helm-install-rke2-ingress-nginx-9gtsl                   0/1     Completed   0          29m
kube-system   helm-install-rke2-metrics-server-vwk77                  0/1     Completed   0          29m
kube-system   kube-apiserver-rke2-1                                   1/1     Running     0          29m
kube-system   kube-controller-manager-rke2-1                          1/1     Running     0          29m
kube-system   kube-proxy-rke2-1                                       1/1     Running     0          29m
kube-system   kube-scheduler-rke2-1                                   1/1     Running     0          29m
kube-system   rke2-canal-xwrfh                                        2/2     Running     0          27m
kube-system   rke2-coredns-rke2-coredns-7bb4f446c-zncz5               1/1     Running     0          27m
kube-system   rke2-coredns-rke2-coredns-autoscaler-7c58bd5b6c-xsh8s   1/1     Running     0          27m
kube-system   rke2-ingress-nginx-controller-b75m9                     1/1     Running     0          24m
kube-system   rke2-metrics-server-5df7d77b5b-d728t                    1/1     Running     0          25m

运行此安装后：

rke2-server 将安装该服务，该rke2-server服务将配置为：在节点重新启动或进程崩溃或被终止后自动重新启动
其他使用程序将安装在/var/lib/rancher/rke2/bin/。他们包括：kubectl，crictl，和 ctr。注意：默认情况下这些不在您的路径下。
两个清理脚本将安装到 /usr/local/bin/rke2 他们是rke2-killall.sh和rke2-uninstall.sh
一个kubeconfig文件将被写入/etc/rancher/rke2/rke2.yaml
可用于注册其他服务或代理节点的令牌将在/var/lib/rancher/rke2/server/node-token

**注意：**如果要添加其他服务器节点，则总数必须为奇数。需要奇数来维持选举人数，有关更多详细信息，请参阅高可用性文档。

手动配置rke2参数和一些设置

注：名字必须是config.yaml

[root@rke2-1 rke2]# cat config.yaml 
token: K105a1bba0a11f93cf7231f0093d16d0d20156f8aa46cb1c5fc8ea8cc6df42a52df::server:5e9d82ee38c21ad5f794c5da30764de7
tls-san:
  - my-kubernetes-domain.com
  - another-kubernetes-domain.com
 
node-name: "rke2-1"

#node-taint:
#  - "CriticalAddinsonly=true:NoExecute"

node-label:
  - "node=Master"
  - "rke2-1=Master"

配置解释

# work 与 master之间通讯需要work提供master上的token 信息
token:
# 创建k8s集群后会生成一系列 tls 证书
tls-san:
  - my-kubernetes-domain.com
  - another-kubernetes-domain.com  # 都是集群的别名，是tls证书所认证的别名或域名，需要认证的别名罗列在这里就可以被tls认证
 
# 节点的名字，会显示在get node 的信息
node-name: "rke2-1"

# 有污点，只做master不做work，没有污点既是master也是work，可以通过kubectl命令修改
#node-taint:
#  - "CriticalAddinsonly=true:NoExecute"

# label 也可以通过kubectl 添加或删除
node-label:
  - "node=Master"
  - "rke2-1=Master"

获取token

# 获取token，填入上面的配置文件
[root@rke2-1 ~]# cat /var/lib/rancher/rke2/server/node-token 
K105a1bba0a11f93cf7231f0093d16d0d20156f8aa46cb1c5fc8ea8cc6df42a52df::server:5e9d82ee38c21ad5f794c5da30764de7

# reload使其生效
[root@rke2-1 rke2]# systemctl daemon-reload

[root@rke2-1 rke2]# systemctl restart rke2-server

[root@rke2-1 rke2]# /var/lib/rancher/rke2/bin/kubectl get node
NAME     STATUS   ROLES                       AGE   VERSION
rke2-1   Ready    control-plane,etcd,master   55m   v1.21.4+rke2r2
[root@rke2-1 rke2]# /var/lib/rancher/rke2/bin/kubectl get pod -A
NAMESPACE     NAME                                                    READY   STATUS      RESTARTS   AGE
kube-system   cloud-controller-manager-rke2-1                         1/1     Running     1          55m
kube-system   etcd-rke2-1                                             1/1     Running     1          55s
kube-system   helm-install-rke2-canal-rtgsc                           0/1     Completed   0          55m
kube-system   helm-install-rke2-coredns-45w76                         0/1     Completed   0          55m
kube-system   helm-install-rke2-ingress-nginx-9gtsl                   0/1     Completed   0          55m
kube-system   helm-install-rke2-metrics-server-99vnw                  0/1     Completed   0          4s
kube-system   kube-apiserver-rke2-1                                   1/1     Running     1          55s
kube-system   kube-controller-manager-rke2-1                          1/1     Running     1          55m
kube-system   kube-proxy-rke2-1                                       1/1     Running     0          55m
kube-system   kube-scheduler-rke2-1                                   1/1     Running     1          55m
kube-system   rke2-canal-xwrfh                                        2/2     Running     0          53m
kube-system   rke2-coredns-rke2-coredns-7bb4f446c-zncz5               1/1     Running     0          53m
kube-system   rke2-coredns-rke2-coredns-autoscaler-7c58bd5b6c-xsh8s   1/1     Running     1          53m
kube-system   rke2-ingress-nginx-controller-b75m9                     1/1     Running     0          50m
kube-system   rke2-metrics-server-5df7d77b5b-d728t                    1/1     Running     1          51m

同样的方式配置其他master节点

scp /etc/rancher/rke2/config.yaml rke2-2:/etc/rancher/rke2/
scp /etc/rancher/rke2/config.yaml rke2-4:/etc/rancher/rke2/

分别修改：node-name

并添加如下：# 需要与server1 产生关系
server: https://192.168.3.131:9345

# rk2-2如下：
[root@rke2-2 rke2]# cat config.yaml 
server: https://192.168.3.131:9345
token: K105a1bba0a11f93cf7231f0093d16d0d20156f8aa46cb1c5fc8ea8cc6df42a52df::server:5e9d82ee38c21ad5f794c5da30764de7
tls-san:
  - my-kubernetes-domain.com
  - another-kubernetes-domain.com
 
node-name: "rke2-2"

#node-taint:
#  - "CriticalAddinsonly=true:NoExecute"

node-label:
  - "node=Master"
  - "rke2-2=Master"

# rke2-4
[root@rke2-4 rke2]# cat /etc/rancher/rke2/config.yaml 
server: https://192.168.3.131:9345
token: K105a1bba0a11f93cf7231f0093d16d0d20156f8aa46cb1c5fc8ea8cc6df42a52df::server:5e9d82ee38c21ad5f794c5da30764de7
tls-san:
  - my-kubernetes-domain.com
  - another-kubernetes-domain.com
 
node-name: "rke2-4"

#node-taint:
#  - "CriticalAddinsonly=true:NoExecute"

node-label:
  - "node=Master"
  - "rke2-4=Master"


# 加载后使其生效
systemctl daemon-reload
systemctl restart rke2-server

# 再次查看node
[root@rke2-1 rke2]# /var/lib/rancher/rke2/bin/kubectl -n kube-system get node
NAME     STATUS   ROLES                       AGE    VERSION
rke2-1   Ready    control-plane,etcd,master   37m    v1.21.4+rke2r3
rke2-2   Ready    control-plane,etcd,master   23m    v1.21.4+rke2r3
rke2-4   Ready    control-plane,etcd,master   118s   v1.21.4+rke2r3

安装worker节点

1.运行安装程序，将rke2-agent 服务 和 rke2 二进制文件安装到机器上
curl -sfL https://get.rke2.io | INSTALL_RKE2_TYPE="agent" sh -

2.设置开机启动
systemctl enable rke2-agent.service

3.配置rke2-agent服务
mkdir -p /etc/rancher/rke2
vim /etc/rancher/rke2/config.yaml

config.yaml 的内容
server: /https://<server>:9345
tonken: <token from server node>

# rke2-3
scp /etc/rancher/rke2/config.yaml rke2-3:/etc/rancher/rke2/

[root@rke2-3 rke2]# cat config.yaml 
server: https://192.168.3.131:9345
token: K105a1bba0a11f93cf7231f0093d16d0d20156f8aa46cb1c5fc8ea8cc6df42a52df::server:5e9d82ee38c21ad5f794c5da30764de7
node-name: "rke2-3"
node-label:
  - "node=worker"
  - "rke2-3=worker"
  
# 加载后使其生效
systemctl daemon-reload
注：该 rke2 server 进程在端口上监听 9345 要注册的新节点。kubernetes API 6443 仍然像往常一样在port 上提供服务。

4.启动服务
systemctl start rke2-agent.service

[root@rke2-3 ~]# systemctl status rke2-agent.service
● rke2-agent.service - Rancher Kubernetes Engine v2 (agent)
   Loaded: loaded (/usr/lib/systemd/system/rke2-agent.service; disabled; vendor preset: disabled)
   Active: active (running) since Mon 2021-09-13 15:46:35 CST; 12s ago
     Docs: https://github.com/rancher/rke2#readme


5.查看日志
journalctl -fu rke2-agent 

6.查看node
[root@rke2-1 rke2]# /var/lib/rancher/rke2/bin/kubectl -n kube-system get node -w
NAME     STATUS   ROLES                       AGE     VERSION
rke2-1   Ready    control-plane,etcd,master   89m     v1.21.4+rke2r3
rke2-2   Ready    control-plane,etcd,master   74m     v1.21.4+rke2r3
rke2-3   Ready    <none>                      6m24s   v1.21.4+rke2r3
rke2-4   Ready    control-plane,etcd,master   53m     v1.21.4+rke2r3

**注意：**每台机器必须有一个唯一的主机名。如果您的机器没有唯一的主机名，请node-name在config.yaml文件中设置参数并为每个节点提供一个具有有效且唯一主机名的值。

要阅读有关 config.yaml 文件的更多信息，请参阅安装选项文档。

其他


[root@rke2-1 rke2]# ls -l /run/k3s/containerd/containerd.sock
srw-rw---- 1 root root 0 Sep 13 14:31 /run/k3s/containerd/containerd.sock

|
[root@rke2-1 rke2]# /var/lib/rancher/rke2/bin/crictl  --runtime-endpoint=unix:///run/k3s/containerd/containerd.sock ps
CONTAINER           IMAGE               CREATED             STATE               NAME                            ATTEMPT             POD ID
b210741aa5491       7589738b9ae11       2 hours ago         Running             coredns                         0                   643b9ef40c4b1
3d4c3184d1ff3       5aa19aa313a9b       2 hours ago         Running             autoscaler                      5                   6724f540c188c
c4ef09c03a22d       5d05c5a9b5533       2 hours ago         Running             metrics-server                  1                   8dfba29b6803a
686f9ae82f6d9       55e81dd7316be       2 hours ago         Running             cloud-controller-manager        2                   66ecf8d51a225
c5fbcfae8def6       9e2f766bd35d6       2 hours ago         Running             kube-scheduler                  2                   87a6b265d5da2
7f740352a479f       9e2f766bd35d6       2 hours ago         Running             kube-controller-manager         2                   c19561eddcf4b
0eb2344d4d26b       9e2f766bd35d6       2 hours ago         Running             kube-apiserver                  1                   afe17cea25ea0
929a20b5f356b       271c0a695260e       2 hours ago         Running             etcd                            1                   c46cf018a870a
4de1d88f8f423       fffb9e128464f       2 hours ago         Running             rke2-ingress-nginx-controller   0                   b82a44372ee28
2a95f5d414d64       7589738b9ae11       2 hours ago         Running             coredns                         0                   aadde4683420b
e30a24115a4c7       366c64051af85       2 hours ago         Running             kube-flannel                    0                   1a11ecf1b650c
d4aedfaf8ee17       736cae9d947ba       2 hours ago         Running             calico-node                     0                   1a11ecf1b650c
044e6e56b933c       9e2f766bd35d6       2 hours ago         Running             kube-proxy                      1         


# 命令太长
[root@rke2-1 rke2]# mkdir -p /etc/rancher/rke2/.kube
[root@rke2-1 rke2]# ln -s /etc/rancher/rke2/rke2.yaml ~/.kube/config
[root@rke2-1 rke2]# ll  ~/.kube/config
lrwxrwxrwx 1 root root 27 Sep 13 16:36 /root/.kube/config -> /etc/rancher/rke2/rke2.yaml

[root@rke2-1 rke2]# ln -s /var/lib/rancher/rke2/agent/etc/crictl.yaml /etc/crictl.yaml
[root@rke2-1 rke2]# chmod 600 ~/.kube/config

[root@rke2-1 rke2]# /var/lib/rancher/rke2/bin/crictl  ps
CONTAINER           IMAGE               CREATED             STATE               NAME                            ATTEMPT             POD ID
b210741aa5491       7589738b9ae11       2 hours ago         Running             coredns                         0                   643b9ef40c4b1
3d4c3184d1ff3       5aa19aa313a9b       2 hours ago         Running             autoscaler                      5                   6724f540c188c
c4ef09c03a22d       5d05c5a9b5533       2 hours ago         Running             metrics-server                  1                   8dfba29b6803a
686f9ae82f6d9       55e81dd7316be       2 hours ago         Running             cloud-controller-manager        2                   66ecf8d51a225
c5fbcfae8def6       9e2f766bd35d6       2 hours ago         Running             kube-scheduler                  2                   87a6b265d5da2
7f740352a479f       9e2f766bd35d6       2 hours ago         Running             kube-controller-manager         2                   c19561eddcf4b
0eb2344d4d26b       9e2f766bd35d6       2 hours ago         Running             kube-apiserver                  1                   afe17cea25ea0
929a20b5f356b       271c0a695260e       2 hours ago         Running             etcd                            1                   c46cf018a870a
4de1d88f8f423       fffb9e128464f       2 hours ago         Running             rke2-ingress-nginx-controller   0                   b82a44372ee28
2a95f5d414d64       7589738b9ae11       2 hours ago         Running             coredns                         0                   aadde4683420b
e30a24115a4c7       366c64051af85       2 hours ago         Running             kube-flannel                    0                   1a11ecf1b650c
d4aedfaf8ee17       736cae9d947ba       2 hours ago         Running             calico-node                     0                   1a11ecf1b650c
044e6e56b933c       9e2f766bd35d6       2 hours ago         Running             kube-proxy

配置自己的镜像仓库地址

# 在/etc/rancher/rke2/ 下定义一个 registries.yaml
mirrors:
  myregistry.com:
    endpoint:
     - "https://myregistry.com:5000"
configs:
  "myregistry.com:5000"
    auth:
       username: xxxx
       password: xxxx
    tls:
       cert_file: /path
       key_file:
       ca_file:

Rke2 升级的问题

# server 升级
再执行
curl -sfL https://get.rke2.io | sh -

# worker 升级
curl -sfL https://get.rke2.io | INSTALL_RKE2_TYPE="agent" sh -


#server指定版本升级
curl -sfL https://get.rke2.io | INSTALL_RKE2_VERSION=vx.y.z sh -

# worker 指定版升级
https://get.rke2.io | INSTALL_RKE2_TYPE="agent"   INSTALL_RKE2_VERSION=vx.y.z sh -

ETCD的问题

# rke2 上本身启动了一个ETCD快照功能，产生的快照文件在 如下目录
[root@rke2-1 ~]# ls -l /var/lib/rancher/rke2/server/db/snapshots/
total 0

# 默认每12个小时生成当前机器的etcd快照，仅限有ETCD的master节点； 在每个master节点配置


# 可以更改备份时间，在 config.yaml 添加如下两行：work节点无需添加 快照参数。
vi  /etc/rancher/rke2/config.yaml
etcd-snapshot-retention: 2
etcd-snapshot-schedule-cron: '*/2 * * * *'
kubelet-arg:
  - "eviction-hard=nodefs.available<1%,memory.available<10Mi"
  - "eviction-soft-grace-period=nodefs.available=30s,imagefs.available=30s"
  - "eviction-soft=nodefs.available<5%,imagefs.available<1%"

注释：
# 快照文件个数，只保存两个，删除旧的保存新的
etcd-snapshot-retention: 2
# 与定时任务写法一样，分时日月周; default 是 '* */12 * * * '
etcd-snapshot-schedule-cron: '*/10 * * * *'
# 自定义快照文件存放位置
etcd-snapshot-dir: /xx/xxx/xxx

# 自定义垃圾回收机制，添加到所所节点 
kubelet-arg:
  - "eviction-hard=nodefs.available<1%,memory.available<10Mi"  # 硬策略
  - "eviction-soft-grace-period=nodefs.available=30s,imagefs.available=30s" # 硬策略
  - "eviction-soft=nodefs.available<5%,imagefs.available<1%"    # 软策略，可用文件系统小于百分之五，可用镜像文件系统小于1% 开始回收


# reload 使其生效
systemctl daemon-reload
systemctl restart rke2-server

 
# 查看是否生效
ps -ef | grep -i kubelet

# 默认快照存储位置
ls /var/lib/rancher/rke2/server/db/snapshots/

[root@rke2-1 ~]#  ls /var/lib/rancher/rke2/server/db/snapshots/
etcd-snapshot-rke2-1-1631600520  etcd-snapshot-rke2-1-1631600640

其他配置参考：https://docs.rke2.io/backup_restore/#options

Cloudpods

开源、云原生的融合云平台

更多推荐

面向未来的 IT 基础设施管理架构——融合云（Unified IaaS）

随着数字化时代的到来，IT系统已成为人类社会正常运转不可或缺的组成部分。不远的未来，智能制造，5G和人工智能等技术将成为推动生产力发展的重要引擎，人类社会将面临前所未有的全面彻底的数字化浪潮。IT基础设施作为IT系统运行的平台和载体，是实现数字化的基石。在这场数字化浪潮中，企业必须积极拥抱云计算技术，采用符合技术发展趋势、面向未来的IT基础构架，才能在未来的竞争中赢得先机。一、云计算历经十余年

Cloudpods

Cloudpods负载均衡的功能介绍

作者:周有松今天的内容会从以下几个方面展开：负载均衡产品简介。主要介绍负载均衡作为一个云上产品，它的功能模型是怎样的，日常使用中会遇到的业务词汇负载均衡的功能与典型应用场景。这部分主要结合业务词汇，对负载均衡服务中常见的一些功能选项进行介绍，并举例介绍一些典型的应用场景最后，我们做一下总结，讨论一下负载均衡产品相比传统方式的优点一、产品简介 1. 以NGINX为例提到负载均衡，我们以

Cloudpods

使用Linux vfio将Nvidia GPU透传给QEMU虚拟机

Linux 上虚拟机 GPU 透传需要使用 vfio 的方式。主要是因为在 vfio 方式下对虚拟设备的权限和 DMA 隔离上做的更好。但是这么做也有个缺点，这个物理设备在主机和其他虚拟机都不能使用了。 qemu 直接使用物理设备本身命令行是很简单的，关键在于事先在主机上对系统、内核和物理设备的一些配置。单纯从 qemu 的命令行来看，其实和普通虚拟机启动就差了最后那个-device的选项。这