k8s中使用基于nfs的storageclass
Gitlab地址:https://github.com/kubernetes-retired/external-storage/tree/master/nfs-client1.搭建nfs服务(1)nfs服务端的安装配置# 安装nfs~]# yum -y install nfs-utils rpcbind# 创建nfs共享目录及设置权限~]# mkdir /data/k8sdata -p~]# ch
Gitlab地址:https://github.com/kubernetes-retired/external-storage/tree/master/nfs-client
1.搭建nfs服务
(1)nfs服务端的安装配置
# 安装nfs
~]# yum -y install nfs-utils rpcbind
# 创建nfs共享目录及设置权限
~]# mkdir /data/k8sdata -p
~]# chmod 755 /data/k8sdata -R
# 配置nfs
~]# cat /etc/exports
/data/k8sdata *(rw,sync,no_root_squash)
nfs主要配置说明:
* :表示任何⼈都有权限连接,这儿也可以配置为⼀个⽹段,⼀个 IP,或者域名
rw:权限为读写
sync:表示⽂件同时写⼊硬盘和内存
no_root_squash:当登录 NFS 主机使⽤共享⽬录的使⽤者是 root 时,其权限将被转换成为匿名使⽤者,通常它的 UID 与 GID,都会变成 nobody 身份
# 启动nfs,需要先启动rpcbind再启动nfs
~]# systemctl start rpcbind
~]# systemctl enable rpcbind
~]# systemctl status rpcbind
● rpcbind.service - RPC bind service
Loaded: loaded (/usr/lib/systemd/system/rpcbind.service; enabled; vendor preset: enabled)
Active: active (running) since Sun 2020-08-09 14:53:03 CST; 50s ago
Main PID: 15672 (rpcbind)
CGroup: /system.slice/rpcbind.service
└─15672 /sbin/rpcbind -w
~]# systemctl start nfs
~]# systemctl enable nfs
Created symlink from /etc/systemd/system/multi-user.target.wants/nfs-server.service to /usr/lib/systemd/system/nfs-server.service.
~]# systemctl status nfs
● nfs-server.service - NFS server and services
Loaded: loaded (/usr/lib/systemd/system/nfs-server.service; enabled; vendor preset: disabled)
Drop-In: /run/systemd/generator/nfs-server.service.d
└─order-with-mounts.conf
Active: active (exited) since Sun 2020-08-09 14:57:22 CST; 17s ago
Main PID: 16369 (code=exited, status=0/SUCCESS)
CGroup: /system.slice/nfs-server.service
# 查看具体的挂载权限
~]# cat /var/lib/nfs/etab
/data/k8sdata *(rw,sync,wdelay,hide,nocrossmnt,secure,no_root_squash,no_all_squash,no_subtree_check,secure_locks,acl,no_pnfs,anonuid=65534,anongid=65534,sec=sys,rw,secure,no_root_squash,no_all_squash)
(2)nfs客户端的安装配置
Nfs客户端需要安装在使用nfs的各个节点上,即k8s集群的各个node节点上。
# 在客户端上安装启动nfs
~]# yum -y install nfs-utils rpcbind
~]# systemctl start rpcbind
~]# systemctl enable rpcbind
~]# systemctl start nfs
~]# systemctl enable nfs
# 查看nfs共享
~]# showmount -e 192.168.16.133
Export list for 192.168.16.133:
/data/k8sdata *
2、创建基于nfs的storageclass
(1)storageclass
要使⽤StorageClass,我们就得安装对应的⾃动配置程序,⽐如我们这⾥存储后端使⽤的是 nfs,那么我们就需要使⽤到⼀个 nfs-client的⾃动配置程序,我们也叫它 Provisioner,这个程序帮我们在配置好的nfs 服务器上⾃动创建持久卷,也就是⾃动帮我们创建PV。⾃动创建的PV以${namespace}-${pvcName}-${pvName}这样的命名格式创建在NFS服务器上的共享数据⽬录中,⽽当这个PV被回收后会以archieved-${namespace}-${pvcName}-${pvName}这样的命名格式存在NFS 服务器上。
(2)、安装配置nfs storageclass
# 拉取github相关的资源至本地
~]# git clone https://github.com/kubernetes-retired/external-storage.git
~]# cd external-storage/tree/master/nfs-client/deploy/
# 将deployment.yaml配置文件中的nfs服务器和路径修改为自己nfs服务器和路径
]# cat deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: nfs-client-provisioner
labels:
app: nfs-client-provisioner
# replace with namespace where provisioner is deployed
namespace: default
spec:
replicas: 1
strategy:
type: Recreate
selector:
matchLabels:
app: nfs-client-provisioner
template:
metadata:
labels:
app: nfs-client-provisioner
spec:
serviceAccountName: nfs-client-provisioner
containers:
- name: nfs-client-provisioner
image: quay.io/external_storage/nfs-client-provisioner:latest
volumeMounts:
- name: nfs-client-root
mountPath: /persistentvolumes
env:
- name: PROVISIONER_NAME
value: fuseim.pri/ifs
- name: NFS_SERVER
value: 192.168.16.133
- name: NFS_PATH
value: /data/k8sdata
volumes:
- name: nfs-client-root
nfs:
server: 192.168.16.133
path: /data/k8sdata
# 修改完成后创建该deployment控制器
deploy]# kubectl apply -f deployment.yaml
deployment.apps/nfs-client-provisioner created
# 创建sa,并绑定相应的权限
deploy]# kubectl apply -f rbac.yaml
serviceaccount/nfs-client-provisioner created
# 创建storageclass
deploy]# kubectl apply -f class.yaml
storageclass.storage.k8s.io/managed-nfs-storage created
# 查看创建的资源
deploy]# kubectl get pods
NAME READY STATUS RESTARTS AGE
nfs-client-provisioner-7d4d65c7b9-l487r 1/1 Running 0 8s
deploy]# kubectl get storageclass
NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE
managed-nfs-storage fuseim.pri/ifs Delete Immediate false 22h
3、使用基于nfs的storageclass
(1)创建pvc调用storageclass动态提供pv
创建了storageclass后,创建pvc之前不需要创建相关的pv,只需调用storageclass提供pv的动态供给。
# 要创建的pvc内容如下
deploy]# cat test-claim.yaml
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: test-claim
annotations:
volume.beta.kubernetes.io/storage-class: "managed-nfs-storage"
spec:
accessModes:
- ReadWriteMany
resources:
requests:
storage: 1Mi
# 创建pvc
deploy]# kubectl apply -f test-claim.yaml
persistentvolumeclaim/test-claim created
deploy]# kubectl get pvc
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
test-claim Bound pvc-15531f5b-b439-43cc-82a8-7152a5ffd50f 1Mi RWX managed-nfs-storage 5s
(2)在pod控制器中使用storageclass
在实际⼯作中,使⽤ StorageClass 更多的是StatefulSet控制器管理的pod,StatefulSet控制器中我们也可以通过⼀个volumeClaimTemplates属性直接使⽤ StorageClass。实际上volumeClaimTemplates下⾯就是⼀个PVC对象的模板,类似于StatefulSet下⾯的template,实际上就是⼀个 Pod 的模板,我们用这种模板就可以动态的去创建pvc对象了。
# statefulset控制器内容
]# cat statefulset-nfs-storageclass.yaml
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: nfs-test
spec:
serviceName: "nginx"
replicas: 2
selector:
matchLabels:
app: nfs-test
template:
metadata:
labels:
app: nfs-test
spec:
terminationGracePeriodSeconds: 5
containers:
- name: nginx
image: nginx:1.18.0
ports:
- name: http
containerPort: 80
volumeMounts:
- name: web
mountPath: /usr/share/nginx/html
volumeClaimTemplates:
- metadata:
name: web
annotations:
volume.beta.kubernetes.io/storage-class: "managed-nfs-storage"
spec:
accessModes: ["ReadWriteOnce"]
resources:
requests:
storage: 1Gi
# 创建
]# kubectl apply -f statefulset-nfs-storageclass.yaml
# 查看创建的pvc
]# kubectl get pvc
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
web-nfs-test-0 Bound pvc-f956a6c8-5382-4a0a-a7ed-d19ac07d83c2 1Gi RWO managed-nfs-storage 2m2s
web-nfs-test-1 Bound pvc-2a25a995-37db-4173-be1e-7a94ea4f9e8f 1Gi RWO managed-nfs-storage 119s
# 创建pvc后会在nfs上生成相关的文件
~]# ll /data/k8sdata/
total 0
drwxrwxrwx 2 root root 6 Aug 11 22:06 default-test-claim-pvc-15531f5b-b439-43cc-82a8-7152a5ffd50f
drwxrwxrwx 2 root root 6 Aug 11 22:59 default-web-nfs-test-0-pvc-f956a6c8-5382-4a0a-a7ed-d19ac07d83c2
drwxrwxrwx 2 root root 6 Aug 11 22:59 default-web-nfs-test-1-pvc-2a25a995-37db-4173-be1e-7a94ea4f9e8f
更多推荐
所有评论(0)