yaml方式部署Ingress Controller + Ingress示例

转自yaml方式部署Ingress Controller + Ingress示例概要kubernetes 部署连接：Installation Guide - NGINX Ingress Controller本案例使用yaml的方式部署，分为3个步骤：部署Ingress Controller创建 Ingress 策略客户端通过 Ingress Controller 访问后端 webapp 服务1.部

昜羊

5143人浏览 · 2022-05-12 12:50:51

昜羊 · 2022-05-12 12:50:51 发布

概要

kubernetes 部署连接：Installation Guide - NGINX Ingress Controller
本案例使用yaml的方式部署，分为3个步骤：

部署Ingress Controller
创建 Ingress 策略
客户端通过 Ingress Controller 访问后端 webapp 服务

1. 部署 Ingress Controller

ingress.yaml 内容如下：

---
apiVersion: v1
kind: Namespace
metadata:
  name: nginx-ingress


---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: nginx-ingress
  namespace: nginx-ingress


---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: nginx-ingress
rules:
- apiGroups:
  - ""
  resources:
  - services
  - endpoints
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - ""
  resources:
  - secrets
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - ""
  resources:
  - configmaps
  verbs:
  - get
  - list
  - watch
  - update
  - create
- apiGroups:
  - ""
  resources:
  - pods
  verbs:
  - list
  - watch
- apiGroups:
  - ""
  resources:
  - events
  verbs:
  - create
  - patch
  - list
- apiGroups:
  - extensions
  resources:
  - ingresses
  verbs:
  - list
  - watch
  - get
- apiGroups:
  - "extensions"
  resources:
  - ingresses/status
  verbs:
  - update
- apiGroups:
  - k8s.nginx.org
  resources:
  - virtualservers
  - virtualserverroutes
  - globalconfigurations
  - transportservers
  - policies
  verbs:
  - list
  - watch
  - get
- apiGroups:
  - k8s.nginx.org
  resources:
  - virtualservers/status
  - virtualserverroutes/status
  verbs:
  - update
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: nginx-ingress
subjects:
- kind: ServiceAccount
  name: nginx-ingress
  namespace: nginx-ingress
roleRef:
  kind: ClusterRole
  name: nginx-ingress
  apiGroup: rbac.authorization.k8s.io


---
apiVersion: v1
kind: Secret
metadata:
  name: default-server-secret
  namespace: nginx-ingress
type: Opaque
data:
  tls.crt: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUN2akNDQWFZQ0NRREFPRjl0THNhWFhEQU5CZ2txaGtpRzl3MEJBUXNGQURBaE1SOHdIUVlEVlFRRERCWk8KUjBsT1dFbHVaM0psYzNORGIyNTBjbTlzYkdWeU1CNFhEVEU0TURreE1qRTRNRE16TlZvWERUSXpNRGt4TVRFNApNRE16TlZvd0lURWZNQjBHQTFVRUF3d1dUa2RKVGxoSmJtZHlaWE56UTI5dWRISnZiR3hsY2pDQ0FTSXdEUVlKCktvWklodmNOQVFFQkJRQURnZ0VQQURDQ0FRb0NnZ0VCQUwvN2hIUEtFWGRMdjNyaUM3QlBrMTNpWkt5eTlyQ08KR2xZUXYyK2EzUDF0azIrS3YwVGF5aGRCbDRrcnNUcTZzZm8vWUk1Y2Vhbkw4WGM3U1pyQkVRYm9EN2REbWs1Qgo4eDZLS2xHWU5IWlg0Rm5UZ0VPaStlM2ptTFFxRlBSY1kzVnNPazFFeUZBL0JnWlJVbkNHZUtGeERSN0tQdGhyCmtqSXVuektURXUyaDU4Tlp0S21ScUJHdDEwcTNRYzhZT3ExM2FnbmovUWRjc0ZYYTJnMjB1K1lYZDdoZ3krZksKWk4vVUkxQUQ0YzZyM1lma1ZWUmVHd1lxQVp1WXN2V0RKbW1GNWRwdEMzN011cDBPRUxVTExSakZJOTZXNXIwSAo1TmdPc25NWFJNV1hYVlpiNWRxT3R0SmRtS3FhZ25TZ1JQQVpQN2MwQjFQU2FqYzZjNGZRVXpNQ0F3RUFBVEFOCkJna3Foa2lHOXcwQkFRc0ZBQU9DQVFFQWpLb2tRdGRPcEsrTzhibWVPc3lySmdJSXJycVFVY2ZOUitjb0hZVUoKdGhrYnhITFMzR3VBTWI5dm15VExPY2xxeC9aYzJPblEwMEJCLzlTb0swcitFZ1U2UlVrRWtWcitTTFA3NTdUWgozZWI4dmdPdEduMS9ienM3bzNBaS9kclkrcUI5Q2k1S3lPc3FHTG1US2xFaUtOYkcyR1ZyTWxjS0ZYQU80YTY3Cklnc1hzYktNbTQwV1U3cG9mcGltU1ZmaXFSdkV5YmN3N0NYODF6cFErUyt1eHRYK2VBZ3V0NHh3VlI5d2IyVXYKelhuZk9HbWhWNThDd1dIQnNKa0kxNXhaa2VUWXdSN0diaEFMSkZUUkk3dkhvQXprTWIzbjAxQjQyWjNrN3RXNQpJUDFmTlpIOFUvOWxiUHNoT21FRFZkdjF5ZytVRVJxbStGSis2R0oxeFJGcGZnPT0KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo=
  tls.key: LS0tLS1CRUdJTiBSU0EgUFJJVkFURSBLRVktLS0tLQpNSUlFcEFJQkFBS0NBUUVBdi91RWM4b1JkMHUvZXVJTHNFK1RYZUprckxMMnNJNGFWaEMvYjVyYy9XMlRiNHEvClJOcktGMEdYaVN1eE9ycXgrajlnamx4NXFjdnhkenRKbXNFUkJ1Z1B0ME9hVGtIekhvb3FVWmcwZGxmZ1dkT0EKUTZMNTdlT1l0Q29VOUZ4amRXdzZUVVRJVUQ4R0JsRlNjSVo0b1hFTkhzbysyR3VTTWk2Zk1wTVM3YUhudzFtMApxWkdvRWEzWFNyZEJ6eGc2clhkcUNlUDlCMXl3VmRyYURiUzc1aGQzdUdETDU4cGszOVFqVUFQaHpxdmRoK1JWClZGNGJCaW9CbTVpeTlZTW1hWVhsMm0wTGZzeTZuUTRRdFFzdEdNVWozcGJtdlFmazJBNnljeGRFeFpkZFZsdmwKMm82MjBsMllxcHFDZEtCRThCay90elFIVTlKcU56cHpoOUJUTXdJREFRQUJBb0lCQVFDZklHbXowOHhRVmorNwpLZnZJUXQwQ0YzR2MxNld6eDhVNml4MHg4Mm15d1kxUUNlL3BzWE9LZlRxT1h1SENyUlp5TnUvZ2IvUUQ4bUFOCmxOMjRZTWl0TWRJODg5TEZoTkp3QU5OODJDeTczckM5bzVvUDlkazAvYzRIbjAzSkVYNzZ5QjgzQm9rR1FvYksKMjhMNk0rdHUzUmFqNjd6Vmc2d2szaEhrU0pXSzBwV1YrSjdrUkRWYmhDYUZhNk5nMUZNRWxhTlozVDhhUUtyQgpDUDNDeEFTdjYxWTk5TEI4KzNXWVFIK3NYaTVGM01pYVNBZ1BkQUk3WEh1dXFET1lvMU5PL0JoSGt1aVg2QnRtCnorNTZud2pZMy8yUytSRmNBc3JMTnIwMDJZZi9oY0IraVlDNzVWYmcydVd6WTY3TWdOTGQ5VW9RU3BDRkYrVm4KM0cyUnhybnhBb0dCQU40U3M0ZVlPU2huMVpQQjdhTUZsY0k2RHR2S2ErTGZTTXFyY2pOZjJlSEpZNnhubmxKdgpGenpGL2RiVWVTbWxSekR0WkdlcXZXaHFISy9iTjIyeWJhOU1WMDlRQ0JFTk5jNmtWajJTVHpUWkJVbEx4QzYrCk93Z0wyZHhKendWelU0VC84ajdHalRUN05BZVpFS2FvRHFyRG5BYWkyaW5oZU1JVWZHRXFGKzJyQW9HQkFOMVAKK0tZL0lsS3RWRzRKSklQNzBjUis3RmpyeXJpY05iWCtQVzUvOXFHaWxnY2grZ3l4b25BWlBpd2NpeDN3QVpGdwpaZC96ZFB2aTBkWEppc1BSZjRMazg5b2pCUmpiRmRmc2l5UmJYbyt3TFU4NUhRU2NGMnN5aUFPaTVBRHdVU0FkCm45YWFweUNweEFkREtERHdObit3ZFhtaTZ0OHRpSFRkK3RoVDhkaVpBb0dCQUt6Wis1bG9OOTBtYlF4VVh5YUwKMjFSUm9tMGJjcndsTmVCaWNFSmlzaEhYa2xpSVVxZ3hSZklNM2hhUVRUcklKZENFaHFsV01aV0xPb2I2NTNyZgo3aFlMSXM1ZUtka3o0aFRVdnpldm9TMHVXcm9CV2xOVHlGanIrSWhKZnZUc0hpOGdsU3FkbXgySkJhZUFVWUNXCndNdlQ4NmNLclNyNkQrZG8wS05FZzFsL0FvR0FlMkFVdHVFbFNqLzBmRzgrV3hHc1RFV1JqclRNUzRSUjhRWXQKeXdjdFA4aDZxTGxKUTRCWGxQU05rMXZLTmtOUkxIb2pZT2pCQTViYjhibXNVU1BlV09NNENoaFJ4QnlHbmR2eAphYkJDRkFwY0IvbEg4d1R0alVZYlN5T294ZGt5OEp0ek90ajJhS0FiZHd6NlArWDZDODhjZmxYVFo5MWpYL3RMCjF3TmRKS2tDZ1lCbyt0UzB5TzJ2SWFmK2UwSkN5TGhzVDQ5cTN3Zis2QWVqWGx2WDJ1VnRYejN5QTZnbXo5aCsKcDNlK2JMRUxwb3B0WFhNdUFRR0xhUkcrYlNNcjR5dERYbE5ZSndUeThXczNKY3dlSTdqZVp2b0ZpbmNvVlVIMwphdmxoTUVCRGYxSjltSDB5cDBwWUNaS2ROdHNvZEZtQktzVEtQMjJhTmtsVVhCS3gyZzR6cFE9PQotLS0tLUVORCBSU0EgUFJJVkFURSBLRVktLS0tLQo=


---
kind: ConfigMap
apiVersion: v1
metadata:
  name: nginx-config
  namespace: nginx-ingress
data:


---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-ingress
  namespace: nginx-ingress
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx-ingress
  template:
    metadata:
      labels:
        app: nginx-ingress
    spec:
      nodeSelector:
        role: ingress-nginx-controller
      serviceAccountName: nginx-ingress
      containers:
      - image: nginx/nginx-ingress:1.7.2
        imagePullPolicy: IfNotPresent
        name: nginx-ingress
        ports:
        - name: http
          containerPort: 80
          hostPort: 80
        - name: https
          containerPort: 443
          hostPort: 443
        securityContext:
          allowPrivilegeEscalation: true
          runAsUser: 101 #nginx
          capabilities:
            drop:
            - ALL
            add:
            - NET_BIND_SERVICE
        env:
        - name: POD_NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
        - name: POD_NAME
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        args:
          - -nginx-configmaps=$(POD_NAMESPACE)/nginx-config
          - -default-server-tls-secret=$(POD_NAMESPACE)/default-server-secret

前提条件：需要在部署的 node 上打标签

[root@master1 ~]# kubectl label node node2 role=ingress-nginx-controller
node/node2 labeled

不然部署的时候，会报如下事件：

[root@master1 ~]# kubectl describe pods  --namespace=nginx-ingress nginx-ingress-75c88594dc-j687p
...
Events:
  Type     Reason            Age   From               Message
  ----     ------            ----  ----               -------
  Warning  FailedScheduling  11m   default-scheduler  0/4 nodes are available: 4 node(s) didn't match node selector.
  Warning  FailedScheduling  11m   default-scheduler  0/4 nodes are available: 4 node(s) didn't match node selector.`

通过 kubectl create 命令创建 nginx-ingres-controller:

[root@master1 ~]# kubectl apply -f  ingress.yaml 
namespace/nginx-ingress created
serviceaccount/nginx-ingress created
clusterrole.rbac.authorization.k8s.io/nginx-ingress created
clusterrolebinding.rbac.authorization.k8s.io/nginx-ingress created
secret/default-server-secret created
configmap/nginx-config created
deployment.apps/nginx-ingress created
ingress.networking.k8s.io/mywebsite-ingress created

查看 nginx-ingress-controller 容器，确认其正常运行：

[root@master1 ~]# kubectl get pods -n nginx-ingress -o wide
NAME                             READY   STATUS    RESTARTS   AGE     IP            NODE    NOMINATED NODE   READINESS GATES
nginx-ingress-75c88594dc-42256   1/1     Running   0          2m48s   10.0.104.42   node2   <none>           <none>

用 curl 访问 Nginx Ingress Controller 所在的宿主机的 80 端口，验证其服务是否正常，在没有配置后端服务时 Nginx 会返回 404 应答：

[root@master1 ~]# curl http://node2
<html>
<head><title>404 Not Found</title></head>
<body>
<center><h1>404 Not Found</h1></center>
<hr><center>nginx/1.19.0</center>
</body>
</html>

2. 创建 Ingress 策略

本例对域名 mywebsite.com 的访问设置 Ingress 策略，定义对其 /demo 路径的访问转发到后端 myweb Service 的规则：

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: mywebsite-ingress
spec:
  rules:
  - host: mywebsite.com
    http:
      paths:
      - path: /demo
        pathType: ImplementationSpecific
        backend:
          service:
            name: myweb
            port:
              number: 8080

通过该 Ingress 定义设置的效果：客户端对目标地址 http://mywebsite.com/demo 的访问将被转发到集群内的服务“myweb”上，完整的URL为：“http://myweb:8080/demo”。
在 Ingress 策略生效之前，需要先确保 webapp 服务正确运行。同时注意 Ingress 中对路径的定义需要与后端 myweb 服务提供的访问路径一直，否则将被转发到一个不存在的路径上引发错误。
创建两个测试yaml文件，分别是 mysql.yaml 和 myweb.yaml
mysql.yaml 内容如下：

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: mysql
  name: mysql
spec:
  replicas: 1
  selector:
    matchLabels:
      app: mysql
  template:
    metadata:
      labels:
        app: mysql
    spec:
      hostname: mysql
      subdomain: database
      containers:
      - image: mysql:5.7
        name: mysql
        ports:
        - containerPort: 3306
        env:
        - name: MYSQL_ROOT_PASSWORD
          value: "123456"


---
apiVersion: v1
kind: Service
metadata:
  name: database
spec:
  ports:
    - port: 3306
  selector:
    app: mysql

myweb.yaml 内容如下：

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: myweb
  name: myweb
spec:
  replicas: 2
  selector:
    matchLabels:
      app: myweb
  template:
    metadata:
      labels:
        app: myweb
    spec:
      containers:
      - image: kubeguide/tomcat-app:v1
        name: myweb
        ports:
        - containerPort: 8080
        env:
        - name: MYSQL_SERVICE_HOST
          value: mysql.database.default.svc.cluster.local


---
apiVersion: v1
kind: Service
metadata:
  name: myweb
spec:
  type: NodePort
  ports:
    - port: 8080
      nodePort: 8080
  selector:
    app: myweb

创建两个资源，并查看mysql的IP是 10.0.104.5，两个web的 IP，是 10.0.166.184 和 10.0.104.2。

[root@master1 ~]# kubectl apply -f mysql.yaml 
deployment.apps/mysql created
service/database created
[root@master1 ~]# kubectl apply -f myweb.yaml 
deployment.apps/myweb created
service/myweb created
[root@master1 ~]# kubectl get pods -l app=mysql -o wide
NAME                     READY   STATUS    RESTARTS   AGE     IP           NODE    NOMINATED NODE   READINESS GATES
mysql-6b479bcd9d-qtr8w   1/1     Running   0          3m17s   10.0.104.5   node2   <none>           <none>
[root@master1 ~]# 
[root@master1 ~]# kubectl get pods -l app=myweb -o wide
NAME                     READY   STATUS    RESTARTS   AGE     IP             NODE    NOMINATED NODE   READINESS GATES
myweb-554c69c9bd-pzn4n   1/1     Running   0          3m22s   10.0.166.184   node1   <none>           <none>
myweb-554c69c9bd-q8vrg   1/1     Running   0          3m21s   10.0.104.2     node2   <none>           <none>

创建ingress对象：

[root@master1 ~]# kubectl apply -f mywebsite-ingress.yaml 
ingress.networking.k8s.io/mywebsite-ingress created
[root@master1 ~]# 
[root@master1 ~]# kubectl get ingress
Warning: extensions/v1beta1 Ingress is deprecated in v1.14+, unavailable in v1.22+; use networking.k8s.io/v1 Ingress
NAME                CLASS    HOSTS           ADDRESS   PORTS   AGE
mywebsite-ingress   <none>   mywebsite.com             80      5s

一旦 Ingress 资源成功创建，Ingress Controller 就会监控到其配置的路由策略，并更新到 Nginx 的配置文件中生效。以本例中的 Nginx Controller 为例，它将更新其配置文件的内容为在 Ingress 中设定的路由策略。

登录一个 nginx-ingress-controller Pod,在/etc/nginx/conf.d 目录下可以看到 Nginx IngressController 自动生成的配置文件default-mywebsite-ingress.conf，查看其内容，可以看到对 mywebsite.com/demo 的转发规则的正确配置：

[root@master1 ~]# kubectl exec -it --namespace nginx-ingress nginx-ingress-75c88594dc-42256 -- bash
nginx@nginx-ingress-75c88594dc-42256:/$ ls /etc/nginx/conf.d/
default-mywebsite-ingress.conf
nginx@nginx-ingress-75c88594dc-42256:/$ cat /etc/nginx/conf.d/default-mywebsite-ingress.conf
# configuration for default/mywebsite-ingress
upstream default-mywebsite-ingress-mywebsite.com-myweb-8080 {
	zone default-mywebsite-ingress-mywebsite.com-myweb-8080 256k;
	random two least_conn;
	server 10.0.104.2:8080 max_fails=1 fail_timeout=10s max_conns=0;
	server 10.0.166.184:8080 max_fails=1 fail_timeout=10s max_conns=0;
}
server {
	listen 80;
	server_tokens on;
	server_name mywebsite.com;
	location /demo {
		proxy_http_version 1.1;
		proxy_connect_timeout 60s;
		proxy_read_timeout 60s;
		proxy_send_timeout 60s;
		client_max_body_size 1m;
		proxy_set_header Host $host;
		proxy_set_header X-Real-IP $remote_addr;
		proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
		proxy_set_header X-Forwarded-Host $host;
		proxy_set_header X-Forwarded-Port $server_port;
		proxy_set_header X-Forwarded-Proto $scheme;
		proxy_buffering on;
		proxy_pass http://default-mywebsite-ingress-mywebsite.com-myweb-8080;
	}
}

发现 nginx 监听 mywebsite.com 的 80 端口，并转发到 upstream 里的 10.0.166.184:8080 和 10.0.104.2:8080。

3. 客户端通过 Ingress Controller 访问后端 webapp 服务

由于 Ingress Controller 容器通过 hostPort 将服务端口号 80 映射到了宿主机上，所以客户端可以通过 Ingress Controller 所在的 Node 访问 Mywebsite.com 提供的服务。

需要注意的是，客户端只能通过域名 mywebsite.com 访问服务，这时要求客户端或者 DNS 将 mywebsite.com 域名解析到 Node 的正式 IP 地址上。刚刚上面执行kubectl get pods -n nginx-ingress -o wide的时候看到，nginx-ingress是在node2上，node2的正式IP是：192.168.100.202。

[root@master1 4.6.1]# curl --resolve mywebsite.com:80:192.168.100.202 http://mywebsite.com/demo/

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<title>HPE University Docker&Kubernetes Learning</title>
</head>
<body  align="center">

	
      <h2>Congratulations!!</h2>
     <br></br>
	 <input type="button" value="Add..." onclick="location.href='input.html'" >
	     <br></br>
      <TABLE align="center"  border="1" width="600px">
   <TR>
      <TD>Name</TD>
      <TD>Level(Score)</TD>
   </TR>

      
 <TR>
      <TD>google</TD>
      <TD>100</TD>
   </TR>
...

或在windows修改hosts文件后使用浏览器访问

参考《Kubernetes权威指南（第五版）》

Cloudpods

开源、云原生的融合云平台

更多推荐

面向未来的 IT 基础设施管理架构——融合云（Unified IaaS）

随着数字化时代的到来，IT系统已成为人类社会正常运转不可或缺的组成部分。不远的未来，智能制造，5G和人工智能等技术将成为推动生产力发展的重要引擎，人类社会将面临前所未有的全面彻底的数字化浪潮。IT基础设施作为IT系统运行的平台和载体，是实现数字化的基石。在这场数字化浪潮中，企业必须积极拥抱云计算技术，采用符合技术发展趋势、面向未来的IT基础构架，才能在未来的竞争中赢得先机。一、云计算历经十余年

Cloudpods

Cloudpods负载均衡的功能介绍

作者:周有松今天的内容会从以下几个方面展开：负载均衡产品简介。主要介绍负载均衡作为一个云上产品，它的功能模型是怎样的，日常使用中会遇到的业务词汇负载均衡的功能与典型应用场景。这部分主要结合业务词汇，对负载均衡服务中常见的一些功能选项进行介绍，并举例介绍一些典型的应用场景最后，我们做一下总结，讨论一下负载均衡产品相比传统方式的优点一、产品简介 1. 以NGINX为例提到负载均衡，我们以

Cloudpods

使用Linux vfio将Nvidia GPU透传给QEMU虚拟机

Linux 上虚拟机 GPU 透传需要使用 vfio 的方式。主要是因为在 vfio 方式下对虚拟设备的权限和 DMA 隔离上做的更好。但是这么做也有个缺点，这个物理设备在主机和其他虚拟机都不能使用了。 qemu 直接使用物理设备本身命令行是很简单的，关键在于事先在主机上对系统、内核和物理设备的一些配置。单纯从 qemu 的命令行来看，其实和普通虚拟机启动就差了最后那个-device的选项。这

Cloudpods

所有评论(0)

查看更多评论

昜羊

@zyy247796143

已为社区贡献1条内容