Bond 随你入坑kubernetes之：启蒙篇

Part 1 照本宣科暗礁多,爬完坑来乐呵呵能动手就绝不BB，开始。# 近日开始学习kubernetes和docker相关知识，一切从一本书开始，从第一个示例开始。但是事情远没有想象那么简单。本书在第一个示例演示上，就有若干内容错误，这些错误或许会让部分读者知难而退，但是也会让部分读者更加体会到kubernets的基础架构。# 这个例子主要是要在k8s中利用docker部署mysql与...

Bond的杂货铺

1635人浏览 · 2018-09-24 15:43:40

Bond的杂货铺 · 2018-09-24 15:43:40 发布

Part 1 照本宣科暗礁多,爬完坑来乐呵呵

能动手就绝不BB，开始。

# 近日开始学习kubernetes和docker相关知识，一切从一本书开始，从第一个示例开始。但是事情远没有想象那么简单。本书在第一个示例演示上，就有若干内容错误，这些错误或许会让部分读者知难而退，但是也会让部分读者更加体会到kubernets的基础架构。

# 这个例子主要是要在k8s中利用docker部署mysql与tomcat集群，并且实现tomcat正常读写mysql的操作。

# 操作系统版本与环境
[root@k8s01 ~]# uname -ra
Linux k8s01 3.10.0-862.el7.x86_64 #1 SMP Fri Apr 20 16:44:24 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
[root@k8s01 ~]# cat /etc/redhat-release
CentOS Linux release 7.5.1804 (Core)

Oracle Vbox 版本： 5.1.3 on MAC Book Pro, High Sierra 10.13.6

#刷新yum资源库
[root@k8s01 ~]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/centos-root 38G 1.1G 37G 3% /
devtmpfs 233M 0 233M 0% /dev
tmpfs 244M 0 244M 0% /dev/shm
tmpfs 244M 4.5M 240M 2% /run
tmpfs 244M 0 244M 0% /sys/fs/cgroup
/dev/sda1 1014M 129M 885M 13% /boot
tmpfs 49M 0 49M 0% /run/user/0
/dev/sr0 4.2G 4.2G 0 100% /media/cdrom
[root@k8s01 ~]# yum clean all
Loaded plugins: fastestmirror
Repodata is over 2 weeks old. Install yum-cron? Or run: yum makecache fast
Cleaning repos: base c7-media extras updates
Cleaning up everything
Maybe you want: rm -rf /var/cache/yum, to also free up space taken by orphaned data from disabled or removed repos
Cleaning up list of fastest mirrors
[root@k8s01 ~]# yum repolist
Loaded plugins: fastestmirror
Determining fastest mirrors
* base: mirrors.aliyun.com
* extras: mirrors.nju.edu.cn
* updates: mirrors.huaweicloud.com
base | 3.6 kB 00:00:00
c7-media | 3.6 kB 00:00:00
extras | 3.4 kB 00:00:00
updates | 3.4 kB 00:00:00
(1/6): c7-media/group_gz | 166 kB 00:00:00
(2/6): c7-media/primary_db | 3.1 MB 00:00:00
(3/6): base/7/x86_64/group_gz | 166 kB 00:00:00
(4/6): extras/7/x86_64/primary_db | 187 kB 00:00:01
(5/6): updates/7/x86_64/primary_db | 5.2 MB 00:00:03
(6/6): base/7/x86_64/primary_db | 5.9 MB 00:00:08
repo id repo name status
base/7/x86_64 CentOS-7 - Base 9,911
c7-media CentOS-7 - Media 3,971
extras/7/x86_64 CentOS-7 - Extras 403
updates/7/x86_64 CentOS-7 - Updates 1,348
repolist: 15,633

#使用yum安装etcd，kubernernets，期间会自动安装匹配的docker版本
[root@k8s01 ~]# yum install -y etcd kubernetes

[root@k8s01 ~]#
systemctl start etcd
systemctl start docker
systemctl start kube-apiserver
systemctl start kube-controller-manager
systemctl start kube-scheduler
systemctl start kubelet
systemctl start kube-proxy

systemctl stop etcd
systemctl stop docker
systemctl stop kube-apiserver
systemctl stop kube-controller-manager
systemctl stop kube-scheduler
systemctl stop kubelet
systemctl stop kube-proxy

#按照书上的例子创建rc
[root@k8s01 ~]# vi mysql-rc.yaml
apiVersion: v1
kind: ReplicationController #制定类型为 RC
metadata:
name: mysql #RC 名称，全局唯一
spec:
replicas: 1 #pod 副本数量为1
selector:
app: mysql #按照标签为mysql进行筛选
template: #当副本数量不足时，按此模版创建新副本
metadata:
labels:
app: mysql #被创建的副本缩拥有的标签，与selector下的标签对应，方能在创建后被选中
spec:
containers: #以下是pod内容器内定义
- name: mysql #定义容器名称
image: mysql #指定容器使用的image名称，注意书写规范,有tag需要带上
ports:
- containerPort: 3306 #容器应用监听的端口号
env:
- name: MYSQL_ROOT_PASSWORD #注入容器的环境变量
value: "123456"

[root@k8s01 ~]# kubectl create -f mysql-rc.yaml

[root@k8s01 ~]# kubectl get rc
NAME DESIRED CURRENT READY AGE
mysql 1 0 0 23h
[root@k8s01 ~]#

[root@k8s01 ~]# kubectl get pod -o wide --all-namespaces
No resources found.

[root@k8s01 ~]# vi /etc/kubernetes/apiserver
delete "ServiceAccount"
service kube-apiserver restart

[root@k8s01 ~]# service kube-apiserver restart

[root@k8s01 ~]# kubectl get pod -o wide --all-namespaces
NAMESPACE NAME READY STATUS RESTARTS AGE IP NODE
default mysql-pfz6q 0/1 ContainerCreating 0 5s <none> 127.0.0.1

[root@k8s01 ~]# kubectl describe pod mysql-pfz6q
Name: mysql-pfz6q
Namespace: default
Node: 127.0.0.1/127.0.0.1
Start Time: Mon, 24 Sep 2018 10:26:12 +0800
Labels: app=mysql
Status: Pending
IP:
Controllers: ReplicationController/mysql
Containers:
mysql:
Container ID:
Image: mysql
Image ID:
Port: 3306/TCP
State: Waiting
Reason: ContainerCreating
Ready: False
Restart Count: 0
Volume Mounts: <none>
Environment Variables:
MYSQL_ROOT_PASSWORD: 123456
Conditions:
Type Status
Initialized True
Ready False
PodScheduled True
No volumes.
QoS Class: BestEffort
Tolerations: <none>
Events:
FirstSeen LastSeen Count From SubObjectPath Type Reason Message
--------- -------- ----- ---- ------------- -------- ------ -------
47s 47s 1 {default-scheduler } Normal Scheduled Successfully assigned mysql-pfz6q to 127.0.0.1
21s 21s 1 {kubelet 127.0.0.1} Warning FailedSync Error syncing pod, skipping: failed to "StartContainer" for "POD" with ImagePullBackOff: "Back-off pulling image \"registry.access.redhat.com/rhel7/pod-infrastructure:latest\""

47s 10s 3 {kubelet 127.0.0.1} Warning FailedSync Error syncing pod, skipping: failed to "StartContainer" for "POD" with ErrImagePull: "image pull failed for registry.access.redhat.com/rhel7/pod-infrastructure:latest, this may be because there are no credentials on this request. details: (open /etc/docker/certs.d/registry.access.redhat.com/redhat-ca.crt: no such file or directory)"

# 坑1 通过 kubectl describe pod xxx 观察具体报错原因，发现基础image pull失败，是由于rhsm证书不存在导致。
# 手动 pull 一次试试，证实了以上的错误原因。注意这个镜像是kubernetes最基础的image，每个container启动，都依赖一个该image
[root@k8s01 ~]# docker pull registry.access.redhat.com/rhel7/pod-infrastructure
Using default tag: latest
Trying to pull repository registry.access.redhat.com/rhel7/pod-infrastructure ...
open /etc/docker/certs.d/registry.access.redhat.com/redhat-ca.crt: no such file or directory

# 查看证书文件，确实不存在
[root@k8s01 ~]# ls -lrt /etc/docker/certs.d/registry.access.redhat.com/redhat-ca.crt
lrwxrwxrwx. 1 root root 27 Sep 24 10:20 /etc/docker/certs.d/registry.access.redhat.com/redhat-ca.crt -> /etc/rhsm/ca/redhat-uep.pem
[root@k8s01 ~]# ls -lrt /etc/rhsm/ca/
total 0

# 但只要安装了 python-rhsm-certificates-1.19.10-1.el7_4.x86_64.rpm 就可以得到该证书文件；相反一旦被卸载，该文件也同时被删除
# 矛盾在与这个包在yum的依赖列表中已经被替代了，因此没有被安装。然而手动安装却会白告知冲突。
[root@k8s01 ~]# rpm -Uvh /media/cdrom/Packages/python-rhsm-certificates-1.19.10-1.el7_4.x86_64.rpm
error: Failed dependencies:
python-rhsm-certificates <= 1.20.3-1 is obsoleted by (installed) subscription-manager-rhsm-certificates-1.20.11-1.el7.centos.x86_64

# 因此只能先卸载现有rhsm包，再安装旧包，以得到redhat-uep.pem证书文件。但是注意到这一下子，连带docker与kubernetes相关部分组建也会被删除！！！
# 另外，一旦重新yum 安装kubernets，会自动清理掉以被废弃的rhsm包，也就意味着证书文件也同时被删除！（网上没有人提到这个）
[root@k8s01 ~]# rpm -qa | grep rhsm
subscription-manager-rhsm-certificates-1.20.11-1.el7.centos.x86_64

[root@k8s01 ~]# yum remove *rhsm*
Loaded plugins: fastestmirror
Resolving Dependencies
--> Running transaction check
---> Package subscription-manager-rhsm-certificates.x86_64 0:1.20.11-1.el7.centos will be erased
--> Processing Dependency: subscription-manager-rhsm-certificates for package: 2:docker-1.13.1-74.git6e3bb8e.el7.centos.x86_64
--> Running transaction check
---> Package docker.x86_64 2:1.13.1-74.git6e3bb8e.el7.centos will be erased
--> Processing Dependency: docker for package: kubernetes-node-1.5.2-0.7.git269f928.el7.x86_64
--> Running transaction check
---> Package kubernetes-node.x86_64 0:1.5.2-0.7.git269f928.el7 will be erased
--> Processing Dependency: kubernetes-node = 1.5.2-0.7.git269f928.el7 for package: kubernetes-1.5.2-0.7.git269f928.el7.x86_64
--> Running transaction check
---> Package kubernetes.x86_64 0:1.5.2-0.7.git269f928.el7 will be erased
--> Finished Dependency Resolution

Dependencies Resolved

=======================================================================================================================================================================================================================
Package Arch Version Repository Size
=======================================================================================================================================================================================================================
Removing:
subscription-manager-rhsm-certificates x86_64 1.20.11-1.el7.centos @base 0.0
Removing for dependencies:
docker x86_64 2:1.13.1-74.git6e3bb8e.el7.centos @extras 57 M
kubernetes x86_64 1.5.2-0.7.git269f928.el7 @extras 0.0
kubernetes-node x86_64 1.5.2-0.7.git269f928.el7 @extras 78 M

Transaction Summary
=======================================================================================================================================================================================================================
Remove 1 Package (+3 Dependent packages)

Installed size: 135 M
Is this ok [y/N]: y
Downloading packages:
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Erasing : kubernetes-1.5.2-0.7.git269f928.el7.x86_64 1/4
Erasing : kubernetes-node-1.5.2-0.7.git269f928.el7.x86_64 2/4
Erasing : 2:docker-1.13.1-74.git6e3bb8e.el7.centos.x86_64 3/4
warning: /etc/sysconfig/docker-storage saved as /etc/sysconfig/docker-storage.rpmsave
Erasing : subscription-manager-rhsm-certificates-1.20.11-1.el7.centos.x86_64 4/4
Verifying : subscription-manager-rhsm-certificates-1.20.11-1.el7.centos.x86_64 1/4
Verifying : kubernetes-1.5.2-0.7.git269f928.el7.x86_64 2/4
Verifying : kubernetes-node-1.5.2-0.7.git269f928.el7.x86_64 3/4
Verifying : 2:docker-1.13.1-74.git6e3bb8e.el7.centos.x86_64 4/4

Removed:
subscription-manager-rhsm-certificates.x86_64 0:1.20.11-1.el7.centos

Dependency Removed:
docker.x86_64 2:1.13.1-74.git6e3bb8e.el7.centos kubernetes.x86_64 0:1.5.2-0.7.git269f928.el7 kubernetes-node.x86_64 0:1.5.2-0.7.git269f928.el7

Complete!

# 再次手动安装rhsm包，终于得到了 /etc/rhsm/ca/redhat-uep.pem。有了它就可以。
# 一切都是为了这个证书，所以，备份一个，到其他位置。待kubernetes重新安装后，旧rhsm虽然被删除，但是我们可以将这个证书文件放回原位，以此跳过这个大坑！才可以顺利pull基础image。
[root@k8s01 ~]# rpm -Uvh /media/cdrom/Packages/python-rhsm-certificates-1.19.10-1.el7_4.x86_64.rpm
Preparing... ################################# [100%]
Updating / installing...
1:python-rhsm-certificates-1.19.10-################################# [100%]
[root@k8s01 ~]# ls -lrt /etc/rhsm/ca/redhat-uep.pem
-rw-r--r--. 1 root root 7732 Oct 20 2017 /etc/rhsm/ca/redhat-uep.pem

## [root@k8s01 ~]# ls -lrt /media/cdrom/Packages/python-rhsm-
python-rhsm-1.19.10-1.el7_4.x86_64.rpm python-rhsm-certificates-1.19.10-1.el7_4.x86_64.rpm

## save redhat-uep.pem
## reinstall all and start all
## put redhat-uep.pem into /etc/rhsm/ca/redhat-uep.pem

# 这时就可以手动pull了。当然有了以上经历，与现成的证书，完全可以在创建rc之前就将证书文件归位，这样pod可以顺利创建。
[root@k8s01 ~]# docker pull registry.access.redhat.com/rhel7/pod-infrastructure:latest
Trying to pull repository registry.access.redhat.com/rhel7/pod-infrastructure ...
latest: Pulling from registry.access.redhat.com/rhel7/pod-infrastructure
26e5ed6899db: Pull complete
66dbe984a319: Pull complete
9138e7863e08: Pull complete
Digest: sha256:92d43c37297da3ab187fc2b9e9ebfb243c1110d446c783ae1b989088495db931
Status: Downloaded newer image for registry.access.redhat.com/rhel7/pod-infrastructure:latest
[root@k8s01 ~]# docker images -a
REPOSITORY TAG IMAGE ID CREATED SIZE
docker.io/mysql latest 6a834f03bd02 10 days ago 484 MB
registry.access.redhat.com/rhel7/pod-infrastructure latest 99965fb98423 11 months ago 209 MB

# 准备重建rc，因此先删除rc，并重启所有服务
[root@k8s01 ~]# kubectl delete -f mysql-rc.yaml
replicationcontroller "mysql" deleted

[root@k8s01 ~]# systemctl stop etcd
[root@k8s01 ~]# systemctl stop docker
[root@k8s01 ~]# systemctl stop kube-apiserver
[root@k8s01 ~]# systemctl stop kube-controller-manager
[root@k8s01 ~]# systemctl stop kube-scheduler
[root@k8s01 ~]# systemctl stop kubelet
[root@k8s01 ~]# systemctl stop kube-proxy
[root@k8s01 ~]# systemctl start etcd

[root@k8s01 ~]# systemctl start docker
[root@k8s01 ~]# systemctl start kube-apiserver
[root@k8s01 ~]# systemctl start kube-controller-manager
[root@k8s01 ~]# systemctl start kube-scheduler
[root@k8s01 ~]# systemctl start kubelet
[root@k8s01 ~]# systemctl start kube-proxy

# 证书文件归位后，重新创建rc，观察到不再报基础image不能pull的错误，并且已经进入pull mysql image阶段，这需要一点点时间。
[root@k8s01 ~]# kubectl create -f mysql-rc.yaml
replicationcontroller "mysql" created
[root@k8s01 ~]# kubectl get pod -o wide --all-namespaces
NAMESPACE NAME READY STATUS RESTARTS AGE IP NODE
default mysql-qjfb9 0/1 ContainerCreating 0 5s <none> 127.0.0.1
[root@k8s01 ~]# kubectl describe pod mysql-qjfb9
Name: mysql-qjfb9
Namespace: default
Node: 127.0.0.1/127.0.0.1
Start Time: Mon, 24 Sep 2018 10:58:33 +0800
Labels: app=mysql
Status: Pending
IP:
Controllers: ReplicationController/mysql
Containers:
mysql:
Container ID:
Image: mysql
Image ID:
Port: 3306/TCP
State: Waiting
Reason: ContainerCreating
Ready: False
Restart Count: 0
Volume Mounts: <none>
Environment Variables:
MYSQL_ROOT_PASSWORD: 123456
Conditions:
Type Status
Initialized True
Ready False
PodScheduled True
No volumes.
QoS Class: BestEffort
Tolerations: <none>
Events:
FirstSeen LastSeen Count From SubObjectPath Type Reason Message
--------- -------- ----- ---- ------------- -------- ------ -------
23s 23s 1 {default-scheduler } Normal Scheduled Successfully assigned mysql-qjfb9 to 127.0.0.1
[root@k8s01 ~]# kubectl get pod -o wide --all-namespaces
NAMESPACE NAME READY STATUS RESTARTS AGE IP NODE
default mysql-qjfb9 0/1 ContainerCreating 0 29s <none> 127.0.0.1
[root@k8s01 ~]# kubectl get pod -o wide --all-namespaces
NAMESPACE NAME READY STATUS RESTARTS AGE IP NODE
default mysql-qjfb9 0/1 ContainerCreating 0 1m <none> 127.0.0.1
[root@k8s01 ~]# kubectl describe pod mysql-qjfb9
Name: mysql-qjfb9
Namespace: default
Node: 127.0.0.1/127.0.0.1
Start Time: Mon, 24 Sep 2018 10:58:33 +0800
Labels: app=mysql
Status: Pending
IP:
Controllers: ReplicationController/mysql
Containers:
mysql:
Container ID:
Image: mysql
Image ID:
Port: 3306/TCP
State: Waiting
Reason: ContainerCreating
Ready: False
Restart Count: 0
Volume Mounts: <none>
Environment Variables:
MYSQL_ROOT_PASSWORD: 123456
Conditions:
Type Status
Initialized True
Ready False
PodScheduled True
No volumes.
QoS Class: BestEffort
Tolerations: <none>
Events:
FirstSeen LastSeen Count From SubObjectPath Type Reason Message
--------- -------- ----- ---- ------------- -------- ------ -------
1m 1m 1 {default-scheduler } Normal Scheduled Successfully assigned mysql-qjfb9 to 127.0.0.1
7s 7s 1 {kubelet 127.0.0.1} Warning MissingClusterDNS kubelet does not have ClusterDNS IP configured and cannot create Pod using "ClusterFirst" policy. Falling back to DNSDefault policy.
5s 5s 1 {kubelet 127.0.0.1} spec.containers{mysql} Normal Pulling pulling image "mysql"
[root@k8s01 ~]#

# 最终，在pod的event里看到 container顺利启动
Events:
FirstSeen LastSeen Count From SubObjectPath Type Reason Message
--------- -------- ----- ---- ------------- -------- ------ -------
3m 3m 1 {default-scheduler } Normal Scheduled Successfully assigned mysql-qjfb9 to 127.0.0.1
1m 1m 1 {kubelet 127.0.0.1} spec.containers{mysql} Normal Pulling pulling image "mysql"
1m 29s 2 {kubelet 127.0.0.1} Warning MissingClusterDNS kubelet does not have ClusterDNS IP configured and cannot create Pod using "ClusterFirst" policy. Falling back to DNSDefault policy.
29s 29s 1 {kubelet 127.0.0.1} spec.containers{mysql} Normal Pulled Successfully pulled image "mysql"
27s 27s 1 {kubelet 127.0.0.1} spec.containers{mysql} Normal Created Created container with docker id 8b4f109879c3; Security:[seccomp=unconfined]
27s 27s 1 {kubelet 127.0.0.1} spec.containers{mysql} Normal Started Started container with docker id 8b4f109879c3

# 看到的pod状态也是running了！
[root@k8s01 ~]# kubectl get pod -o wide --all-namespaces
NAMESPACE NAME READY STATUS RESTARTS AGE IP NODE
default mysql-qjfb9 1/1 Running 0 6m 172.17.0.2 127.0.0.1

# 可以看到docker内基础image与mysql image 两个image
[root@k8s01 ~]# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
docker.io/mysql latest 6a834f03bd02 2 weeks ago 484 MB
registry.access.redhat.com/rhel7/pod-infrastructure latest 99965fb98423 11 months ago 209 MB

# 此时还没有为rc创建svc，因此在ep和services中还没有mysql的任何信息。
[root@k8s01 ~]# kubectl get ep
NAME ENDPOINTS AGE
kubernetes 10.0.2.15:6443 43m
[root@k8s01 ~]# kubectl get services
NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes 10.254.0.1 <none> 443/TCP 43m

# 为mysql rc创建对应的svc
[root@k8s01 ~]# cat mysql-svc.yaml
apiVersion: v1
kind: Service #表明是 kubernetes service
metadata:
name: mysql #service 全局唯一名称
spec:
type: NodePort #指明对外开放nodePort，node外网的终端可以访问。注意端口范围为 30000-
ports:
- port: 3306 #service在Pod内提供的端口号
nodePort: 30002 #对node外开放的端口号
selector:
app: mysql #service对应的pod拥有这里定义的标签

[root@k8s01 ~]# kubectl create -f mysql-svc.yaml
service "mysql" created

# 可以看到 mysql的nodePort和container port的映射关系，30002是可以被外网访问的。（注意vbox上的Port Forward设置）
[root@k8s01 ~]# kubectl get services
NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes 10.254.0.1 <none> 443/TCP 52m
mysql 10.254.40.213 <nodes> 3306:30002/TCP 6s
[root@k8s01 ~]# kubectl get ep
NAME ENDPOINTS AGE
kubernetes 10.0.2.15:6443 52m
mysql 172.17.0.2:3306 15s

# 注意防火墙会阻止host的30002端口被外网访问，关掉它即可。此时，任意mysql的jdbc的客户端都可以访问到mysql container内的数据库了。
[root@k8s01 ~]# systemctl stop firewalld
[root@k8s01 ~]# systemctl disable firewalld
Removed symlink /etc/systemd/system/multi-user.target.wants/firewalld.service.
Removed symlink /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service.

# 为了演示web连接mysql，还需要部署一套tomcat，这里同样使用一个docker上的示例image，kubeguide/tomcat-app:v1，各参数作用不再赘述。
# 按照书上的例子创建rc
[root@k8s01 ~]# vi myweb-rc.yaml
apiVersion: v1
kind: ReplicationController
metadata:
name: myweb
spec:
replicas: 2 #注意这里副本数量设置为2个
selector:
app: myweb
template:
metadata:
labels:
app: myweb
spec:
containers:
- name: myweb
image: kubeguide/tomcat-app:v1
ports:
- containerPort: 8080
env:
- name: MYSQL_SERVICE_PORT
value: '3306'

# 创建myweb rc后，一段时间内可以看到在自动pull image kubeguide/tomcat-app:v1，需要一点点时间。
[root@k8s01 ~]# kubectl create -f myweb-rc.yaml
replicationcontroller "myweb" created
[root@k8s01 ~]# kubectl get pod myweb
Error from server (NotFound): pods "myweb" not found
[root@k8s01 ~]# kubectl get pod
NAME READY STATUS RESTARTS AGE
mysql-qjfb9 1/1 Running 0 42m
myweb-j1tnr 0/1 ContainerCreating 0 21s
myweb-x47zl 0/1 ContainerCreating 0 21s
[root@k8s01 ~]# kubectl get rc
NAME DESIRED CURRENT READY AGE
mysql 1 1 1 42m
myweb 2 2 0 52s
[root@k8s01 ~]# kubectl describe pod myweb
Name: myweb-j1tnr
Namespace: default
Node: 127.0.0.1/127.0.0.1
Start Time: Mon, 24 Sep 2018 11:40:41 +0800
Labels: app=myweb
Status: Pending
IP:
Controllers: ReplicationController/myweb
Containers:
myweb:
Container ID:
Image: kubeguide/tomcat-app:v1
Image ID:
Port: 8080/TCP
State: Waiting
Reason: ContainerCreating
Ready: False
Restart Count: 0
Volume Mounts: <none>
Environment Variables:
MYSQL_SERVICE_PORT: 3306
Conditions:
Type Status
Initialized True
Ready False
PodScheduled True
No volumes.
QoS Class: BestEffort
Tolerations: <none>
Events:
FirstSeen LastSeen Count From SubObjectPath Type Reason Message
--------- -------- ----- ---- ------------- -------- ------ -------
1m 1m 1 {default-scheduler } Normal Scheduled Successfully assigned myweb-j1tnr to 127.0.0.1
1m 1m 1 {kubelet 127.0.0.1} Warning MissingClusterDNS kubelet does not have ClusterDNS IP configured and cannot create Pod using "ClusterFirst" policy. Falling back to DNSDefault policy.
1m 1m 1 {kubelet 127.0.0.1} spec.containers{myweb} Normal Pulling pulling image "kubeguide/tomcat-app:v1"

Name: myweb-x47zl
Namespace: default
Node: 127.0.0.1/127.0.0.1
Start Time: Mon, 24 Sep 2018 11:40:40 +0800
Labels: app=myweb
Status: Pending
IP:
Controllers: ReplicationController/myweb
Containers:
myweb:
Container ID:
Image: kubeguide/tomcat-app:v1
Image ID:
Port: 8080/TCP
State: Waiting
Reason: ContainerCreating
Ready: False
Restart Count: 0
Volume Mounts: <none>
Environment Variables:
MYSQL_SERVICE_PORT: 3306
Conditions:
Type Status
Initialized True
Ready False
PodScheduled True
No volumes.
QoS Class: BestEffort
Tolerations: <none>
Events:
FirstSeen LastSeen Count From SubObjectPath Type Reason Message
--------- -------- ----- ---- ------------- -------- ------ -------
1m 1m 1 {default-scheduler } Normal Scheduled Successfully assigned myweb-x47zl to 127.0.0.1
1m 1m 1 {kubelet 127.0.0.1} Warning MissingClusterDNS kubelet does not have ClusterDNS IP configured and cannot create Pod using "ClusterFirst" policy. Falling back to DNSDefault policy.
1m 1m 1 {kubelet 127.0.0.1} spec.containers{myweb} Normal Pulling pulling image "kubeguide/tomcat-app:v1"

# 过一会儿从events看到 kubeguide/tomcat-app:v1 image pull完毕并且 container 顺利启动
Events:
FirstSeen LastSeen Count From SubObjectPath Type Reason Message
--------- -------- ----- ---- ------------- -------- ------ -------
5m 5m 1 {default-scheduler } Normal Scheduled Successfully assigned myweb-x47zl to 127.0.0.1
5m 5m 1 {kubelet 127.0.0.1} spec.containers{myweb} Normal Pulling pulling image "kubeguide/tomcat-app:v1"
5m 2m 2 {kubelet 127.0.0.1} Warning MissingClusterDNS kubelet does not have ClusterDNS IP configured and cannot create Pod using "ClusterFirst" policy. Falling back to DNSDefault policy.
2m 2m 1 {kubelet 127.0.0.1} spec.containers{myweb} Normal Pulled Successfully pulled image "kubeguide/tomcat-app:v1"
2m 2m 1 {kubelet 127.0.0.1} spec.containers{myweb} Normal Created Created container with docker id 4ceda9cf2201; Security:[seccomp=unconfined]
2m 2m 1 {kubelet 127.0.0.1} spec.containers{myweb} Normal Started Started container with docker id 4ceda9cf2201

# 可以看到myweb rc抽取的image
[root@k8s01 ~]# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
docker.io/mysql latest 6a834f03bd02 2 weeks ago 484 MB
registry.access.redhat.com/rhel7/pod-infrastructure latest 99965fb98423 11 months ago 209 MB
docker.io/kubeguide/tomcat-app v1 a29e200a18e9 2 years ago 358 MB

# 可以看到目前mysql与tomcat的container均在运行中
[root@k8s01 ~]# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
4ceda9cf2201 kubeguide/tomcat-app:v1 "catalina.sh run" 5 minutes ago Up 5 minutes k8s_myweb.39362350_myweb-x47zl_default_9badb46a-bfab-11e8-8eb0-080027104eb1_6d54a9b7
7e7d59a979e1 kubeguide/tomcat-app:v1 "catalina.sh run" 5 minutes ago Up 5 minutes k8s_myweb.39362350_myweb-j1tnr_default_9baea256-bfab-11e8-8eb0-080027104eb1_acd10545
1b2eee020c79 registry.access.redhat.com/rhel7/pod-infrastructure:latest "/usr/bin/pod" 8 minutes ago Up 8 minutes k8s_POD.24f70ba9_myweb-j1tnr_default_9baea256-bfab-11e8-8eb0-080027104eb1_a55863f0
d3df3493caa8 registry.access.redhat.com/rhel7/pod-infrastructure:latest "/usr/bin/pod" 8 minutes ago Up 8 minutes k8s_POD.24f70ba9_myweb-x47zl_default_9badb46a-bfab-11e8-8eb0-080027104eb1_fc05c692
8b4f109879c3 mysql "docker-entrypoint..." 48 minutes ago Up 48 minutes k8s_mysql.f6601b53_mysql-qjfb9_default_b98e0e93-bfa5-11e8-8eb0-080027104eb1_1e500e77
b21ecfd57466 registry.access.redhat.com/rhel7/pod-infrastructure:latest "/usr/bin/pod" 49 minutes ago Up 49 minutes k8s_POD.1d520ba5_mysql-qjfb9_default_b98e0e93-bfa5-11e8-8eb0-080027104eb1_c70efb71

# 再为myweb rc创建对应的svc，参数含义不再赘述
[root@k8s01 ~]# vi myweb-svc.yaml
apiVersion: v1
kind: Service
metadata:
name: myweb
spec:
type: NodePort
ports:
- port: 8080
nodePort: 30001
selector:
app: myweb

[root@k8s01 ~]# kubectl create -f myweb-svc.yaml
service "myweb" created

# 可以看到myweb的service的nodePort 30001已经开启（注意vbox上的Port Forward设置）
[root@k8s01 ~]# kubectl get services
NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes 10.254.0.1 <none> 443/TCP 1h
mysql 10.254.40.213 <nodes> 3306:30002/TCP 44m
myweb 10.254.93.139 <nodes> 8080:30001/TCP 28s

# 注意到这里myweb有两个container，均使用的8080端口（前面host的防火墙已经关闭，这里不赘述）
[root@k8s01 ~]# kubectl get ep
NAME ENDPOINTS AGE
kubernetes 10.0.2.15:6443 1h
mysql 172.17.0.2:3306 44m
myweb 172.17.0.3:8080,172.17.0.4:8080 34s

#在本机使用一下url就可以打开tomcat主页了。
http://127.0.0.1:30001/

#当前端口路由是这样子的：
mysql container port 3306 ---> service nodePort 30002 ---> vbox本地端口 30002
tomcat container port 8080 ---> service nodePort 30001 ---> vbox本地端口 30001

# 坑2. 但是用于演示tomcat连接mysql的示例却无法打开，是怎么回事呢？
http://127.0.0.1:30001/demo
页面报错： Error:com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: Could not create connection to database server.

分析：首先确认几个问题
1）mysql container启动正常，并且通过30002端口可以访问该数据库证明，说明整个容器是运行正常的。
2）myweb container启动正常，并且通过30001端口可以访问tomcat主页，说明整个容器也是运行正常的。
3）从报错看，是tomcat项目在尝试连接mysql数据库时，没有找到其所在主机。因此需要看看 kubeguide/tomcat-app:v1 中是如何定义数据库连接的。

# 手动run一个 kubeguide/tomcat-app:v1 ，并进入container 观察项目代码中的 index.jsp文件
[root@k8s01 ~]# docker run -it docker.io/kubeguide/tomcat-app:v1 /bin/bash
root@cc6c9fec806e:/usr/local/tomcat# ls -lrt webapps/demo/
total 12
-rw-r--r--. 1 root root 650 Jun 12 2016 input.html
-rw-r--r--. 1 root root 3471 Jun 12 2016 index.jsp
-rw-r--r--. 1 root root 2006 Jun 12 2016 insert.jsp
drwxr-xr-x. 3 root root 32 Jun 28 2016 WEB-INF

root@cc6c9fec806e:/usr/local/tomcat# cat webapps/demo/index.jsp
................
try{
Class.forName("com.mysql.jdbc.Driver");
String ip=System.getenv("MYSQL_SERVICE_HOST");
String port=System.getenv("MYSQL_SERVICE_PORT");
ip=(ip==null)?"localhost":ip;
port=(port==null)?"3306":port;
System.out.println("Connecting to database...");

conn = java.sql.DriverManager.getConnection("jdbc:mysql://"+ip+":"+port+"?useUnicode=true&characterEncoding=UTF-8", "root","123456");

stmt = conn.createStatement();
String sql = "show databases like 'HPE_APP'";
rs =stmt.executeQuery(sql);
...............
#以上发现，该项目使用了env注入的方式获取mysql的主机地址与端口。默认值分别为localhost与3306. 这就难怪了，因为在myweb-rc.yaml仅定义了MYSQL_SERVICE_PORT，而没有MYSQL_SERVICE_HOST。
#因此去myweb-rc.yaml增加MYSQL_SERVICE_HOST，先找到mysql的ep是172.17.0.2
[root@k8s01 ~]# kubectl get ep
NAME ENDPOINTS AGE
kubernetes 10.0.2.15:6443 2h
mysql 172.17.0.2:3306 1h
myweb 172.17.0.3:8080,172.17.0.4:8080 40m

[root@k8s01 ~]# vi myweb-rc.yaml
apiVersion: v1
kind: ReplicationController
metadata:
name: myweb
spec:
replicas: 2
selector:
app: myweb
template:
metadata:
labels:
app: myweb
spec:
containers:
- name: myweb
image: kubeguide/tomcat-app:v1
# image: 241374050/tomcat:v2
ports:
- containerPort: 8080
env:
- name: MYSQL_SERVICE_HOST
value: 172.17.0.2
- name: MYSQL_SERVICE_PORT
value: '3306'

# 重建myweb rc
[root@k8s01 ~]# kubectl delete -f myweb-rc.yaml
replicationcontroller "myweb" deleted
[root@k8s01 ~]# kubectl create -f myweb-rc.yaml
replicationcontroller "myweb" created

#问题仍然存在，此时尝试更换与mysql匹配的jdbc版本 mysql-connector-java-8.0.11.jar
#顺便发现了demo项目使用的mysql jdbc版本是5.1.37
root@cc6c9fec806e:/usr/local/tomcat# ls -lrt webapps/demo/WEB-INF/lib/mysql-connector-java-5.1.37.jar
-rw-r--r--. 1 root root 985600 Jun 12 2016 webapps/demo/WEB-INF/lib/mysql-connector-java-5.1.37.jar

#注意，这里将会把mysql-connector-java-8.0.11.jar替换进tomcat的image，但container一旦exit就会丢失所有变更，所以唯一的方法是手动run container，并替换jdbc后，提交image做保存。

[root@k8s01 ~]# docker login
Login with your Docker ID to push and pull images from Docker Hub. If you don't have a Docker ID, head over to https://hub.docker.com to create one.
Username (241374050): 241374050
Password:
Login Succeeded
[root@k8s01 ~]# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
docker.io/mysql latest 6a834f03bd02 2 weeks ago 484 MB
registry.access.redhat.com/rhel7/pod-infrastructure latest 99965fb98423 11 months ago 209 MB
docker.io/kubeguide/tomcat-app v1 a29e200a18e9 2 years ago 358 MB
[root@k8s01 ~]# docker run -it kubeguide/tomcat-app:v1 /bin/bash
root@00d0103f014d:/usr/local/tomcat# ls -lrt /usr/local/tomcat/webapps/demo/WEB-INF/lib/
total 964
-rw-r--r--. 1 root root 985600 Jun 12 2016 mysql-connector-java-5.1.37.jar

#注意，依次按 control+P,+Q, 即可不关闭container的情况下回到host，然后使用docker cp将jdbc 8的文件拷贝至container
[root@k8s01 ~]# ls -lrt mysql-connector-java-8.0.11.jar
-rw-r--r--. 1 root root 2036609 Sep 24 12:59 mysql-connector-java-8.0.11.jar
[root@k8s01 ~]# docker cp mysql-connector-java-8.0.11.jar 00d0103f014d:/usr/local/tomcat/webapps/demo/WEB-INF/lib/
[root@k8s01 ~]# docker attach 00d0103f014d

root@00d0103f014d:/usr/local/tomcat# ls -lrt /usr/local/tomcat/webapps/demo/WEB-INF/lib/
total 2956
-rw-r--r--. 1 root root 985600 Jun 12 2016 mysql-connector-java-5.1.37.jar
-rw-r--r--. 1 root root 2036609 Sep 24 04:59 mysql-connector-java-8.0.11.jar
root@00d0103f014d:/usr/local/tomcat# cd /usr/local/tomcat/webapps/demo/WEB-INF/lib/
root@00d0103f014d:/usr/local/tomcat/webapps/demo/WEB-INF/lib# mv mysql-connector-java-5.1.37.jar mysql-connector-java-5.1.37.jar_bak
root@00d0103f014d:/usr/local/tomcat/webapps/demo/WEB-INF/lib# ls -lrt
total 2956
-rw-r--r--. 1 root root 985600 Jun 12 2016 mysql-connector-java-5.1.37.jar_bak
-rw-r--r--. 1 root root 2036609 Sep 24 04:59 mysql-connector-java-8.0.11.jar

# 对现有image 做提交，并保存下来
[root@k8s01 ~]# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
00d0103f014d kubeguide/tomcat-app:v1 "/bin/bash" 5 minutes ago Up 5 minutes 8080/tcp epic_euclid
dfa8a622c286 kubeguide/tomcat-app:v1 "catalina.sh run" 36 minutes ago Up 36 minutes k8s_myweb.6cfa4229_myweb-pnpfl_default_f11d8bcb-bfb5-11e8-8eb0-080027104eb1_c9e8805c
da1dc5d7e9ce kubeguide/tomcat-app:v1 "catalina.sh run" 36 minutes ago Up 36 minutes k8s_myweb.6cfa4229_myweb-lz39h_default_f111f0a9-bfb5-11e8-8eb0-080027104eb1_50beb441
f91e2947dca9 registry.access.redhat.com/rhel7/pod-infrastructure:latest "/usr/bin/pod" 36 minutes ago Up 36 minutes k8s_POD.24f70ba9_myweb-pnpfl_default_f11d8bcb-bfb5-11e8-8eb0-080027104eb1_7d550d4f
5f9775645d02 registry.access.redhat.com/rhel7/pod-infrastructure:latest "/usr/bin/pod" 36 minutes ago Up 36 minutes k8s_POD.24f70ba9_myweb-lz39h_default_f111f0a9-bfb5-11e8-8eb0-080027104eb1_384d575c
8b4f109879c3 mysql "docker-entrypoint..." 2 hours ago Up 2 hours k8s_mysql.f6601b53_mysql-qjfb9_default_b98e0e93-bfa5-11e8-8eb0-080027104eb1_1e500e77
b21ecfd57466 registry.access.redhat.com/rhel7/pod-infrastructure:latest "/usr/bin/pod" 2 hours ago Up 2 hours k8s_POD.1d520ba5_mysql-qjfb9_default_b98e0e93-bfa5-11e8-8eb0-080027104eb1_c70efb71

[root@k8s01 ~]# docker commit -m "tomcat" -a "241374050" 00d0103f014d 241374050/tomcat:v2
sha256:717db5b75338b6c9f3829261f63cc64ee3b9071bb41e693a51976cc6850073c4
[root@k8s01 ~]# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
241374050/tomcat v2 717db5b75338 4 seconds ago 361 MB
docker.io/mysql latest 6a834f03bd02 2 weeks ago 484 MB
registry.access.redhat.com/rhel7/pod-infrastructure latest 99965fb98423 11 months ago 209 MB
docker.io/kubeguide/tomcat-app v1 a29e200a18e9 2 years ago 358 MB
[root@k8s01 ~]#

# 其中，-m指定说明信息；-a指定用户信息；00d0103f014d 代表容器的id；241374050/tomcat:v2 指定目标镜像的用户名、仓库名和 tag 信息。

# 替换jdbc后的image将作为新的container 连接 mysql。因此需要为新的container定义rc和svc。
[root@k8s01 ~]# cp myweb-rc.yaml myweb-rc-1.yaml
[root@k8s01 ~]# cp myweb-svc.yaml myweb-svc-1.yaml

# 除了要修改image，同时全局唯一的myweb修改为myweb1，且nodePort需要改成新端口30003，简便起见也将副本数量改成1
[root@k8s01 ~]# vi myweb-rc-1.yaml
apiVersion: v1
kind: ReplicationController
metadata:
name: myweb1
spec:
replicas: 1
selector:
app: myweb1
template:
metadata:
labels:
app: myweb1
spec:
containers:
- name: myweb1
# image: kubeguide/tomcat-app:v1
image: 241374050/tomcat:v2 #使用自己保存过的image
ports:
- containerPort: 8080
env:
- name: MYSQL_SERVICE_HOST
value: 172.17.0.2
- name: MYSQL_SERVICE_PORT
value: '3306'

[root@k8s01 ~]# vi myweb-svc-1.yaml
apiVersion: v1
kind: Service
metadata:
name: myweb1
spec:
type: NodePort
ports:
- port: 8080
nodePort: 30003
selector:
app: myweb1

[root@k8s01 ~]# kubectl create -f myweb-rc-1.yaml
replicationcontroller "myweb1" created

[root@k8s01 ~]# kubectl create -f myweb-svc-1.yaml
service "myweb1" created

# 但是pod启动后，发现一直在报错，且RESTARTS一直在增加表示master在一次次重启container
[root@k8s01 ~]# kubectl get pod
NAME READY STATUS RESTARTS AGE
mysql-qjfb9 1/1 Running 0 2h
myweb-lz39h 1/1 Running 0 57m
myweb-pnpfl 1/1 Running 0 57m
myweb1-pgw0s 0/1 CrashLoopBackOff 4 2m
myweb1-zrjw8 0/1 CrashLoopBackOff 4 2m

# 需要看看新的镜像为何启动报错，且无法分配ep，则手动run一个该image，该container与前面独立，互不干涉。
[root@k8s01 ~]# docker run -it 241374050/tomcat:v2 /bin/bash
root@11465bf087eb:/usr/local/tomcat# catalina.sh start
Using CATALINA_BASE: /usr/local/tomcat
Using CATALINA_HOME: /usr/local/tomcat
Using CATALINA_TMPDIR: /usr/local/tomcat/temp
Using JRE_HOME: /usr/lib/jvm/java-7-openjdk-amd64/jre
Using CLASSPATH: /usr/local/tomcat/bin/bootstrap.jar:/usr/local/tomcat/bin/tomcat-juli.jar
Tomcat started.

# 使用curl测试，可以获取tomcat主页
root@11465bf087eb:/usr/local/tomcat# curl 127.0.0.1:8080

# 但是使用demo时，项目无法初始化，报错
root@11465bf087eb:/usr/local/tomcat# curl 127.0.0.1:8080/demo/
java.lang.UnsupportedClassVersionError: com/mysql/jdbc/Driver : Unsupported major.minor version 52.0 (unable to load class com.mysql.jdbc.Driver)

# 因为demo项目时jdk7开发，更换的mysql-connector-java-8.0.11.jar 是基于jdk8，当访问demo时，因不兼容，导致调用失败。
# demo项目 jdbc版本太旧。坑3，默认的tomcat image中项目使用的jdk版本是7（5.2），相应jdbc是5.1，而默认mysql image中允许的jdbc版本最低是8，且对应jdk8（5.2）。
# 此时，为了演示成功，tomcat的demo项目时必须使用的，那么就考虑更换mysql版本，即使用mysql5.7的docker image。

[root@k8s01 ~]# docker search mysql
INDEX NAME DESCRIPTION STARS OFFICIAL AUTOMATED
docker.io docker.io/mysql MySQL is a widely used, open-source relati... 6990 [OK]
docker.io docker.io/mariadb MariaDB is a community-developed fork of M... 2236 [OK]
docker.io docker.io/mysql/mysql-server Optimized MySQL Server Docker images. Crea... 513 [OK]
docker.io docker.io/percona Percona Server is a fork of the MySQL rela... 369 [OK]
docker.io docker.io/zabbix/zabbix-server-mysql Zabbix Server with MySQL database support 127 [OK]
docker.io docker.io/hypriot/rpi-mysql RPi-compatible Docker Image with Mysql 96
docker.io docker.io/zabbix/zabbix-web-nginx-mysql Zabbix frontend based on Nginx web-server ... 68 [OK]
docker.io docker.io/centurylink/mysql Image containing mysql. Optimized to be li... 59 [OK]
docker.io docker.io/1and1internet/ubuntu-16-nginx-php-phpmyadmin-mysql-5 ubuntu-16-nginx-php-phpmyadmin-mysql-5 44 [OK]
docker.io docker.io/centos/mysql-57-centos7 MySQL 5.7 SQL database server 39
docker.io docker.io/mysql/mysql-cluster Experimental MySQL Cluster Docker images. ... 34
docker.io docker.io/tutum/mysql Base docker image to run a MySQL database ... 32
docker.io docker.io/schickling/mysql-backup-s3 Backup MySQL to S3 (supports periodic back... 23 [OK]
docker.io docker.io/bitnami/mysql Bitnami MySQL Docker Image 19 [OK]
docker.io docker.io/linuxserver/mysql A Mysql container, brought to you by Linux... 16
docker.io docker.io/zabbix/zabbix-proxy-mysql Zabbix proxy with MySQL database support 15 [OK]
docker.io docker.io/centos/mysql-56-centos7 MySQL 5.6 SQL database server 10
docker.io docker.io/circleci/mysql MySQL is a widely used, open-source relati... 6
docker.io docker.io/openshift/mysql-55-centos7 DEPRECATED: A Centos7 based MySQL v5.5 ima... 6
docker.io docker.io/mysql/mysql-router MySQL Router provides transparent routing ... 4
docker.io docker.io/jelastic/mysql An image of the MySQL database server main... 1
docker.io docker.io/openzipkin/zipkin-mysql Mirror of https://quay.io/repository/openz... 1
docker.io docker.io/ansibleplaybookbundle/mysql-apb An APB which deploys RHSCL MySQL 0 [OK]
docker.io docker.io/cloudfoundry/cf-mysql-ci Image used in CI of cf-mysql-release 0
docker.io docker.io/cloudposse/mysql Improved `mysql` service with support for ... 0 [OK]

# 找到一个mysql5.7的image，pull下来
[root@k8s01 ~]# docker pull docker.io/centos/mysql-57-centos7
Using default tag: latest
Trying to pull repository docker.io/centos/mysql-57-centos7 ...
latest: Pulling from docker.io/centos/mysql-57-centos7
256b176beaff: Pull complete
efb19fea0fdb: Pull complete
b4570fdc208c: Pull complete
213e4e250552: Pull complete
63f85aa3129f: Pull complete
fa602875dfe8: Pull complete
68f544f308c0: Pull complete
d9559e104974: Pull complete
7275a92d5226: Pull complete
Digest: sha256:d42718d13b5ff10d61e1d560d225428ba2837f8d36dc38b6957b39cc0006f144
Status: Downloaded newer image for docker.io/centos/mysql-57-centos7:latest
[root@k8s01 ~]# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
241374050/tomcat v2 717db5b75338 About an hour ago 361 MB
docker.io/centos/mysql-57-centos7 latest aa73af2338a4 4 days ago 445 MB
docker.io/mysql latest 6a834f03bd02 2 weeks ago 484 MB
registry.access.redhat.com/rhel7/pod-infrastructure latest 99965fb98423 11 months ago 209 MB
docker.io/kubeguide/tomcat-app v1 a29e200a18e9 2 years ago 358 MB

# 为了使用这个新的image，docker.io/centos/mysql-57-centos7，需要配置新的rc和svc
# 新的container启动与原有mysql container没有任何冲突，但是全局唯一的lable必须更改；在给mysql57 分配nodePort时，要注意避免冲突；另外myweb的MYSQL_SERVICE_HOST 也需要指向mysql57的ep

[root@k8s01 ~]# kubectl get pod
NAME READY STATUS RESTARTS AGE
mysql-qjfb9 1/1 Running 0 3h
myweb-267nw 1/1 Running 0 5m
myweb-jgnf8 1/1 Running 0 5m

[root@k8s01 ~]# cp mysql-rc.yaml mysql-rc1.yaml
[root@k8s01 ~]# cp mysql-svc.yaml mysql-svc1.yaml
[root@k8s01 ~]# vi mysql-rc1.yaml
apiVersion: v1
kind: ReplicationController
metadata:
name: mysql1 #改
spec:
replicas: 1
selector:
app: mysql1 #改
template:
metadata:
labels:
app: mysql1 #改
spec:
containers:
- name: mysql1 #改
#image: mysql
image: centos/mysql-57-centos7
ports:
- containerPort: 3306
env:
- name: MYSQL_ROOT_PASSWORD
value: "123456"

[root@k8s01 ~]# vi mysql-svc1.yaml
apiVersion: v1
kind: Service
metadata:
name: mysql1 #改
spec:
type: NodePort
ports:
- port: 3306
nodePort: 30004 #改
selector:
app: mysql1

[root@k8s01 ~]# kubectl create -f mysql-rc1.yaml
replicationcontroller "mysql1" created
[root@k8s01 ~]# kubectl create -f mysql-svc1.yaml

[root@k8s01 ~]# kubectl get pod -o wide --all-namespaces
NAMESPACE NAME READY STATUS RESTARTS AGE IP NODE
default mysql-qjfb9 1/1 Running 0 4h 172.17.0.2 127.0.0.1
default mysql1-rtr8x 1/1 Running 0 39s 172.17.0.5 127.0.0.1
default myweb-267nw 1/1 Running 0 14m 172.17.0.3 127.0.0.1
default myweb-jgnf8 1/1 Running 0 14m 172.17.0.4 127.0.0.1

# 确认mysql1的ep分配成功，因为myweb需要对应ip信息
[root@k8s01 ~]# kubectl get ep
NAME ENDPOINTS AGE
kubernetes 10.0.2.15:6443 4h
mysql 172.17.0.2:3306 3h
mysql1 172.17.0.5:3306 50s
myweb 172.17.0.3:8080,172.17.0.4:8080 15m

# 重新配置MYSQL_SERVICE_HOST
[root@k8s01 ~]# kubectl delete -f myweb-rc.yaml
replicationcontroller "myweb" deleted

[root@k8s01 ~]# kubectl create -f myweb-rc.yaml
replicationcontroller "myweb" created

[root@k8s01 ~]# kubectl get pod
NAME READY STATUS RESTARTS AGE
mysql-qjfb9 1/1 Running 0 4h
mysql1-rtr8x 1/1 Running 0 5m
myweb-cwlrq 1/1 Running 0 21s
myweb-lq1g5 1/1 Running 0 21s
[root@k8s01 ~]#
[root@k8s01 ~]# kubectl get services
NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes 10.254.0.1 <none> 443/TCP 4h
mysql 10.254.40.213 <nodes> 3306:30002/TCP 3h
mysql1 10.254.231.157 <nodes> 3306:30004/TCP 5m
myweb 10.254.254.203 <nodes> 8080:30001/TCP 20m

#至此，http://127.0.0.1:30001/demo/ 终于正常显示，并且完成数据库初始化，然后在页面输入插入一行记录，成功！

## 如果关闭防火墙后仍有通信问题，可以尝试手动开放30001端口，然后禁用防火墙。
[root@k8s01 ~]# systemctl start firewalld
[root@k8s01 ~]# firewall-cmd --zone=public --add-port=30001/tcp --permanent
Warning: ALREADY_ENABLED: 30001:tcp
success
[root@k8s01 ~]# systemctl stop firewalld
[root@k8s01 ~]# systemctl disable firewalld
[root@k8s01 ~]# systemctl status firewalld

总结：
1）最后看来，实际上就是“换了mysql5.7的镜像”就把事情解决了。但如果真有人一开始就告诉我，那么我将会错过不少东西。
为了诊断问题，首先学会了describe，并且理解rc和svc的关系，理解了nodePort，手动pull了image，体会了各container隔离性，学会使用docker cp，并attache container进行问题诊断，等等。。。
2）实践永远是检验真理的唯一标准。《kubernetes 权威指南纪念版》是本好书。但是里面的勘误还是相当有必要。但从另一方面说，这些内容上的失误，磨练了并帮助读者更好理解了这个入门实例。
3) 这第一个示例的入门过程从看书到爬出坑，加上整个过程重演、总结等花了一个星期完成。如果你看到这里，仅仅是做完一遍，甚至是复制粘贴命令完成了整个过程，就准备抛到脑后了，那么很遗憾你错过了很多内容。
4）对我来说，写到这里，一顿操作猛如虎后，瞬间觉得索然无味：）便可以继续后面的内容了。但是子曰：”温故而知新“，已经不止一次亲身体会到它的玄妙了！

以上命令、文字内容全部手敲，难免疏漏，若有错误敬请指正，谢谢！

-- To be continue --

Cloudpods

开源、云原生的融合云平台

更多推荐

面向未来的 IT 基础设施管理架构——融合云（Unified IaaS）

随着数字化时代的到来，IT系统已成为人类社会正常运转不可或缺的组成部分。不远的未来，智能制造，5G和人工智能等技术将成为推动生产力发展的重要引擎，人类社会将面临前所未有的全面彻底的数字化浪潮。IT基础设施作为IT系统运行的平台和载体，是实现数字化的基石。在这场数字化浪潮中，企业必须积极拥抱云计算技术，采用符合技术发展趋势、面向未来的IT基础构架，才能在未来的竞争中赢得先机。一、云计算历经十余年

Cloudpods

Cloudpods负载均衡的功能介绍

作者:周有松今天的内容会从以下几个方面展开：负载均衡产品简介。主要介绍负载均衡作为一个云上产品，它的功能模型是怎样的，日常使用中会遇到的业务词汇负载均衡的功能与典型应用场景。这部分主要结合业务词汇，对负载均衡服务中常见的一些功能选项进行介绍，并举例介绍一些典型的应用场景最后，我们做一下总结，讨论一下负载均衡产品相比传统方式的优点一、产品简介 1. 以NGINX为例提到负载均衡，我们以

Cloudpods

使用Linux vfio将Nvidia GPU透传给QEMU虚拟机

Linux 上虚拟机 GPU 透传需要使用 vfio 的方式。主要是因为在 vfio 方式下对虚拟设备的权限和 DMA 隔离上做的更好。但是这么做也有个缺点，这个物理设备在主机和其他虚拟机都不能使用了。 qemu 直接使用物理设备本身命令行是很简单的，关键在于事先在主机上对系统、内核和物理设备的一些配置。单纯从 qemu 的命令行来看，其实和普通虚拟机启动就差了最后那个-device的选项。这