kubernetes-statefulset
statefulset 有状态的应用，主备集群部署，例如：mysql主备
特点：
1）通过statefulset部署出来的pod各自用拥有独立的存储，在statefulset中定义pvc template,每一个pod通过pvc template,各自关联pv.
2） statefulset的svc不会LB 无头服务，无clusterIp，不同ip地址访问，只通过podname.svcname访问。不全局可见，项目内可见。。可以通过服务发现访问
3） statefulset 保证pod 的高可用，#删除pod，会重建，但pod 名字不变，
实验： 利用statefulset，创建一个多副本的mysql拓扑架构
只读从，读写主。
存储独立，服务独立。
创建pv
前置条件准备了pv
[root@master01 mysql]# cat pv.yaml
---
apiVersion: v1
kind: PersistentVolume
metadata:
name: master01-pv
labels:
app: mysql
app.kubernetes.io/name: mysql
spec:
capacity:
storage: 5Gi
volumeMode: Filesystem
accessModes:
- ReadWriteMany
persistentVolumeReclaimPolicy: Recycle
storageClassName: slow
mountOptions:
- hard
- nfsvers=4.1
nfs:
path: /data/master01
server: 192.168.5.140
---
apiVersion: v1
kind: PersistentVolume
metadata:
name: slave01-pv
spec:
capacity:
storage: 5Gi
volumeMode: Filesystem
accessModes:
- ReadWriteMany
persistentVolumeReclaimPolicy: Recycle
storageClassName: slow
mountOptions:
- hard
- nfsvers=4.1
nfs:
path: /data/slave01
server: 192.168.5.140
---
apiVersion: v1
kind: PersistentVolume
metadata:
name: slave02-pv
spec:
capacity:
storage: 5Gi
volumeMode: Filesystem
accessModes:
- ReadWriteMany
persistentVolumeReclaimPolicy: Recycle
storageClassName: slow
mountOptions:
- hard
- nfsvers=4.1
nfs:
path: /data/slave02
server: 192.168.5.140
[root@master01 mysql]# kubectl get pv
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
master01-pv 5Gi RWX Recycle Available slow 12m
slave01-pv 5Gi RWX Recycle Available slow 12m
slave02-pv 5Gi RWX Recycle Available slow 12m
创建mysql主从配置文件
[root@master01 mysql]# cat config.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: mysql
labels:
app: mysql
app.kubernetes.io/name: mysql
data:
primary.cnf: |
# 仅在主服务器上应用此配置
[mysqld]
log-bin
replica.cnf: |
# 仅在副本服务器上应用此配置
[mysqld]
super-read-only
创建两个service
[root@master01 mysql]# cat mysql-svc.yaml
apiVersion: v1
kind: Service
metadata:
name: mysql
labels:
app: mysql
app.kubernetes.io/name: mysql
spec:
ports:
- name: mysql
port: 3306
clusterIP: None #### 这里为重点无头服务 none
selector:
app: mysql # 这里的筛选器为sts 中定义的
---
apiVersion: v1
kind: Service
metadata:
name: mysql-read #只读，从数据库
labels:
app: mysql
app.kubernetes.io/name: mysql
readonly: "true"
spec:
ports:
- name: mysql
port: 3306
selector:
app: mysql
创建statefulset：
[root@master01 mysql]# cat mysql-statefulset.yaml
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: mysql
spec:
selector:
matchLabels:
app: mysql
app.kubernetes.io/name: mysql
serviceName: mysql #service name与service筛选器中一致
replicas: 3
template:
metadata:
labels:
app: mysql
app.kubernetes.io/name: mysql
spec:
initContainers:
- name: init-mysql
image: mysql:5.7
command:
- bash
- "-c"
- |
set -ex
# Generate mysql server-id from pod ordinal index.
[[ $HOSTNAME =~ -([0-9]+)$ ]] || exit 1
ordinal=${BASH_REMATCH[1]}
echo [mysqld] > /mnt/conf.d/server-id.cnf
# Add an offset to avoid reserved server-id=0 value.
echo server-id=$((100 + $ordinal)) >> /mnt/conf.d/server-id.cnf
# Copy appropriate conf.d files from config-map to emptyDir.
if [[ $ordinal -eq 0 ]]; then
cp /mnt/config-map/primary.cnf /mnt/conf.d/
else
cp /mnt/config-map/replica.cnf /mnt/conf.d/
fi
volumeMounts:
- name: conf
mountPath: /mnt/conf.d
- name: config-map
mountPath: /mnt/config-map
- name: clone-mysql
image: ist0ne/xtrabackup #改下为国内加速的
command:
- bash
- "-c"
- |
set -ex
# Skip the clone if data already exists.
[[ -d /var/lib/mysql/mysql ]] && exit 0
# Skip the clone on primary (ordinal index 0).
[[ `hostname` =~ -([0-9]+)$ ]] || exit 1
ordinal=${BASH_REMATCH[1]}
[[ $ordinal -eq 0 ]] && exit 0
# Clone data from previous peer.
ncat --recv-only mysql-$(($ordinal-1)).mysql 3307 | xbstream -x -C /var/lib/mysql
# Prepare the backup.
xtrabackup --prepare --target-dir=/var/lib/mysql
volumeMounts:
- name: data
mountPath: /var/lib/mysql
subPath: mysql
- name: conf
mountPath: /etc/mysql/conf.d
containers:
- name: mysql
image: mysql:5.7
env:
- name: MYSQL_ALLOW_EMPTY_PASSWORD
value: "1"
ports:
- name: mysql
containerPort: 3306
volumeMounts:
- name: data
mountPath: /var/lib/mysql
subPath: mysql
- name: conf
mountPath: /etc/mysql/conf.d
resources:
requests:
cpu: 200m
memory: 500m #资源限额没这个大，我做实验的没必要这个高
livenessProbe:
exec:
command: ["mysqladmin", "ping"]
initialDelaySeconds: 30
periodSeconds: 10
timeoutSeconds: 5
readinessProbe:
exec:
# Check we can execute queries over TCP (skip-networking is off).
command: ["mysql", "-h", "127.0.0.1", "-e", "SELECT 1"]
initialDelaySeconds: 5
periodSeconds: 2
timeoutSeconds: 1
- name: xtrabackup
image: ist0ne/xtrabackup
ports:
- name: xtrabackup
containerPort: 3307
command:
- bash
- "-c"
- |
set -ex
cd /var/lib/mysql
# Determine binlog position of cloned data, if any.
if [[ -f xtrabackup_slave_info && "x$(<xtrabackup_slave_info)" != "x" ]]; then
# XtraBackup already generated a partial "CHANGE MASTER TO" query
# because we're cloning from an existing replica. (Need to remove the tailing semicolon!)
cat xtrabackup_slave_info | sed -E 's/;$//g' > change_master_to.sql.in
# Ignore xtrabackup_binlog_info in this case (it's useless).
rm -f xtrabackup_slave_info xtrabackup_binlog_info
elif [[ -f xtrabackup_binlog_info ]]; then
# We're cloning directly from primary. Parse binlog position.
[[ `cat xtrabackup_binlog_info` =~ ^(.*?)[[:space:]]+(.*?)$ ]] || exit 1
rm -f xtrabackup_binlog_info xtrabackup_slave_info
echo "CHANGE MASTER TO MASTER_LOG_FILE='${BASH_REMATCH[1]}',\
MASTER_LOG_POS=${BASH_REMATCH[2]}" > change_master_to.sql.in
fi
# Check if we need to complete a clone by starting replication.
if [[ -f change_master_to.sql.in ]]; then
echo "Waiting for mysqld to be ready (accepting connections)"
until mysql -h 127.0.0.1 -e "SELECT 1"; do sleep 1; done
echo "Initializing replication from clone position"
mysql -h 127.0.0.1 \
-e "$(<change_master_to.sql.in), \
MASTER_HOST='mysql-0.mysql', \
MASTER_USER='root', \
MASTER_PASSWORD='', \
MASTER_CONNECT_RETRY=10; \
START SLAVE;" || exit 1
# In case of container restart, attempt this at-most-once.
mv change_master_to.sql.in change_master_to.sql.orig
fi
# Start a server to send backups when requested by peers.
exec ncat --listen --keep-open --send-only --max-conns=1 3307 -c \
"xtrabackup --backup --slave-info --stream=xbstream --host=127.0.0.1 --user=root"
volumeMounts:
- name: data
mountPath: /var/lib/mysql
subPath: mysql
- name: conf
mountPath: /etc/mysql/conf.d
resources:
requests:
cpu: 100m
memory: 100Mi
volumes:
- name: conf
emptyDir: {}
- name: config-map
configMap:
name: mysql
volumeClaimTemplates: ###下面为重点，存储卷模板
- metadata:
name: data #定义name 和pvc 一样的写法不过是写到statefulset中了
spec:
accessModes: ["ReadWriteMany"] #访问模式和pv中写的一致
storageClassName: slow # classname 也与pv中一致
resources:
requests:
storage: 5Gi #小于等于pv
查看下：
[root@master01 mysql]# kubectl get pvc,pv,pods
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
persistentvolumeclaim/data-mysql-0 Bound slave01-pv 5Gi RWX slow 10m
persistentvolumeclaim/data-mysql-1 Bound slave02-pv 5Gi RWX slow 9m
persistentvolumeclaim/data-mysql-2 Bound master01-pv 5Gi RWX slow 7m22s
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
persistentvolume/master01-pv 5Gi RWX Recycle Bound default/data-mysql-2 slow 137m
persistentvolume/slave01-pv 5Gi RWX Recycle Bound default/data-mysql-0 slow 137m
persistentvolume/slave02-pv 5Gi RWX Recycle Bound default/data-mysql-1 slow 137m
NAME READY STATUS RESTARTS AGE
pod/mysql-0 2/2 Running 0 10m
pod/mysql-1 2/2 Running 0 9m
pod/mysql-2 2/2 Running 0 2m41s
pod/tom-dep-5cd5954c9-l874k 1/1 Running 7 (4h50m ago) 13d
pod/web-test-f47f968f6-wbkv4 1/1 Running 8 (4h50m ago) 14d
其实实验操作官网都有，我只是做个实验理解下里面的注意点，还有一些特点
创建statefulset资源，一个PV只能被一个PVC绑定，PVC与PV一一对应 。对应一个pod
测试下mysql读写和访问。
访问方式podname,svcname
还可以结合服务发现前面有写，svcname,namespacename.cluster.local
kubectl run test --image=mysql:5.7 -ti --rm -- bash
当前namespace下
podname.svcname
root@test:/# mysql -h mysql-0.mysql
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 494
Server version: 5.7.36-log MySQL Community Server (GPL)
Copyright (c) 2000, 2021, Oracle and/or its affiliates.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> show databases;
+------------------------+
| Database |
+------------------------+
| information_schema |
| mysql |
| performance_schema |
| sys |
| xtrabackup_backupfiles |
+------------------------+
5 rows in set (0.02 sec)
服务发现,这里的svc 会LB
[root@master01 mysql]# kubectl describe svc mysql
Name: mysql
Namespace: default
Labels: app=mysql
app.kubernetes.io/name=mysql
Annotations: <none>
Selector: app=mysql
Type: ClusterIP
IP Family Policy: SingleStack
IP Families: IPv4
IP: None
IPs: None
Port: mysql 3306/TCP
TargetPort: 3306/TCP
Endpoints: 10.244.140.92:3306,10.244.140.94:3306,10.244.196.150:3306 #服务发现会LB
Session Affinity: None
Events: <none>
[root@master01 mysql]# kubectl get pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
mysql-0 2/2 Running 0 19m 10.244.140.92 node02 <none> <none>
mysql-1 2/2 Running 0 18m 10.244.196.150 node01 <none> <none>
mysql-2 2/2 Running 0 12m 10.244.140.94 node02 <none> <none>
test 1/1 Running 0 5m48s 10.244.196.151 node01 <none> <none>
root@test:/# mysql -h mysql.default.svc.cluster.local
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 583
Server version: 5.7.36-log MySQL Community Server (GPL)
Copyright (c) 2000, 2021, Oracle and/or its affiliates.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> show databases;
+------------------------+
| Database |
+------------------------+
| information_schema |
| mysql |
| performance_schema |
| sys |
| xtrabackup_backupfiles |
+------------------------+
5 rows in set (0.06 sec)
验证下不同namespace下
利用podname.svcname 无法访问，使用服务发现
[root@master01 mysql]# kubectl run test --namespace=project --image=mysql:5.7 -ti --rm -- bash
If you don't see a command prompt, try pressing enter.
root@test:/# mysql -h mysql-0.mysql
ERROR 2005 (HY000): Unknown MySQL server host 'mysql-0.mysql' (11)
root@test:/# mysql -h mysql-1.mysql
ERROR 2005 (HY000): Unknown MySQL server host 'mysql-1.mysql' (22)
root@test:/# mysql -h mysql-2.mysql
ERROR 2005 (HY000): Unknown MySQL server host 'mysql-2.mysql' (22)
root@test:/# mysql -h mysql.default.svc.cluster.local
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 772
Server version: 5.7.36-log MySQL Community Server (GPL)
Copyright (c) 2000, 2021, Oracle and/or its affiliates.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql>
验证mysql集群
statefulset 有高可用，且pod名字不会改变。
验证mysql集群是否成功 mysql-0 新建库，mysql-1,mysql-2 是否可读
主库创建一个test库
[root@master01 ~]# kubectl run test --image=mysql:5.7 -ti --rm -- bash
If you don't see a command prompt, try pressing enter.
root@test:/# mysql -h mysql-0.mysql #mysql主库
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 211
Server version: 5.7.36-log MySQL Community Server (GPL)
Copyright (c) 2000, 2021, Oracle and/or its affiliates.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> show databases;
+------------------------+
| Database |
+------------------------+
| information_schema |
| mysql |
| performance_schema |
| sys |
| xtrabackup_backupfiles |
+------------------------+
5 rows in set (0.03 sec)
mysql> create database test1;
Query OK, 1 row affected (0.02 sec)
mysql> show databases;
+------------------------+
| Database |
+------------------------+
| information_schema |
| mysql |
| performance_schema |
| sys |
| test1 |
| xtrabackup_backupfiles |
+------------------------+
6 rows in set (0.01 sec)
利用test1查看：
[root@master01 ~]# kubectl run test1 --image=mysql:5.7 -ti --rm -- bash
If you don't see a command prompt, try pressing enter.
root@test1:/# mysql -h mysql-1.mysql #进入从库mysql-1
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 270
Server version: 5.7.36 MySQL Community Server (GPL)
Copyright (c) 2000, 2021, Oracle and/or its affiliates.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> show databases;
+------------------------+
| Database |
+------------------------+
| information_schema |
| mysql |
| performance_schema |
| sys |
| test1 |
| xtrabackup_backupfiles |
+------------------------+
6 rows in set (0.03 sec)
mysql>
总结：这里只是一个实例，,mysql配置文件可以用实际生产配置文件。pv的回收策略等，实际情况而定。后面还可以利用helm 部署一个mha 集群。。万丈高楼平地起，所以前面我才写了一个mysql-ha集群部署。
本文链接：
/archives/statefulset
版权声明：
本站所有文章除特别声明外，均采用 CC BY-NC-SA 4.0 许可协议。转载请注明来自
Emporer-Linux！
喜欢就支持一下吧