๐ฅ๏ธ 02. ๋ก์ปฌ K8S ํด๋ฌ์คํฐ ์์ฑ
1. What is Kind?
Kind ๋ ๋ก์ปฌ ์ปดํจํฐ ํ๊ฒฝ์ ์ฟ ๋ฒ๋คํฐ์ค ํด๋ฌ์คํฐ๋ฅผ ์์ฝ๊ณ ๋น ๋ฅด๊ฒ ์ค์น ํ๊ธฐ ์ํด ๋ง๋ค์ด์ง ๋๊ตฌ ์ ๋๋ค.
Kind๋ Go ์ธ์ด๋ฅผ ๊ธฐ๋ฐ์ผ๋ก ๋ง๋ค์ด ์ก์ผ๋ฉฐ, Docker ์ด๋ฏธ์ง๋ฅผ ๊ธฐ๋ฐ์ผ๋ก kubeadm์ ์ด์ฉํ์ฌ ํด๋ฌ์คํฐ๋ฅผ ๋ฐฐํฌ ํฉ๋๋ค.
kind ๊ณต์ ํํ์ด์ง : kind.sigs.k8s.io
kind์ ์ ์ฌํ๊ฒ ๋ฉํฐ๋ ธ๋ ๊ธฐ๋ฐ ์ฟ ๋ฒ๋คํฐ์ค ๋ก์ปฌ ํด๋ฌ์คํฐ ๊ตฌ์ถ ๋๊ตฌ์๋ ์๋์ ๊ฐ์ ๊ฒ๋ค์ด ์์ต๋๋ค.
| ๋๊ตฌ๋ช | ๊ณต์ URL |
|---|---|
| minikube | https://minikube.sigs.k8s.io |
| k3s | https://k3s.io |
| MicroK8s | https://microk8s.io |
| k3d | https://k3d.io |
2. Kind ์ค์น ํ๊ธฐ
์ค์น ๊ฐ์ด๋ ์๋ณธ URL : https://kind.sigs.k8s.io/docs/user/quick-start/#installation
- MacOS
brew install kind
- Windows
choco install kind
- Linux
curl.exe -Lo kind-windows-amd64.exe https://kind.sigs.k8s.io/dl/v0.15.0/kind-windows-amd64
Move-Item .\kind-windows-amd64.exe c:\some-dir-in-your-PATH\kind.exe
- Kubectx, Kubens ์ค์น
choco install kubectx
choco install kubens
3. Kind ๋ก ํด๋ฌ์คํฐ ์์ฑ (์ด๊ฐ๋จ)
- ํด๋ฌ์คํฐ ์์ฑ
kind create cluster # Default cluster context ์ด๋ฆ์ 'kind' ๋ก ์์ฑ
kind create cluster --name dangtong # cluster context ์ด๋ฆ์ 'dangtong' ์ผ๋ก ์ง์
- ํด๋ฌ์คํฐ ์์ฑ ํ์ธ
kind get clusters
kubectl cluster-info --context dangtong
- ํด๋ฌ์คํฐ ์ญ์
kind delete cluster
kind delete clusters kind-local-cluster
4. ์ค์ ํ์ผ์ ์ด์ฉํ Kind ํด๋ฌ์คํฐ ์์ฑ
- ์ค์ ํ์ผ์ ์ด์ฉํ ํด๋ฌ์คํฐ ์์ฑ
์ค์ ํ์ผ์ ์ด์ฉํด์ kind ํด๋ฌ์คํฐ๋ฅผ ์์ฑํ ์ ์์ต๋๋ค.
kind create cluster --config kind-example-config.yaml
- 3๊ฐ ๋ ธ๋ ํด๋ฌ์คํฐ ์์ฑ ์์
3๊ฐ ๋ ธ๋(1 controller, 2worker) ํด๋ฌ์คํฐ ์ค์
# three node (two workers) cluster config
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
- role: worker
- role: worker
- 6๊ฐ ๋ ธ๋ ํด๋ฌ์คํฐ ์์ฑ ์์
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
- role: control-plane
- role: control-plane
- role: worker
- role: worker
- role: worker
- ์ฟ ๋ฒ๋คํฐ์ค ๋ฒ์ ์ค์
์ฟ ๋ฒ๋คํฐ์ค ๋ฒ์ ์ ๋ฐ๋ฅธ ์ด๋ฏธ์ง๋ ๋งํฌ์์ ํ์ธ ๊ฐ๋ฅ : https://github.com/kubernetes-sigs/kind/releases
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
image: kindest/node:v1.16.4@sha256:b91a2c2317a000f3a783489dfb755064177dbc3a0b2f4147d50f04825d016f55
- role: worker
image: kindest/node:v1.16.4@sha256:b91a2c2317a000f3a783489dfb755064177dbc3a0b2f4147d50f04825d016f55
- ๋คํธ์ํฌ ์ค์
- Pod Subnet ์ค์
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
networking:
podSubnet: "10.244.0.0/16"
- Service Subnet ์ค์
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
networking:
serviceSubnet: "10.96.0.0/12"
- Default CNI ์ค์
Caliaco ์ ๊ฐ์ 3rd party CNI ์ฌ์ฉ์ ์ํด์๋ default CNI ์ค์น๋ฅผ ํ์ง ๋ง์์ผ ํฉ๋๋ค.
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
networking:
# default CNI๊ฐ ์ค์น ๋์ง ์์ต๋๋ค.
disableDefaultCNI: true
- kube-proxy ๋ชจ๋ ์ค์
iptables ๋๋ IPVS ์ค์ ์ ํํด์ ์ฌ์ฉ ๊ฐ๋ฅ. default ๋ iptables
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
networking:
kubeProxyMode: "ipvs"
- ์ต์ข ํด๋ฌ์คํฐ ์์ฑ
- ํด๋ฌ์คํฐ yaml ์์ฑ
- ํ์ผ๋ช : 3-node-cluster.yml
- ๋ ธ๋ ์ด๋ฏธ์ง ๋ฒ์ ์ฐธ์กฐ : https://github.com/kubernetes-sigs/kind/releases
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
name: cwave-cluster
nodes:
- role: control-plane
kubeadmConfigPatches:
- |
kind: InitConfiguration
nodeRegistration:
kubeletExtraArgs:
node-labels: "ingress-ready=true"
image: kindest/node:v1.32.5
extraPortMappings:
- containerPort: 80
hostPort: 80
protocol: TCP
- containerPort: 443
hostPort: 443
protocol: TCP
- role: worker
image: kindest/node:v1.32.5
- role: worker
image: kindest/node:v1.32.5
networking:
serviceSubnet: "10.120.0.0/16"
podSubnet: "10.110.0.0/16"
- ํด๋ฌ์คํฐ ์์ฑ
# ์์ฑ
kind create cluster --config ./3-node-cluster.yml
# ์ญ์
kind delete cluster --name cwave-cluster
- ํด๋ฌ์คํฐ ์ ์ ์ ๋ณด ํ์ธ
kind get kubeconfig --internal --name cwave-cluster
- IDE ์ปจํ ์ด๋์ Kind ๋คํธ์ํฌ ์ฃผ์์ ๊ฑฐ
name: "aws-cicd-practice"
services:
code-server:
image: dangtong76/cicd-devops-ide:arm64-v2
container_name: "ide"
networks: # ๋คํธ์ํฌ ํญ๋ชฉ ์ฃผ์์ ๊ฑฐ
- kind_network
environment:
AUTH: none
#FILE__PASSWORD: /run/secrets/code-server-password
env_file:
- .env
working_dir: /code
ports:
- "8080:8080" # istory-web
- "1314:1314" # hugo port 1
- "1315:1315" # hugo port 2
- "8444:8443" # vscode service port
- "5500:5500"
# secrets:s
# - code-server-password
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- devops-cicd-apps:/code/devops-cicd-apps
- devops-cicd-vscode:/config
networks: # ๋คํธ์ํฌ ํญ๋ชฉ ์ฃผ์์ ๊ฑฐ
kind_network:
name: kind
external: true
volumes:
devops-cicd-apps:
external: true
name: devops-cicd-apps
devops-cicd-vscode:
external: true
name: devops-cicd-vscode
- ~/.kube/config ํ์ผ์ API ์ ์ ์ ๋ณด ์์
docker network inspect kind # cwave-cluster-control-plane ์ IP์ฃผ์ ํ์ธ
vi ~/.kube/config
์๋์ ๊ฐ์ด kubectl ์ ์ ์ ๋ณด ์์
apiVersion: v1
clusters:
- cluster:
server: https://<cwave-cluster-control-plane ์ IP์ฃผ์>:6443
name: kind-cwave-cluster
API ์๋ฒ๊ฐ ์ ์ ๋๋์ง ํ์ธํ๊ธฐ
kubectl get no
### ์ถ๋ ฅ์ด ์๋์ ๊ฐ์ ํ์์ผ๋ก ๋์ค๋ฉด ์ ์
root@85bed0161b08:~/.kube# kubectl get no
NAME STATUS ROLES AGE VERSION
cwave-cluster-control-plane Ready control-plane 30m v1.32.5
cwave-cluster-worker Ready <none> 30m v1.32.5
cwave-cluster-worker2 Ready <none> 30m v1.32.5
5. MetalLB ์ค์น
- MetalLB ์ค์น
# kubectl ์ด์ฉ
kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.15.2/config/manifests/metallb-native.yaml
# helm ์ฐจํธ ์ด์ฉ
helm repo add metallb https://metallb.github.io/metallb
helm install metallb metallb/metallb
- MetalLB ์ค์
apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
name: cwave-pool
namespace: metallb-system
spec:
addresses:
- 192.168.247.100-192.168.247.110 # set by make target
---
apiVersion: metallb.io/v1beta1
kind: L2Advertisement
metadata:
name: cwave-loadbalancer-advertisement
namespace: metallb-system
spec:
ipAddressPools:
- cwave-pool
6. Ingress ๋ฐ LoadBalancer ์ค์
Ingress ๋ฐ Loadbalancer ๋ฅผ ์ค์ ํ๊ธฐ ์ํด์๋ KIND ๋ฅผ ์ด์ฉํ ํด๋ฌ์คํฐ ์์ฑ์ extraPortMapping ์ค์ ์ ํ๊ณ , kubeadmํด์ ํตํด custom node label ์ ๋ ธ๋์ ์ค์ ํด์ผ ํฉ๋๋ค.
- Ingress ๊ฐ๋ฅํ ํด๋ฌ์คํฐ ์์ฑ
kind ํด๋ฌ์คํฐ๋ฅผ extraPortMappings ๋ฐ node-lables ์ค์ ๊ณผ ํจ๊ป ์์ฑ ํฉ๋๋ค.
- ExtreaPortMappings : ๋ก์ปฌ ํธ์คํธ๊ฐ 80 ๋ฐ 443 ํฌํธ๋ฅผ ํตํด Ingress Controller๋ก ์์ฒญ์ด ๊ฐ๋ฅํ๊ฒ ์ค์ ํฉ๋๋ค.
- node–labels : Ingress Controller ๊ฐ ํน์ ๋ผ๋ฒจ์ ๊ฐ์ง ๋ ธ๋์์๋ง ์ํ ๋๋๋ก ํฉ๋๋ค.
cat <<EOF | kind create cluster --config=-
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
kubeadmConfigPatches:
- |
kind: InitConfiguration
nodeRegistration:
kubeletExtraArgs:
node-labels: "ingress-ready=true"
extraPortMappings:
- containerPort: 80
hostPort: 80
protocol: TCP
- containerPort: 443
hostPort: 443
protocol: TCP
EOF
- Countour Ingress ์์ฑ
- Contour ์ค์น
kubectl apply -f https://projectcontour.io/quickstart/contour.yaml
- Contour ์ค์ ์ ๋ฐ์ดํธ
kubectl patch daemonsets -n projectcontour envoy -p '{"spec":{"template":{"spec":{"nodeSelector":{"ingress-ready":"true"},"tolerations":[{"key":"node-role.kubernetes.io/control-plane","operator":"Equal","effect":"NoSchedule"},{"key":"node-role.kubernetes.io/master","operator":"Equal","effect":"NoSchedule"}]}}}}'
- Kong Ingress ์์ฑ
- Kong ์ค์น
kubectl apply -f https://raw.githubusercontent.com/Kong/kubernetes-ingress-controller/master/deploy/single/all-in-one-dbless.yaml
- Kong ์ค์ ์ ๋ฐ์ดํธ
kubectl patch deployment -n kong ingress-kong -p '{"spec":{"template":{"spec":{"containers":[{"name":"proxy","ports":[{"containerPort":8000,"hostPort":80,"name":"proxy","protocol":"TCP"},{"containerPort":8443,"hostPort":443,"name":"proxy-ssl","protocol":"TCP"}]}],"nodeSelector":{"ingress-ready":"true"},"tolerations":[{"key":"node-role.kubernetes.io/control-plane","operator":"Equal","effect":"NoSchedule"},{"key":"node-role.kubernetes.io/master","operator":"Equal","effect":"NoSchedule"}]}}}}'
- Nginx Ingress ์์ฑ
- Nginx ์ค์น
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/main/deploy/static/provider/kind/deploy.yaml
- Nginx ์ค์ ์ ๋ฐ์ดํธ
kubectl wait --namespace ingress-nginx \
--for=condition=ready pod \
--selector=app.kubernetes.io/component=controller \
--timeout=90s
- Ingress ์ฌ์ฉ ์์
kind: Pod
apiVersion: v1
metadata:
name: foo-app
labels:
app: foo
spec:
containers:
- name: foo-app
image: hashicorp/http-echo:0.2.3
args:
- "-text=foo"
---
kind: Service
apiVersion: v1
metadata:
name: foo-service
spec:
selector:
app: foo
ports:
# Default port used by the image
- port: 5678
---
kind: Pod
apiVersion: v1
metadata:
name: bar-app
labels:
app: bar
spec:
containers:
- name: bar-app
image: hashicorp/http-echo:0.2.3
args:
- "-text=bar"
---
kind: Service
apiVersion: v1
metadata:
name: bar-service
spec:
selector:
app: bar
ports:
# Default port used by the image
- port: 5678
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: example-ingress
spec:
rules:
- http:
paths:
- pathType: Prefix
path: "/foo"
backend:
service:
name: foo-service
port:
number: 5678
- pathType: Prefix
path: "/bar"
backend:
service:
name: bar-service
port:
number: 5678
---