1. What is Kind?

Kind ๋Š” ๋กœ์ปฌ ์ปดํ“จํ„ฐ ํ™˜๊ฒฝ์— ์ฟ ๋ฒ„๋„คํ‹ฐ์Šค ํด๋Ÿฌ์Šคํ„ฐ๋ฅผ ์†์‰ฝ๊ณ  ๋น ๋ฅด๊ฒŒ ์„ค์น˜ ํ•˜๊ธฐ ์œ„ํ•ด ๋งŒ๋“ค์–ด์ง„ ๋„๊ตฌ ์ž…๋‹ˆ๋‹ค.

Kind๋Š” Go ์–ธ์–ด๋ฅผ ๊ธฐ๋ฐ˜์œผ๋กœ ๋งŒ๋“ค์–ด ์กŒ์œผ๋ฉฐ, Docker ์ด๋ฏธ์ง€๋ฅผ ๊ธฐ๋ฐ˜์œผ๋กœ kubeadm์„ ์ด์šฉํ•˜์—ฌ ํด๋Ÿฌ์Šคํ„ฐ๋ฅผ ๋ฐฐํฌ ํ•ฉ๋‹ˆ๋‹ค.

kind ๊ณต์‹ ํ™ˆํŽ˜์ด์ง€ : kind.sigs.k8s.io

kind์™€ ์œ ์‚ฌํ•˜๊ฒŒ ๋ฉ€ํ‹ฐ๋…ธ๋“œ ๊ธฐ๋ฐ˜ ์ฟ ๋ฒ„๋„คํ‹ฐ์Šค ๋กœ์ปฌ ํด๋Ÿฌ์Šคํ„ฐ ๊ตฌ์ถ• ๋„๊ตฌ์—๋Š” ์•„๋ž˜์™€ ๊ฐ™์€ ๊ฒƒ๋“ค์ด ์žˆ์Šต๋‹ˆ๋‹ค.

๋„๊ตฌ๋ช…๊ณต์‹ URL
minikubehttps://minikube.sigs.k8s.io
k3shttps://k3s.io
MicroK8shttps://microk8s.io
k3dhttps://k3d.io

2. Kind ์„ค์น˜ ํ•˜๊ธฐ

์„ค์น˜ ๊ฐ€์ด๋“œ ์›๋ณธ URL : https://kind.sigs.k8s.io/docs/user/quick-start/#installation

- MacOS

  brew install kind
  

- Windows

  choco install kind
  

- Linux

  curl.exe -Lo kind-windows-amd64.exe https://kind.sigs.k8s.io/dl/v0.15.0/kind-windows-amd64
Move-Item .\kind-windows-amd64.exe c:\some-dir-in-your-PATH\kind.exe
  

- Kubectx, Kubens ์„ค์น˜

  choco install kubectx
choco install kubens
  

3. Kind ๋กœ ํด๋Ÿฌ์Šคํ„ฐ ์ƒ์„ฑ (์ดˆ๊ฐ„๋‹จ)

- ํด๋Ÿฌ์Šคํ„ฐ ์ƒ์„ฑ

  kind create cluster # Default cluster context ์ด๋ฆ„์€ 'kind' ๋กœ ์ƒ์„ฑ
kind create cluster --name dangtong # cluster context ์ด๋ฆ„์„ 'dangtong' ์œผ๋กœ ์ง€์ •
  

- ํด๋Ÿฌ์Šคํ„ฐ ์ƒ์„ฑ ํ™•์ธ

  kind get clusters
kubectl cluster-info --context dangtong
  

- ํด๋Ÿฌ์Šคํ„ฐ ์‚ญ์ œ

  kind delete cluster

kind delete clusters kind-local-cluster
  

4. ์„ค์ • ํŒŒ์ผ์„ ์ด์šฉํ•œ Kind ํด๋Ÿฌ์Šคํ„ฐ ์ƒ์„ฑ

- ์„ค์ • ํŒŒ์ผ์„ ์ด์šฉํ•œ ํด๋Ÿฌ์Šคํ„ฐ ์ƒ์„ฑ

์„ค์ •ํŒŒ์ผ์„ ์ด์šฉํ•ด์„œ kind ํด๋Ÿฌ์Šคํ„ฐ๋ฅผ ์ƒ์„ฑํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.

  kind create cluster --config kind-example-config.yaml
  

- 3๊ฐœ ๋…ธ๋“œ ํด๋Ÿฌ์Šคํ„ฐ ์ƒ์„ฑ ์˜ˆ์‹œ

3๊ฐœ ๋…ธ๋“œ(1 controller, 2worker) ํด๋Ÿฌ์Šคํ„ฐ ์„ค์ •

  # three node (two workers) cluster config
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
- role: worker
- role: worker
  

- 6๊ฐœ ๋…ธ๋“œ ํด๋Ÿฌ์Šคํ„ฐ ์ƒ์„ฑ ์˜ˆ์‹œ

  kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
- role: control-plane
- role: control-plane
- role: worker
- role: worker
- role: worker
  

- ์ฟ ๋ฒ„๋„คํ‹ฐ์Šค ๋ฒ„์ „ ์„ค์ •

์ฟ ๋ฒ„๋„คํ‹ฐ์Šค ๋ฒ„์ „์— ๋”ฐ๋ฅธ ์ด๋ฏธ์ง€๋Š” ๋งํฌ์—์„œ ํ™•์ธ ๊ฐ€๋Šฅ : https://github.com/kubernetes-sigs/kind/releases

  kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
  image: kindest/node:v1.16.4@sha256:b91a2c2317a000f3a783489dfb755064177dbc3a0b2f4147d50f04825d016f55
- role: worker
  image: kindest/node:v1.16.4@sha256:b91a2c2317a000f3a783489dfb755064177dbc3a0b2f4147d50f04825d016f55
  

- ๋„คํŠธ์›Œํฌ ์„ค์ •

  • Pod Subnet ์„ค์ •
  kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
networking:
  podSubnet: "10.244.0.0/16"
  
  • Service Subnet ์„ค์ •
  kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
networking:
  serviceSubnet: "10.96.0.0/12"
  
  • Default CNI ์„ค์ •

Caliaco ์™„ ๊ฐ™์€ 3rd party CNI ์‚ฌ์šฉ์„ ์œ„ํ•ด์„œ๋Š” default CNI ์„ค์น˜๋ฅผ ํ•˜์ง€ ๋ง์•„์•ผ ํ•ฉ๋‹ˆ๋‹ค.

  kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
networking:
  # default CNI๊ฐ€ ์„ค์น˜ ๋˜์ง€ ์•Š์Šต๋‹ˆ๋‹ค.
  disableDefaultCNI: true
  
  • kube-proxy ๋ชจ๋“œ ์„ค์ •

iptables ๋˜๋Š” IPVS ์ค‘์— ์„ ํƒํ•ด์„œ ์‚ฌ์šฉ ๊ฐ€๋Šฅ. default ๋Š” iptables

  kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
networking:
  kubeProxyMode: "ipvs"
  

- ์ตœ์ข… ํด๋Ÿฌ์Šคํ„ฐ ์ƒ์„ฑ

  kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
name: cwave-cluster
nodes:
- role: control-plane
  kubeadmConfigPatches:
  - |
    kind: InitConfiguration
    nodeRegistration:
      kubeletExtraArgs:
        node-labels: "ingress-ready=true"
  image: kindest/node:v1.32.5
  extraPortMappings:
  - containerPort: 80
    hostPort: 80
    protocol: TCP
  - containerPort: 443
    hostPort: 443
    protocol: TCP
- role: worker
  image: kindest/node:v1.32.5
- role: worker
  image: kindest/node:v1.32.5
networking:
  serviceSubnet: "10.120.0.0/16"
  podSubnet: "10.110.0.0/16"
  
  • ํด๋Ÿฌ์Šคํ„ฐ ์ƒ์„ฑ
  # ์ƒ์„ฑ
kind create cluster  --config ./3-node-cluster.yml

# ์‚ญ์ œ
kind delete cluster --name cwave-cluster
  
  • ํด๋Ÿฌ์Šคํ„ฐ ์ ‘์† ์ •๋ณด ํ™•์ธ
  kind get kubeconfig --internal --name cwave-cluster
  
  • IDE ์ปจํ…Œ์ด๋„ˆ์— Kind ๋„คํŠธ์›Œํฌ ์ฃผ์„์ œ๊ฑฐ
  name: "aws-cicd-practice"
services:
  code-server:
    image: dangtong76/cicd-devops-ide:arm64-v2 
    container_name: "ide"
    networks: # ๋„คํŠธ์›Œํฌ ํ•ญ๋ชฉ ์ฃผ์„์ œ๊ฑฐ
      - kind_network 
    environment:
      AUTH: none
      #FILE__PASSWORD: /run/secrets/code-server-password
    env_file:
      - .env
    working_dir: /code
    ports:
      - "8080:8080" # istory-web
      - "1314:1314" # hugo port 1
      - "1315:1315" # hugo port 2
      - "8444:8443" # vscode service port
      - "5500:5500"
    # secrets:s
    #   - code-server-password
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - devops-cicd-apps:/code/devops-cicd-apps
      - devops-cicd-vscode:/config
networks: # ๋„คํŠธ์›Œํฌ ํ•ญ๋ชฉ ์ฃผ์„์ œ๊ฑฐ
  kind_network:
    name: kind
    external: true
volumes:
  devops-cicd-apps:
    external: true
    name: devops-cicd-apps
  devops-cicd-vscode:
    external: true
    name: devops-cicd-vscode
  
  • ~/.kube/config ํŒŒ์ผ์˜ API ์ ‘์† ์ •๋ณด ์ˆ˜์ •
  docker network inspect kind # cwave-cluster-control-plane ์˜ IP์ฃผ์†Œ ํ™•์ธ

vi ~/.kube/config
  

์•„๋ž˜์™€ ๊ฐ™์ด kubectl ์ ‘์† ์ •๋ณด ์ˆ˜์ •

  apiVersion: v1
clusters:
- cluster:
    server: https://<cwave-cluster-control-plane ์˜ IP์ฃผ์†Œ>:6443
  name: kind-cwave-cluster
  

API ์„œ๋ฒ„๊ฐ€ ์ ‘์† ๋˜๋Š”์ง€ ํ™•์ธํ•˜๊ธฐ

  kubectl get no 

### ์ถœ๋ ฅ์ด ์•„๋ž˜์™€ ๊ฐ™์€ ํ˜•์‹์œผ๋กœ ๋‚˜์˜ค๋ฉด ์ •์ƒ
root@85bed0161b08:~/.kube# kubectl get no
NAME                          STATUS   ROLES           AGE   VERSION
cwave-cluster-control-plane   Ready    control-plane   30m   v1.32.5
cwave-cluster-worker          Ready    <none>          30m   v1.32.5
cwave-cluster-worker2         Ready    <none>          30m   v1.32.5
  

5. MetalLB ์„ค์น˜

  • MetalLB ์„ค์น˜
  # kubectl ์ด์šฉ
kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.15.2/config/manifests/metallb-native.yaml

# helm ์ฐจํŠธ ์ด์šฉ
helm repo add metallb https://metallb.github.io/metallb
helm install metallb metallb/metallb
  
  • MetalLB ์„ค์ •
  apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
  name: cwave-pool
  namespace: metallb-system
spec:
  addresses:
  - 192.168.247.100-192.168.247.110 # set by make target
---
apiVersion: metallb.io/v1beta1
kind: L2Advertisement
metadata:
  name: cwave-loadbalancer-advertisement
  namespace: metallb-system
spec:
  ipAddressPools:
  - cwave-pool
  

6. Ingress ๋ฐ LoadBalancer ์„ค์ •

Ingress ๋ฐ Loadbalancer ๋ฅผ ์„ค์ •ํ•˜๊ธฐ ์œ„ํ•ด์„œ๋Š” KIND ๋ฅผ ์ด์šฉํ•œ ํด๋Ÿฌ์Šคํ„ฐ ์ƒ์„ฑ์‹œ extraPortMapping ์„ค์ •์„ ํ•˜๊ณ , kubeadmํˆด์„ ํ†ตํ•ด custom node label ์„ ๋…ธ๋“œ์— ์„ค์ •ํ•ด์•ผ ํ•ฉ๋‹ˆ๋‹ค.

- Ingress ๊ฐ€๋Šฅํ•œ ํด๋Ÿฌ์Šคํ„ฐ ์ƒ์„ฑ

kind ํด๋Ÿฌ์Šคํ„ฐ๋ฅผ extraPortMappings ๋ฐ node-lables ์„ค์ •๊ณผ ํ•จ๊ป˜ ์ƒ์„ฑ ํ•ฉ๋‹ˆ๋‹ค.

  • ExtreaPortMappings : ๋กœ์ปฌ ํ˜ธ์ŠคํŠธ๊ฐ€ 80 ๋ฐ 443 ํฌํŠธ๋ฅผ ํ†ตํ•ด Ingress Controller๋กœ ์š”์ฒญ์ด ๊ฐ€๋Šฅํ•˜๊ฒŒ ์„ค์ •ํ•ฉ๋‹ˆ๋‹ค.
  • node–labels : Ingress Controller ๊ฐ€ ํŠน์ • ๋ผ๋ฒจ์„ ๊ฐ€์ง„ ๋…ธ๋“œ์—์„œ๋งŒ ์ˆ˜ํ–‰ ๋˜๋„๋ก ํ•ฉ๋‹ˆ๋‹ค.
  cat <<EOF | kind create cluster --config=-
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
  kubeadmConfigPatches:
  - |
    kind: InitConfiguration
    nodeRegistration:
      kubeletExtraArgs:
        node-labels: "ingress-ready=true"
  extraPortMappings:
  - containerPort: 80
    hostPort: 80
    protocol: TCP
  - containerPort: 443
    hostPort: 443
    protocol: TCP
EOF
  

- Countour Ingress ์ƒ์„ฑ

  • Contour ์„ค์น˜
  kubectl apply -f https://projectcontour.io/quickstart/contour.yaml
  
  • Contour ์„ค์ • ์—…๋ฐ์ดํŠธ
  kubectl patch daemonsets -n projectcontour envoy -p '{"spec":{"template":{"spec":{"nodeSelector":{"ingress-ready":"true"},"tolerations":[{"key":"node-role.kubernetes.io/control-plane","operator":"Equal","effect":"NoSchedule"},{"key":"node-role.kubernetes.io/master","operator":"Equal","effect":"NoSchedule"}]}}}}'
  

- Kong Ingress ์ƒ์„ฑ

  • Kong ์„ค์น˜
  kubectl apply -f https://raw.githubusercontent.com/Kong/kubernetes-ingress-controller/master/deploy/single/all-in-one-dbless.yaml
  
  • Kong ์„ค์ • ์—…๋ฐ์ดํŠธ
  kubectl patch deployment -n kong ingress-kong -p '{"spec":{"template":{"spec":{"containers":[{"name":"proxy","ports":[{"containerPort":8000,"hostPort":80,"name":"proxy","protocol":"TCP"},{"containerPort":8443,"hostPort":443,"name":"proxy-ssl","protocol":"TCP"}]}],"nodeSelector":{"ingress-ready":"true"},"tolerations":[{"key":"node-role.kubernetes.io/control-plane","operator":"Equal","effect":"NoSchedule"},{"key":"node-role.kubernetes.io/master","operator":"Equal","effect":"NoSchedule"}]}}}}'
  

- Nginx Ingress ์ƒ์„ฑ

  • Nginx ์„ค์น˜
  kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/main/deploy/static/provider/kind/deploy.yaml
  
  • Nginx ์„ค์ • ์—…๋ฐ์ดํŠธ
  kubectl wait --namespace ingress-nginx \
  --for=condition=ready pod \
  --selector=app.kubernetes.io/component=controller \
  --timeout=90s
  

- Ingress ์‚ฌ์šฉ ์˜ˆ์ œ

  kind: Pod
apiVersion: v1
metadata:
  name: foo-app
  labels:
    app: foo
spec:
  containers:
  - name: foo-app
    image: hashicorp/http-echo:0.2.3
    args:
    - "-text=foo"
---
kind: Service
apiVersion: v1
metadata:
  name: foo-service
spec:
  selector:
    app: foo
  ports:
  # Default port used by the image
  - port: 5678
---
kind: Pod
apiVersion: v1
metadata:
  name: bar-app
  labels:
    app: bar
spec:
  containers:
  - name: bar-app
    image: hashicorp/http-echo:0.2.3
    args:
    - "-text=bar"
---
kind: Service
apiVersion: v1
metadata:
  name: bar-service
spec:
  selector:
    app: bar
  ports:
  # Default port used by the image
  - port: 5678
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: example-ingress
spec:
  rules:
  - http:
      paths:
      - pathType: Prefix
        path: "/foo"
        backend:
          service:
            name: foo-service
            port:
              number: 5678
      - pathType: Prefix
        path: "/bar"
        backend:
          service:
            name: bar-service
            port:
              number: 5678
---