Kubernetes Docker-in-Docker Sidecar Deployment Guide - kubernetes Cheatsheets

This document outlines a Kubernetes Deployment configuration for running a Docker-in-Docker (dind) service as a sidecar container. This setup is commonly used in Continuous Integration and Continuous Deployment (CI/CD) pipelines within Kubernetes environments, allowing you to build and manage Docker images directly inside your cluster.

Understanding the Docker-in-Docker Sidecar Deployment

The provided YAML defines a Kubernetes Deployment that includes two containers within a single Pod:

docker-cmds: This container uses the standard docker image and is configured to run a command that keeps it alive indefinitely (tail -f /dev/null). It's essential for interacting with the Docker daemon. It mounts the Docker socket and sets necessary environment variables.
dind-daemon: This container uses the docker:stable-dind image, specifically designed for running Docker daemon inside a container. It requires privileged access and mounts the Docker graph storage and the Docker socket.

Both containers share access to the Docker daemon via the mounted /var/run/docker.sock. The privileged: true setting is crucial for the Docker daemon to function correctly within the Kubernetes environment.

Kubernetes Deployment YAML for DIND Sidecar

apiVersion: apps/v1
kind: Deployment
metadata:
  name: dind
  labels:
    app: dind
spec:
  replicas: 1
  selector:
    matchLabels:
      app: dind
  template:
    metadata:
      labels:
        app: dind
    spec:
      containers:
      - name: docker-cmds
        image: docker:19.03.14
        command: ['docker', 'run', 'alpine', 'tail', '-f', '/dev/null']
        securityContext:
          privileged: true
        resources:
          requests:
            cpu: "50m"
            memory: "256Mi"
        env:
          - name: DOCKER_HOST
            value: unix:///var/run/docker.sock
          - name: DOCKER_TLS_CERTDIR
            value: ""
        volumeMounts:
          - name: docker-socket-dir
            mountPath: /var/run
      - name: dind-daemon
        image: docker:stable-dind
        resources:
          limits:
            cpu: "1"
            memory: "512Mi"
          requests:
            cpu: 500m
            memory: "128Mi"
        securityContext:
          privileged: true
        volumeMounts:
          - name: docker-graph-storage
            mountPath: /var/lib/docker
          - name: docker-socket-dir
            mountPath: /var/run
      volumes:
        - name: docker-graph-storage
          emptyDir: {}
        - name: docker-socket-dir
          emptyDir: {}

Key Configuration Details

Privileged Containers: Both containers are marked as privileged: true. This is a security-sensitive setting and should be used with caution, typically only in CI/CD contexts where necessary.
Resource Requests and Limits: The YAML includes basic resource requests and limits for CPU and memory to help Kubernetes manage the Pod's resources effectively.
Volume Mounts:
- /var/run: Mounted via docker-socket-dir (an emptyDir volume) to share the Docker socket between containers.
- /var/lib/docker: Mounted via docker-graph-storage (an emptyDir volume) to store Docker images, containers, and volumes.
Environment Variables: DOCKER_HOST is set to point to the Unix socket, and DOCKER_TLS_CERTDIR is unset to disable TLS for simplicity in this example.

Best Practices and Considerations

When deploying Docker-in-Docker in Kubernetes, consider the following:

Security: Running privileged containers poses security risks. Ensure your Kubernetes cluster is secured and access to these Pods is restricted. Consider alternatives like Kaniko or Buildah if privileged containers are not feasible.
Resource Management: Carefully tune resource requests and limits based on your build workloads to prevent resource starvation or excessive consumption.
Storage: For production environments, consider using persistent volumes (e.g., PersistentVolumeClaim) instead of emptyDir for /var/lib/docker to retain Docker data across Pod restarts.
Image Versions: Pinning to specific Docker image versions (e.g., docker:19.03.14) is recommended for reproducibility.