Diagnosing Docker Container CrashLoopBackOff and Restart Cycles

When a Docker container continually restarts or remains trapped in a restart loop, it indicates that the container’s PID 1 process exited immediately after invocation. Unlike virtual machines, a Docker container runs only as long as its foreground process stays alive.

Production Note: A container exit code of 137 specifically denotes an out-of-memory SIGKILL dispatched by the host Linux kernel OOM killer.

At a Glance

Check container exit codes with docker ps -a.
Inspect application startup logs using docker logs --tail 100.
Check docker inspect for the "OOMKilled": true flag.
Override the entrypoint with an interactive shell /bin/sh to test missing environment variables.
Adjust memory limits and restart policies in docker-compose.yml.

Prerequisites

Before troubleshooting container loops:

Root or sudo access with Docker daemon permissions.
Docker CLI and Docker Compose installed.
Access to host kernel messages (dmesg).

Step 1: Inspect Container Status and Exit Code Signatures

List all active and exited containers along with their exit codes:

# List containers with exit status codes
docker ps -a --format "table {{.ID}}\t{{.Names}}\t{{.Status}}\t{{.Image}}"

Common Exit Code Meanings:

Exit 0: Process completed normally (e.g., a one-off batch script exited because it had no persistent foreground server).
Exit 1 / Exit 2: Application error, missing environment variable, or configuration syntax failure.
Exit 137 (128 + 9): Container terminated by Linux kernel OOM (Out Of Memory) Killer via SIGKILL.
Exit 139 (128 + 11): Segmentation fault in compiled application binary.

Step 2: Tail Container Logs and State Metadata

Inspect the log output emitted before the crash:

# View last 100 log lines with timestamps
docker logs --tail 100 -t CONTAINER_NAME

# Follow logs in real-time
docker logs -f CONTAINER_NAME

Inspect why the Docker daemon terminated the container:

# Inspect container state in JSON
docker inspect CONTAINER_NAME | grep -A 12 '"State":'

Step 3: Debug Entrypoint Scripts Interactively

If the container crashes before logs can be flushed, override the entrypoint with an interactive debug shell:

# Launch interactive container instance
docker run -it --rm --entrypoint /bin/sh IMAGE_NAME

Once inside the container:

Validate required environment variables: env
Test database/network connectivity: nc -zv DB_HOST 3306 or curl -Iv ENDPOINT
Verify directory permissions: ls -la /var/www/html

Step 4: Resolve OOM Kills (Exit 137)

If the container was OOM killed, increase memory reservations in docker-compose.yml:

version: '3.8'
services:
  app:
    image: node:20-alpine
    deploy:
      resources:
        limits:
          memory: 2048M
        reservations:
          memory: 512M

Troubleshooting

Problem: “Container with restart: always creates extreme CPU load”

Possible cause: A rapid crash loop without backoff delay continuously forks new containers, consuming host CPU resources.

Solution: Stop the container immediately:

docker stop CONTAINER_NAME

Then configure a health check or rate-limited restart policy.

Verify the Configuration

Verify that the container runs steadily without incrementing restart counters:

# Monitor container uptime
docker ps --filter "name=CONTAINER_NAME"

Production Checklist

Identified exit code with docker ps -a.
Checked docker logs for fatal application exceptions.
Verified host memory available with free -h.
Configured appropriate container memory limits in compose file.
Tested network connectivity to upstream databases.

Frequently Asked Questions

Why does a container exit immediately with Exit Code 0?

Containers require a foreground process (such as nginx -g 'daemon off;' or node server.js). If your startup script forks a daemon into the background, PID 1 exits and Docker halts the container.

Need Help Managing Production Containers?

If your team runs Docker or Kubernetes microservices and needs 24/7 reliability engineering and container optimization, connect with ServerCare360.