ec2 image cloud-init docker startup issue #219
Description
Output of docker version:
Client:
Version: 1.10.3
API version: 1.22
Package version: docker-common-1.10.3-46.el7.centos.14.x86_64
Go version: go1.6.3
Git commit: cb079f6-unsupported
Built: Fri Sep 16 13:24:25 2016
OS/Arch: linux/amd64
Server:
Version: 1.10.3
API version: 1.22
Package version: docker-common-1.10.3-46.el7.centos.14.x86_64
Go version: go1.6.3
Git commit: cb079f6-unsupported
Built: Fri Sep 16 13:24:25 2016
OS/Arch: linux/amd64
Output of docker info:
Containers: 1
Running: 1
Paused: 0
Stopped: 0
Images: 2
Server Version: 1.10.3
Storage Driver: devicemapper
Pool Name: atomicos-docker--pool
Pool Blocksize: 524.3 kB
Base Device Size: 10.74 GB
Backing Filesystem: xfs
Data file:
Metadata file:
Data Space Used: 826.8 MB
Data Space Total: 20.07 GB
Data Space Available: 19.24 GB
Metadata Space Used: 192.5 kB
Metadata Space Total: 54.53 MB
Metadata Space Available: 54.33 MB
Udev Sync Supported: true
Deferred Removal Enabled: true
Deferred Deletion Enabled: true
Deferred Deleted Device Count: 0
Library Version: 1.02.107-RHEL7 (2016-06-09)
Execution Driver: native-0.2
Logging Driver: json-file
Plugins:
Volume: local
Network: null host bridge
Kernel Version: 3.10.0-327.36.3.el7.x86_64
Operating System: CentOS Linux 7 (Core)
OSType: linux
Architecture: x86_64
Number of Docker Hooks: 2
CPUs: 1
Total Memory: 991.6 MiB
Name: ip-10-216-10-43.ec2.internal
ID: M6XS:4YA3:PQCG:3U4A:5REJ:LF3C:FRU2:5ARN:CC63:3ZC7:BMJ7:HOWJ
Registries: docker.io (secure)
Additional environment details (AWS, VirtualBox, physical, etc.):
- AWS VPC
- us-east-1 region
- official ami-cfeca0d8 AMI
Steps to reproduce the issue:
- start an instance with the following/similar user-data:
#cloud-configs
write_files:
- path: /etc/systemd/system/docker-zk-exhibitor.service
permissions: 0644
owner: root
content: |
[Unit]
Description=Run a zk-exhibitor container
Requires=docker.service
After=docker.service
[Service]
Restart=always
TimeoutStartSec=30m
ExecStartPre=/usr/bin/bash -c 'eval $(/usr/bin/docker run --rm dockerhubtungsten/awscli ecr get-login --region us-east-1)'
ExecStartPre=/usr/bin/docker pull myaccount.dkr.ecr.us-east-1.amazonaws.com/tn/zk-exhibitor:latest
ExecStartPre=-/usr/bin/docker rm zk-exhibitor
ExecStart=/usr/bin/docker run --rm -p 8181:8181 -p 2181:2181 -p 2888:2888 -p 3888:3888 --name zk-exhibitor -e S3_BUCKET=mybucket-us-east-1 -e S3_PREFIX=1_ -e AWS_REGION=us-east-1 -e HOSTNAME=%H myaccount.dkr.ecr.us-east-1.amazonaws.com/tn/zk-exhibitor:latest
ExecStop=/usr/bin/docker stop -t 2 zk-exhibitor
[Install]
WantedBy=multi-user.target
runcmd:
- systemctl daemon-reload
- systemctl enable docker-zk-exhibitor.service
- systemctl start docker.service
- systemctl start docker-zk-exhibitor.service
Describe the results you received:
docker daemon is not running when the instance starts. It's not possible to start it with systemctl start docker.service because it hangs
Nothing in journalctl -u docker.service
The way to make it to start is actually to do the following steps
- lvremove atomicos/docker-pool -y
- rm -rf /var/lib/docker/*
- docker-storage-setup --reset
- docker-storage-setup
- reboot
Describe the results you expected:
docker daemon running upon startup
Additional information you deem important (e.g. issue happens only occasionally):
The issue is somehow related to device-mapper setup. Should be pretty easy to reproduce