If you are a docker user, have you tried looking for the where exactly docker images stored on disk?

I tried to dig around for the answer and found these information.

Docker Storage.

Since docker 0.7 release, docker supports various storage drivers like AUFS, devicemapper, btrfs, vfs etc.

What does this means?

Earlier, docker used to support only AUFS, which not part of upstream kernel and had no plan also. By providing support like devicemapper and btrfs, docker can be used on other distributions like RHEL/CentOS etc. Using VFS, It can be ported and supported on all *nix OS.

How docker utilizes these storage drivers?

Docker uses Copy On Write (COW) mechanism for creating containers and Images. So when any container is created, Practically container metadata is created and no data is copied. Once, some data is modified, new copy of data is made and modification happens on the new copy. This is also one of the main reason, docker can start a container so quickly almost no delay, as no copy of data is required!! I wont go much deep here, but you can go through docker docs for details.

So here apart from VFS, rest of the storage drivers support snapshot mechanism and docker utilizes the snapshot mechanism to achieve COW.

Docker folder layout.

Docker uses a /var/lib/docker folder for storing all its data and metadata. Few config files are there in /etc folder for docker daemon, But everything related to docker images and containers are managed in /var/lib/docker folder.

Lets take a look inside /var/lib/docker folder. Since I am using CentOS 7 machine for this, so storage driver is being used here is devicemapper.

Below is 1st level file/folders of this. Lets try to understand each one by one.

± tree -L 1 /var/lib/docker
/var/lib/docker/
|-- containers
|-- devicemapper
|-- graph
|-- init
|-- linkgraph.db
|-- netns
|-- repositories-devicemapper
|-- trust
|-- vfs
`-- volumes

  1. containers : This folder keeps all the containers information with folder names as container-id .

    • As shown below for one container that is created on system, the folder with its id is created.
    • All files generated by docker at creation time resides under that.
    • .json file is log file, which is used when docker log command is used.
    • This does not hold any COW data for container.``` ± tree /var/lib/docker/containers /var/lib/docker/containers -- 79edcdc9c75ca4bddd889423a3a7e880d2da3975ffdaf43a614a75b440875725 |-- 79edcdc9c75ca4bddd889423a3a7e880d2da3975ffdaf43a614a75b440875725-json.log |-- config.json |-- hostconfig.json |-- hostname |-- hosts |-- resolv.conf – resolv.conf.hash
2. devicemapper: This is the folder, under which all images and containers data is stored.

   - By default the device mapper creates two sparse files data and metadata under `devicemapper/devicemapper` folder,
     which is formatted as xfs or ext4 as per configuration.
     - These files, will be created, if dedicated physical volumes or disks assigned for data and meta-data in docker configuration.
   - All images are stored here.
   - The metadata on `/devicemapper` folder, have metadata in form of json.```
± tree -L 1  /var/lib/docker/devicemapper
/var/lib/docker/devicemapper
|-- devicemapper
|   |-- data
|   `-- metadata
|-- metadata
`-- mnt

± ls -lh   /var/lib/docker/devicemapper/devicemapper
total 4.8G
-rw-r--r--. 1 root root 250G  7月 10 04:55 data
-rw-------. 1 root root 2.0G  7月 10 05:52 metadata

  1. graph : This folder keeps layerwise image information i.e. One folder for each layer of images.

    • Folders name is uuid of layer.
    • each folder keeps information about its checksum, layersize and a Json file which keeps all information associated with image.
    • The docker history shows information of this json file.``` /var/lib/docker/graph |– 005c90a2563599e0021e6cb02d8ae34d2f948c7d05d1a6322ffbacd16932f0b7 | |– checksum | |– json | -- layersize |-- 02e7035d95ed3e570b1bbddb457a59e2fc2aabd24777f7e1a43fdfc89ca076eb |-- checksum |-- json – layersize
4. init : This folder has a binary file, which is used while creating every container. This is specific to libcontainer.

± tree -L 1 /var/lib/docker/init /var/lib/docker/init |– dockerinit-1.6.0 |– dockerinit-1.6.1 |– dockerinit-1.6.2 |– dockerinit-1.7.0 `– dockerinit-1.7.0-dev

5. linkgraph.db
6. netns
7. repositories-devicemapper
8. trust

± tree -L 1 /var/lib/docker/trust /var/lib/docker/trust `– official.json

9. volumes

/var/lib/docker/volumes -- 5aa5ee0f370733b4c5c82f82de9715ad87d1e6670bd144f602f1453d92317594 – _data |– somefile1 `– somefile4

/var/lib/docker/ ├── aufs # Storage area for AUFS driver │ ├── diff # Branch directory of layer │ ├── layers # Infomation about docker layer │ └── mnt # Mount point of aufs, root of containers ├── containers # Container configurations │ (both LXC and Docker-specific) ├── graph # Storage for the images ├── init │ └── dockerinit-0.7.3 # Used as /sbin/init in containers ├── linkgraph.db # SQLite database storing links │ and names. ├── lxc-start-unconfined -> /usr/bin/lxc-start # When starting a privileged │ container, this is used in │ lieu of lxc-start, to evade │ AppArmor confinement (which │ matches by exact path). ├── repositories-aufs # repository infomation └── volumes # Storage for “anonymous” volumes (those which are not bind-mounts)