This chapter covers new Docker commands and details on how to share data between host system and containerized applications. Learn when and how to use persistent volumes and access host filesystems.

Intermediate Docker Commands

Help at a glance

Existing containers

Automatic container cleanup

Mounting the host filesystem

Passing a file to a running container

Gone, but not forgotten

Different methods, same result

Persistent volumes

Creating a container volume

Different containers, same volume

Commands and filesystems

Learn to expose networking services in Docker, create Docker networks, and use Dockerfiles to maintain network settings.

Networking refresher

What's my IP?

Valid ports

Making network services available in Docker

Mapping a port

Multiple containers, same host port?

Exposing ports with Dockerfiles

So many options

Mapping a container port

Using ephemeral mapping

Docker networks

Creating a docker network

Adding containers to a network

IP Ranges

Removing container networks

Networking

Learn to use Docker's advanced image management and build tools to create compact effective container images. Create multi-platform images capable of running on different processor architectures.

Optimizing Docker images

Contain your concerns

Docker image exploration

Determining the best image to use

Understanding layers

Layer cake

How many layers?

Multi-stage builds

package main
import "fmt"

func main() {
  fmt.Println("Application running...")
}


FROM golang:1.21-alpine3.19
WORKDIR /src
COPY /src/main.go /src/main.go

# Build the go application from source code
RUN go build -o /bin/app_runner /src/main.go

CMD ["/bin/app_runner"]


Run the following commands:
  - docker build . -t single-stage-app
  - docker run single-stage-app
  - docker images
Check the size of the single-stage-app image


Single-stage build

FROM golang:1.21-alpine3.19 AS gobuild

WORKDIR /src
COPY /src/main.go /src/main.go

# Build the go application from source code
RUN go build -o /bin/app_runner /src/main.go

# Create application in standalone container
# Update with the image name from instructions

FROM ____

# Update with name of previous build stage
COPY --from=____ /bin/app_runner /bin/app_runner

CMD ["/bin/app_runner"]


FROM golang:1.21-alpine3.19 AS gobuild

WORKDIR /src
COPY src/main.go /src/main.go

# Build the go application from source code
RUN go build -o /bin/app_runner /src/main.go

# Create application in standalone container
# Update with the image name from instructions

FROM scratch

# Update with name of previous build stage
COPY --from=gobuild /bin/app_runner /bin/app_runner

CMD ["/bin/app_runner"]


Update the Dockerfile as shown in the example.

Once complete, run the following commands in the terminal:

- docker build . -t multi-stage-app
- docker run multi-stage-app
- docker images

Check the size of the single-stage-app and multi-stage-app
containers.


Multi-stage build

Docker engine build steps

Understanding multi-stage builds

Multi-platform builds

Multi-platform build order

Multi-platform steps

#!/bin/bash

# Note this is an example of string concatenation in jq
#
# There is not a Platform attribute returned by docker image inspect
# so we create our own. This is simply a string returned as Platform,
# consisting of the .Os attribute and the .Architecture attribute joined
# with a slash to produce the Platform field.
#
# This is outside the realm of the course, but is hopefully a further
# example of the power available within the jq tool.

docker image inspect multi-platform-app | jq '.[0] | {Platform: (.Os + "/" + .Architecture)}'


# Update the FROM line to support a multi-platform build
FROM ____=$BUILDPLATFORM golang:1.21-alpine3.19 AS gobuild

WORKDIR /src
COPY src/main.go /src/main.go

# Define the option to pull environment variables from the host
____ TARGETOS TARGETARCH

# Build the go application from source code
RUN env GOOS=$TARGETOS GOARCH=$TARGETARCH go build -o /bin/app_runner /src/main.go

# Create application in standalone container
# Update with the image name from instructions

FROM ____

# Update with name of previous build stage
COPY --from=____ /bin/app_runner /bin/app_runner

CMD ["/bin/app_runner"]


# Update the FROM line to support a multi-platform build
FROM --platform=$BUILDPLATFORM golang:1.21-alpine3.19 AS gobuild

WORKDIR /src
COPY src/main.go /src/main.go

# Define the option to pull environment variables from the host
ARG TARGETOS TARGETARCH

# Build the go application from source code
RUN env GOOS=$TARGETOS GOARCH=$TARGETARCH go build -o /bin/app_runner /src/main.go

# Create application in standalone container
# Update with the image name from instructions

FROM scratch

# Update with name of previous build stage
COPY --from=gobuild /bin/app_runner /bin/app_runner

CMD ["/bin/app_runner"]


Update the Dockerfile as shown in the example.

Once complete, run the following command in the terminal:

- docker buildx build --platform linux/arm64 . -t multi-platform-app 

Verify the platform image built as expected by running the command:

./check_platform.sh

in the terminal window and verify that it returns "linux/arm64"


Multi-platform build

Building advanced container images

Combine the the volume, networking, and container handling capabilities of Docker into multi-container applications using Docker Compose. Update and manage application deployments via the docker-compose.yml file.

Introduction to Docker Compose

Managing an application with Docker Compose

Troubleshooting application startup

Creating compose.yaml files

Writing a compose.yaml file

Identifying sections

How many resources?

Dependencies and troubleshooting in Docker Compose

Dependency calculation

Shutdown order

Troubleshooting dependencies

Creating a data service within Docker

Building a data service backend

Adding a data client

Using Docker Compose to define a full application

Congratulations!

Docker Compose

This course builds upon the foundations of learning Docker and containerization found in the Introduction to Docker course. We extend the concepts and tools covered in that course, adding the ideas of container image management and optimization, networking, file system communication, multi-platform and multi-container applications. When completed, you'll be able to:

- Create multi-stage builds to optimize the size, security, and reusability of containerized applications.
- Use Docker networking tools and concepts to add inter-container communications and provide networking services to the local network and the outside world.
- Share files and data between containers and the host machine using Docker file system mounts and volumes.
- Create mutli-container application deployments using Docker Compose, drastically simplifying the setup of complex tools common in data science and engineering environments.

You'll gain the most from this course if you have worked with and built some simple containers, but are looking to add to your development, testing, and distribution toolbox. We look forward to helping guide you during your journey of learning Docker.

This course expands on the knowledge gained from the Introduction to Docker course, delving deeper into containerization concepts and tools. We'll explore advanced topics such as container image management, networking, file system communication, and the development of multi-platform and multi-container applications.
<h2>Optimizing Containerized Applications</h2>
Learn to create multi-stage builds to enhance the size, security, and reusability of containerized applications. Through practical exercises, master techniques for optimizing container images, ensuring efficient deployment and management in diverse environments.
<h2>Enhancing Container Deployments with Docker Compose</h2>
Discover how to leverage Docker networking tools and concepts to facilitate inter-container communication and provide networking services to both local and external networks. Explore file system mounts and volumes for seamless data sharing between containers and the host machine. Additionally, gain proficiency in Docker Compose for orchestrating multi-container application deployments, simplifying the setup of complex tools prevalent in data science and engineering environments.
<h2>Empowering Development and Deployment</h2>
This course equips you with essential skills to enhance your development, testing, and distribution processes. While not essential, you will gain the most from this course if you have worked with and built some simple containers. By the course's end, you'll have the knowledge and tools to optimize containerized applications, streamline communication between containers, and deploy multi-container applications with ease, empowering you on your Docker learning journey.

Introduction to Docker

Master multi-stage builds, Docker networking tools, and Docker Compose for optimal containerized applications!

Docker image exploration

Intermediate Docker

Uygulamalı interaktif egzersiz