Docker Pipeline Taking Too Much Time to Finish?

I used to build docker image step by step, that’s before I found out about docker’s BuildKit. With the BuildKit, the building time was reduced by 60% (in my case), I mean that’s crazy.

How Does the Concurrent Building Process Work?

Just split your Dockerfile into stages, the docker BuildKit will analyse all steps’ dependency of each other, and run multiple stages parallel. If a step does not depend on any other step, it will go straight forward. If a step depends on any other step, it will wait for its prerequisite to finish.

Here’s a piece of Dockerfile from my project:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
FROM shinchven/node:16-build

WORKDIR /root/build/web
COPY web /root/build/web
RUN npm i
RUN npm run build

FROM shinchven/node:16-build

WORKDIR /root/build/app
COPY app /root/build/app
RUN npm i -g shx
RUN npm install
RUN npm run compile

FROM shinchven/node:16-deployment

WORKDIR /usr/src/app

COPY --from=1 /root/build/app /usr/src/app
COPY --from=0 /root/build/web/dist /usr/src/app/public/web

ENV NODE_ENV=production

EXPOSE 3030

CMD ["pm2-docker","ecosystem.config.js"]

All stages started at the same time, while step 3 of stage-2 waited for stage-0 and stage-1 to finish, because the --from=1 and --from=0 flag of COPY command tells the step to wait for the stage-0 and stage-1 to finish. Thus, I saved a lot of time.

Use BuildKit

There are a few ways to use BuildKit:

1
2
3
4
5
# Enable BuildKit for docker and docker-compose
DOCKER_BUILDKIT=1 docker build .
DOCKER_BUILDKIT=1 docker-compose build
# Use [buildx](https://docs.docker.com/buildx/working-with-buildx/) command
docker buildx build .

Please see official documentation for more details.