Earlier this week, I modified into writing a Dockerfile to download and bustle a binary after I spotted the image dimension modified into manner extra
than what I would seek files from. I’m the utilization of ubuntu: 21.10 because the sinful image, which is set 70MB. The binary I’m working
is set 80MB. Various packages I’m inserting in would add 10-ish MB. But the image dimension is 267MB?
Clearly I’m doing something unsuitable. But what? My Dockerfile is pretty simple and, what I even handed, idiomatic:
FROM ubuntu: 21.10 AS downloader # Install wget, gnupg; download a zip archive; check checksum; unzip the binary FROM ubuntu: 21.10 LABEL ... COPY --from=downloader /bin/
/bin/ RUN ethical-acquire update && ethical-acquire install -y openssl tiring-init iproute2 ca-certificates && rm -rf /var/lib/ethical/lists/ && chmod +x /bin/ && mkdir -p && ... ...
I checked the history of the image to review the scale of particular person layers. The anguish became very apparent…
$ podman history vamc19/nomad:newest ID CREATED CREATED BY SIZE COMMENT ...
36 minutes ago /bin/sh -c ethical-acquire update && ethical-acquire insta... 94.4 MB374515aec770 36 minutes ago /bin/sh -c # (nop) COPY file:6dbfa42743cc65... 87.7 MB 22cd380ad224 36 minutes ago /bin/sh -c # (nop) LABEL maintainer="Vamsi"... 0 B FROM docker.io/library/ubuntu: 21.10 ...
The layer created by
COPY is 87.7MB, which is precisely the scale of the extracted binary. So, that is regular. Why is the
layer created by
RUN 94.4MB? What am I doing in it? I’m making a few empty directories, working
chmod on the
binary and inserting in 4 packages. Upright says the packages would greatest bask in ~6MB of extra disk home and it is fully
unlikely that these packages would build something loopy post install. So, is
chmod creating an anguish?
To rapidly test this, I removed
RUN and rebuilt the image. And bingo – the image dimension is down to 174MB.
RUN layer’s dimension is down to 6.7MB. So, OverlayFS is copying the binary into
RUN layer although
greatest updating the metadata of the file…?
My figuring out of CoW filesystems is extremely superficial – unless I write to a file, the filesystem would never
copy the file to upper layer. And since chmod is now not any longer writing to the binary (did file’s hash alternate?), it’ll nonetheless no longer be
copied, superior? Clearly no longer. The truth is, I never even handed it. I regarded Up OverlayFS’ documentation.
When a file in the lower filesystem is accessed in a capability the requires write-acquire admission to, akin to opening for write acquire admission to,
changing some metadata etc., the file is first copied from the lower filesystem to the upper filesystem (copy_up).
Properly, I even comprise been doing it unsuitable all these years. I’ve written loads of Dockerfiles with shell scripts in
RUN. Presumably I never realized this because these files are in most cases very miniature to invent a noticable distinction
in the image dimension.
So what’s the resolution? In my case, since I’m the utilization of Podman (which makes insist of Buildah), I’m able to insist
--chmod arg with
copy a file and self-discipline factual permissions in the same layer. While you are the utilization of Docker, it is availabe in
Repeat that any metadata update will result in the same result – no longer factual
chmod. Each and every Docker and Podman already crimson meat Up
--chown for each
ADD. Presumably this ought to nonetheless be added to the Dockerfile Entirely Practices page.
PS: While you are wondering why a metadata update would invent OverlayFS reproduction your complete file, it is for safety
causes. Chances are high you’ll well enable “metadata greatest copy up” operate that will greatest copy the metadata instead
of your complete file.
Function no longer insist metacopy=on with untrusted upper/lower directories. In every other case it is that possibilities are you’ll well also imagine that an attacker can invent a
handcrafted file with appropriate REDIRECT and METACOPY xattrs, and attain acquire admission to to file on lower pointed by REDIRECT.
This ought to nonetheless no longer be that possibilities are you’ll well also imagine on native machine as surroundings “trusted.” xattrs will require CAP_SYS_ADMIN. But it’ll nonetheless be
that possibilities are you’ll well also imagine for untrusted layers fancy from a pen drive.