I don’t find it satisfying: I can copy files from one stage to another, but I can’t install a .deb (or any other package for that matter).
Copying a list of paths will break if the package changes, and it can’t easily do anything done by pre-install and post-install scripts (like changing a file, registering a GPG key, etc.)
The official excuse from Docker is that “multi-stage builds cover that use”, but that’s not true if you need to install local packages. I think the buildkit feature that lets you mount folder in the build container is a hidden admission that this is wrong, but it’s annoying that the build process will then only work on machines where a non-standard option is set.
I think I understand your use case, although I don't follow the part about mounting a host dir in the build (how it would help).
I do not really consider this limitation a design flaw of Dockerfiles; this is more of a mismatch between Dockerfile functionality and what you want to do. Dockerfiles are not designed to execute general-purpose programs, so there is an assumption in the design of Dockerfiles that you know (more or less) which files you want to COPY ahead of time. It sounds like you want a mechanism for COPYing all filesystem changes resulting from a RUN instruction without having to know what those changes are ahead of time.
In the future, maybe a buildkit HLB language will provide a macro system that makes this sort of thing straightforward to implement.
> this is more of a mismatch between Dockerfile functionality and what you want to do. Dockerfiles are not designed to execute general-purpose programs, so there is an assumption in the design of Dockerfiles that you know (more or less) which files you want to COPY ahead of time.
I don't follow that. In a Dockerfile, I want to install software, and software is usually distributed in packages like .deb. Why can I `RUN apt-get install libabc` and have a clean, automatic install of libabc, but not do the same with a libabc.deb file on my host filesystem?
> It sounds like you want a mechanism for COPYing all filesystem changes resulting from a RUN instruction without having to know what those changes are ahead of time.
That's one way of doing it, but it's more complex than the obvious solution:
> I don't follow the part about mounting a host dir in the build (how it would help)
It simply lets me give the build container access to libabc.deb, so I can `RUN apt-get install /mnt/host_folder/libabc.deb`, which will install libabc without putting `libabc.deb` itself in an image layer.
It would have been trivial to add this, but the Docker devs rejected it because "multi-stage builds can do this" (wrong), and it would lead to "non-reproducible builds" (because downloading packages from the network is somehow more reproducible?).
They finally added it as a non-standard option, which means a build process that relies on it won't work with a Docker install that uses the default config. It's the devs making it needlessly hard to use Docker for a perfectly valid use case.
If you're wondering why I want to install that locally-built software via a .deb, it's because we want to distribute it to all our systems, and we want to have only one packaging method to maintain.
Copying a list of paths will break if the package changes, and it can’t easily do anything done by pre-install and post-install scripts (like changing a file, registering a GPG key, etc.)
The official excuse from Docker is that “multi-stage builds cover that use”, but that’s not true if you need to install local packages. I think the buildkit feature that lets you mount folder in the build container is a hidden admission that this is wrong, but it’s annoying that the build process will then only work on machines where a non-standard option is set.