OK, after spending hours, I found the problem and fixed it. So I'll answer my own question.
First of all, I was using --no-deps parameter in the pip wheel command. This parameter causes that the pip wheel only downloads the main packages, without their dependencies. So in the second build image (stage), pip was trying to download all the sub dependencies for these main dependencies. So these dependencies were in need for some system packages to build.
Removing this --no-deps parameter would normally fix the problem in such situation, but it caused another error for my situation. There were some version conflicts between the dependencies, which were not exist with regular pip install -r ...
Instead of the pip wheel technique, I just used pip install --user -r option to install my requirements in the folder /root/.local. Then I moved the /root/.local folder to the second stage's /root:
..
COPY --from=builder /root/.local /home/.local
..
That's all. All my requirements were installed in the second build image too.
Do not forget to add the /root/.local/bin path to your $PATH:
ENV PATH=/root/.local/bin:$PATH
Answer from kozhioyrin on Stack OverflowI'm tired of making bloated docker containers that include packages like python-pip and build-essential. I have a project with modules like twisted which compile things during install. What's the sane way for me to first build these modules, then copy them into the docker container? Should I be using wheels? Should I be fully installing modules then copying their paths in? Thanks for your time everyone
Edit: asking more questions :D
Our approach is to have a wheelbuilder container with all the building dependencies etc in that image, once all wheels are built into a shared volume can be "consumed" by other containers just pointing pip to that folder. It works pretty well for us.
We package our wheels in a different build process and upload them to a private repository. I created https://github.com/sherzberg/docker-curdserver just for this use case. Curdling isn't perfect, but it is a really nice wheel repository (as well as non-wheel packages).
We still need pip in the docker container, but the wheels really speeds up the docker build and we also don't need build-essentials.
» pip install docker