Our approach is to have a wheelbuilder container with all the building dependencies etc in that image, once all wheels are built into a shared volume can be "consumed" by other containers just pointing pip to that folder. It works pretty well for us.

Install all wheels without listing them explicitly:

pip install /path/to/*.whl

OK, after spending hours, I found the problem and fixed it. So I'll answer my own question.

First of all, I was using --no-deps parameter in the pip wheel command. This parameter causes that the pip wheel only downloads the main packages, without their dependencies. So in the second build image (stage), pip was trying to download all the sub dependencies for these main dependencies. So these dependencies were in need for some system packages to build.

Removing this --no-deps parameter would normally fix the problem in such situation, but it caused another error for my situation. There were some version conflicts between the dependencies, which were not exist with regular pip install -r ...

Instead of the pip wheel technique, I just used pip install --user -r option to install my requirements in the folder /root/.local. Then I moved the /root/.local folder to the second stage's /root:

..
COPY --from=builder /root/.local /home/.local
..

That's all. All my requirements were installed in the second build image too.

Do not forget to add the /root/.local/bin path to your $PATH:

ENV PATH=/root/.local/bin:$PATH