I observed 3 different reasons for problems like this:

jq not installed

bin/build-storefront.sh and bin/build-administration.sh should install this for you. but only if commandline tool jq is installed and file var/plugins.json is correct. here is the part of this this script that installs the npm dependencies if jq is installed:

if [[ $(command -v jq) ]]; then
    OLDPWD=$(pwd)
    cd "$PROJECT_ROOT" || exit

    jq -c '.[]' "var/plugins.json" | while read -r config; do
        srcPath=$(echo "$config" | jq -r '(.basePath + .storefront.path)')

        # the package.json files are always one upper
        path=$(dirname "$srcPath")
        name=$(echo "$config" | jq -r '.technicalName' )

        if [[ -f "$path/package.json" && ! -f "$path/node_modules" && $name != "storefront" ]]; then
            echo "=> Installing npm dependencies for ${name}"

            if [[ -f "$path/package-lock.json" ]]; then
                npm clean-install --prefix "$path"
            else
                npm install --prefix "$path"
            fi
        fi
    done
    cd "$OLDPWD" || exit
else
    echo "Cannot check extensions for required npm installations as jq is not installed"
fi

SHOPWARE_SKIP_BUNDLE_DUMP prevents plugins.json update

if jq is installed than maybe your plugins.json is not up to date. if env variable SHOPWARE_SKIP_BUNDLE_DUMP is defined, this will prevent this file from being updated.

wrong npm version

From Version 5 to 6 swag/paypal will not build with too old npm version installed. I had npm@6 installed and got errors. After switching to npm@9 everything builds well.

Especially because you wrote:

it is working within our CD

It may be this kind of wrong version.