I'm here to help anyone with Xdebug on a Windows system and PHP 7.4 installed. Can try to set the php.ini as shown below. Can see my xdebug extension is called php_xdebug.dll inside this folder C:\xampp\php\ext\

xdebug.remote_autostart = 0
xdebug.mode = debug
xdebug.start_with_request = yes
zend_extension = C:\xampp\php\ext\php_xdebug.dll
xdebug.stopOnEntry = true
xdebug.profiler_enable = off
xdebug.profiler_enable_trigger = Off
xdebug.profiler_output_name = cachegrind.out.%t.%p
xdebug.output_dir ="c:\xampp\tmp"
xdebug.show_local_vars=0
xdebug.remote_handler = "dbgp"
xdebug.client_host = "127.0.0.1"
xdebug.log = "C:\xampp\tmp\xdebug.txt"
xdebug.client_port = 9000
xdebug.remote_cookie_expire_time = 36000

*Extras for the visual studio code user. Make sure the port inside the configuration is the same as the one that you set.

{
    // Use IntelliSense to learn about possible attributes.
    // Hover to view descriptions of existing attributes.
    // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
    "version": "0.2.0",
    "configurations": [
        {
            "name": "Listen for Xdebug",
            "type": "php",
            "request": "launch",
            "port": 9000
        },
        {
            "name": "Launch currently open script",
            "type": "php",
            "request": "launch",
            "program": "${file}",
            "cwd": "${fileDirname}",
            "port": 0,
            "runtimeArgs": [
                "-dxdebug.start_with_request=yes"
            ],
            "env": {
                "XDEBUG_MODE": "debug,develop",
                "XDEBUG_CONFIG": "client_port=${port}"
            }
        },
        {
            "name": "Launch Built-in web server",
            "type": "php",
            "request": "launch",
            "runtimeArgs": [
                "-dxdebug.mode=debug",
                "-dxdebug.start_with_request=yes",
                "-S",
                "localhost:0"
            ],
            "program": "",
            "cwd": "${workspaceRoot}",
            "port": 9000,
            "serverReadyAction": {
                "pattern": "Development Server \\(http://localhost:([0-9]+)\\) started",
                "uriFormat": "http://localhost:%s",
                "action": "openExternally"
            }
        }
    ]
}

The XDEBUG_MODE environment defined in docker-compose definition is overriding the default value from Dockerfile.

Your way only works if you invoke docker-compose with the --env-file (> docker-compose --env-file=xdebug.env ...)

I guess what you want is to inherit a XDEBUG_MODE you are defining inline or is already present in the terminal/shell environment. In that case you just need to declare the var without any value

services:
  php:
    environment:
      XDEBUG_MODE

and call docker-compose similar to > XDEBUG_MODE=Off docker-compose