Any process executed by docker run must handle a signal itself to receive that signal.
Alternatively use the --init flag to run the tini init as PID 1 which will handle and forward signals for you.
The sh shell can also be injected as the PID 1 process in the container, depending on how the command (CMD) is specified.
Detail
A SIGTERM is propagated by the docker run command to the Docker daemon by default but it will not take effect unless the signal is specifically handled in main process being run by Docker.
The first process you run in a container will have PID 1 in that containers PID namespace. This process is treated as a special process by the linux kernel. It will not be sent a signal unless the process has a handler installed for that signal. It is also PID 1's job to forward signals onto other child processes.
docker run and other commands are API clients for the Docker Remote API hosted by the docker daemon. The docker daemon runs as a seperate process and is the parent for the commands you run inside a container context. This means that there is no direct sending of signals between docker run and the daemon, in the standard unix manner.
The docker run and docker attach commands have a --sig-proxy flag that defaults signal proxying to true. You can turn this off if you want.
docker exec does not proxy signals.
In a Dockerfile, be careful to use the "exec form" when specifying CMD and ENTRYPOINT defaults, if you don't want sh to become the PID 1 process (Kevin Burke):
CMD [ "/path/executable", "param1", "param2" ]
Signal Handling Go Example
Using the sample Go code here: https://gobyexample.com/signals
Run a regular process that doesn't handle signals in the background. Then the example Go daemon that traps signals also in the background. I'm using sleep for the process that doesn't install "daemon" signal handlers.
$ docker run busybox sleep 6000 &
$ docker run gosignal &
With a ps tool that has a "tree" view, you can see the two distinct process trees. One for the docker run process under sshd. The other for the actual container processes, under docker daemon.
$ pstree -p
init(1)-+-VBoxService(1287)
|-docker(1356)---docker-containe(1369)-+-docker-containe(1511)---gitlab-ci-multi(1520)
| |-docker-containe(4069)---sleep(4078)
| `-docker-containe(4638)---main(4649)
`-sshd(1307)---sshd(1565)---sshd(1567)---sh(1568)-+-docker(4060)
|-docker(4632)
`-pstree(4671)
The details of docker hosts processes:
$ ps -ef | grep "docker r\|sleep\|main"
docker 4060 1568 0 02:57 pts/0 00:00:00 docker run busybox sleep 6000
root 4078 4069 0 02:58 ? 00:00:00 sleep 6000
docker 4632 1568 0 03:10 pts/0 00:00:00 docker run gosignal
root 4649 4638 0 03:10 ? 00:00:00 /main
Killing
I can't kill the docker run busybox sleep command:
$ kill 4060
$ ps -ef | grep 4060
docker 4060 1568 0 02:57 pts/0 00:00:00 docker run busybox sleep 6000
I can kill the docker run gosignal command, that has launched the go process with the signal handlers:
$ kill 4632
$
terminated
exiting
[2]+ Done docker run gosignal
Signals via docker exec
If I docker exec a new sleep process in the already running sleep container, I can send an ctrl-c and interrupt the docker exec itself, but that doesn't forward to the actual process:
$ docker exec 30b6652cfc04 sleep 600
^C
$ docker exec 30b6652cfc04 ps -ef
PID USER TIME COMMAND
1 root 0:00 sleep 6000 <- original
97 root 0:00 sleep 600 <- execed still running
102 root 0:00 ps -ef
Answer from Matt on Stack OverflowAny process executed by docker run must handle a signal itself to receive that signal.
Alternatively use the --init flag to run the tini init as PID 1 which will handle and forward signals for you.
The sh shell can also be injected as the PID 1 process in the container, depending on how the command (CMD) is specified.
Detail
A SIGTERM is propagated by the docker run command to the Docker daemon by default but it will not take effect unless the signal is specifically handled in main process being run by Docker.
The first process you run in a container will have PID 1 in that containers PID namespace. This process is treated as a special process by the linux kernel. It will not be sent a signal unless the process has a handler installed for that signal. It is also PID 1's job to forward signals onto other child processes.
docker run and other commands are API clients for the Docker Remote API hosted by the docker daemon. The docker daemon runs as a seperate process and is the parent for the commands you run inside a container context. This means that there is no direct sending of signals between docker run and the daemon, in the standard unix manner.
The docker run and docker attach commands have a --sig-proxy flag that defaults signal proxying to true. You can turn this off if you want.
docker exec does not proxy signals.
In a Dockerfile, be careful to use the "exec form" when specifying CMD and ENTRYPOINT defaults, if you don't want sh to become the PID 1 process (Kevin Burke):
CMD [ "/path/executable", "param1", "param2" ]
Signal Handling Go Example
Using the sample Go code here: https://gobyexample.com/signals
Run a regular process that doesn't handle signals in the background. Then the example Go daemon that traps signals also in the background. I'm using sleep for the process that doesn't install "daemon" signal handlers.
$ docker run busybox sleep 6000 &
$ docker run gosignal &
With a ps tool that has a "tree" view, you can see the two distinct process trees. One for the docker run process under sshd. The other for the actual container processes, under docker daemon.
$ pstree -p
init(1)-+-VBoxService(1287)
|-docker(1356)---docker-containe(1369)-+-docker-containe(1511)---gitlab-ci-multi(1520)
| |-docker-containe(4069)---sleep(4078)
| `-docker-containe(4638)---main(4649)
`-sshd(1307)---sshd(1565)---sshd(1567)---sh(1568)-+-docker(4060)
|-docker(4632)
`-pstree(4671)
The details of docker hosts processes:
$ ps -ef | grep "docker r\|sleep\|main"
docker 4060 1568 0 02:57 pts/0 00:00:00 docker run busybox sleep 6000
root 4078 4069 0 02:58 ? 00:00:00 sleep 6000
docker 4632 1568 0 03:10 pts/0 00:00:00 docker run gosignal
root 4649 4638 0 03:10 ? 00:00:00 /main
Killing
I can't kill the docker run busybox sleep command:
$ kill 4060
$ ps -ef | grep 4060
docker 4060 1568 0 02:57 pts/0 00:00:00 docker run busybox sleep 6000
I can kill the docker run gosignal command, that has launched the go process with the signal handlers:
$ kill 4632
$
terminated
exiting
[2]+ Done docker run gosignal
Signals via docker exec
If I docker exec a new sleep process in the already running sleep container, I can send an ctrl-c and interrupt the docker exec itself, but that doesn't forward to the actual process:
$ docker exec 30b6652cfc04 sleep 600
^C
$ docker exec 30b6652cfc04 ps -ef
PID USER TIME COMMAND
1 root 0:00 sleep 6000 <- original
97 root 0:00 sleep 600 <- execed still running
102 root 0:00 ps -ef
So there are two factors at play here:
1) If you specify a string for an entrypoint, like this:
ENTRYPOINT /go/bin/myapp
Docker runs the script with /bin/sh -c 'command'. This intermediate script gets the SIGTERM, but doesn't send it to the running server app.
To avoid the intermediate layer, specify your entrypoint as an array of strings.
ENTRYPOINT ["/go/bin/myapp"]
2) I built the app I was trying to run with the following string:
docker build -t first-app .
This tagged the container with the name first-app. Unfortunately when I tried to rebuild/rerun the container I ran:
docker build .
Which didn't overwrite the tag, so my changes weren't being applied.
Once I did both of those things, I was able to kill the process with ctrl+c, and bring down the running container.
signals - Process not receiving SIGTERM in Docker container - Unix & Linux Stack Exchange
[BUG] Inconsistent handling of SIGTERM based on whether shell is interactive
bash - How to catch SIGTERM properly in Docker? - Stack Overflow
Received a SIGTERM signal for docker container
Greetings,
I have jellyfin server running as a docker container (rather a podman container) and I have been using it for months now but just this evening, I have tried to watch something off of it and every few minutes, the jellyfin service in the container will shutdown and restart.
The following is the only thing that I can see in the logs that appears to be related `Main: Received a SIGTERM signal, shutting down`. I am not sure how to diagnose this issue since that appears to be the only log entry in both docker and in the server that relates to this issue. Is there something I am missing?
The thing that perplexes me is that this issue just came out of nowhere. I tried updating the whole system as well as the container but the issue persists
Actually, your Dockerfile and start.sh entrypoint script work as is for me with Ctrl+C, provided you run the container with one of the following commands:
docker run --name tmp -it tmpdocker run --rm -it tmp
Documentation details
As specified in docker run --help:
- the
--interactive=-iCLI flag asks to keep STDIN open even if not attached
(typically useful for an interactive shell, or when also passing the--detach=-dCLI flag) - the
--tty=-tCLI flag asks to allocate a pseudo-TTY
(which notably forwards signals to the shell entrypoint, especially useful for your use case)
Related remarks
For completeness, note that there are several related issues that can make docker stop take too much time and "fall back" to docker kill, which can arise when the shell entrypoint starts some other process(es):
- First, when the last line of the shell entrypoint runs another, main program, don't forget to prepend this line with the
execbuiltin:
exec prog arg1 arg2 ... - But when the shell entrypoint is intended to run for a long time, trapping signals (at least
INT/TERM, but notKILL) is very important;
{see also this SO question: Docker Run Script to catch interruption signal} - Otherwise, if the signals are not forwarded to the children processes, we run the risk of hitting the "PID 1 zombie reaping problem", for instance
{see also this SO question for details: Speed up docker-compose shutdown}
The reason is that bash will not handle signals until the foreground process terminates, the sleep 10000 in your case. Your traps do work, but you would have to wait 10000 seconds first.
Your fix is
#!/bin/bash
pid=
trap 'echo SIGINT; [[ $pid ]] && kill $pid; exit' SIGINT
trap 'echo SIGTERM; [[ $pid ]] && kill $pid; exit' SIGTERM
echo Starting script
sleep 10000 & pid=$!
wait
pid=
Also you probably want to replace trapping specific signals with trapping any exit
trap 'echo EXIT; [[ $pid ]] && kill $pid; exit' EXIT
These and more are very well explained at http://mywiki.wooledge.org/SignalTrap#When_is_the_signal_handled.3F