It's just like a "hello world" program but from the XSS world. It's easy to check, minimalistic, checking that you can at least execute some javascript function (alert). While you're looking for an XSS, the payload itself is not as important as the "can I actually inject some javascript here?" question.

Basically, it's a 2 steps approach.

1. Find a vulnerable parameter. (using alert or any other simple function)
2. Now let's have some fun with it.

If I am not mistaken XSS means that the attacker should be able to establish two-way communication between infected client and attacker's server.

Not always. Sometimes, 1-way communication is enough:

1. Just send data to your server, no response required. It's very useful for the stored XSS case (when let's say you can put random javascript code into a comment visible to other users)
2. You can inject some HTML asking the user to open another website and do whatever you want. (XSS + social engineering)

To summarise: alert is a simple function sufficient to check if you can inject javascript, like "hello world" to check that your setup is working. If you're successful -> it's time to make it more complicated.

Edit: in a real attack, people usually check more options, because the "alert" keyword is blocked by most security filters. It doesn't mean that the XSS is not there ;) But "alert" is a very convenient example for tutorials, so you'll see it everywhere.