However, I can call yield() on my Nano or ESP8266 without including the Scheduler lib

The yield() function is also implemented inside the ESP8266 libraries:

Yielding

This is one of the most critical differences between the ESP8266 and a more classical Arduino microcontroller. The ESP8266 runs a lot of utility functions in the background – keeping WiFi connected, managing the TCP/IP stack, and performing other duties. Blocking these functions from running can cause the ESP8266 to crash and reset itself. To avoid these mysterious resets, avoid long, blocking loops in your sketch.

The amazing creators of the ESP8266 Arduino libraries also implemented a yield() function, which calls on the background functions to allow them to do their things.

That's why you can call yield() from within your main program where the ESP8266 header is included.

See ESP8266 Thing Hookup Guide.

Update:

yield() is defined in Arduino.h as:

void yield(void);

yield() is also declared in hooks.h as follows:

/**
 * Empty yield() hook.
 *
 * This function is intended to be used by library writers to build
 * libraries or sketches that supports cooperative threads.
 *
 * Its defined as a weak symbol and it can be redefined to implement a
 * real cooperative scheduler.
 */
static void __empty() {
    // Empty
}
void yield(void) __attribute__ ((weak, alias("__empty")));

So, on the Nano, it probably does nothing (unless you have other libraries #included).

Delay (traditionally) has two functions that it performs:

1. Wait for a period
2. Yield processing to other threads through the yield() weak symbol.

By using a delay(0) the author thinks they are saying "I don't want to delay here, but if anything is using the yield() function it can run now."

However, that is not correct.

The code for delay() in the AVR core looks like this:

void delay(unsigned long ms) 
{
    uint32_t start = micros();

    while (ms > 0) {
        yield();
        while ( ms > 0 && (micros() - start) >= 1000) {
            ms--;
            start += 1000;
        }
    }   
}

You can see that, as well as delaying for a period, yield() is regularly called. However that only ever happens if ms is greater than 0. If you pass 0 as the delay length then the delay() function essentially does nothing at all.

Now it could be that an earlier version of the core had yield() always being called even with ms==0, but the current version doesn't. Maybe this is a hangup from an old coding style. But for the current core if you want to yield you must explicitly call yield() or delay for at least one millisecond.

Here's how it's implemented in the Arduino AVR Boards core: https://github.com/arduino/Arduino/blob/1.8.3/hardware/arduino/avr/cores/arduino/hooks.c#L19-L31

void yield(void) __attribute__ ((weak, alias("__empty")));

As the comments in that file explain:

Empty yield() hook. This function is intended to be used by library writers to build libraries or sketches that supports cooperative threads. Its defined as a weak symbol and it can be redefined to implement a real cooperative scheduler.

So you can define yield() in your own code if you like. Otherwise the empty definition will prevent the undefined reference error you encountered.

Of course it's better to write proper non-blocking code rather than using this yield() workaround but you asked how the Arduino IDE handles it and so there you have it.