In general, running binaries that were compiled for an older glibc version (e.g. 2.13) will run fine on a system with a newer glibc (e.g. 2.14, like your system).

Running a binary that was built for a newer glibc (e.g. 2.15, like the one that fails) on a system with an older glibc will probably not work.

In short, glibc is backward-compatible, not forward-compatible.

Distributions ship with older/newer glibc versions depending in the stability wanted.

From all reading it seems that glibc cannot be manually updated on a system; the only way to version up with it is to jump to a complete newer version of a Linux operating system. Is this true?

No. glibc is updated every time your package manager finds a new version of glibc. The main file is /lib/libc.so.6. Which is the dynamic library programs are linked to (so they can run C functions included in the C standard library).

Although you can update glibc manually by compiling a desired version of glib and copying the resulting libc.so.6 to lib, this isn't recommended because the binaries could complain at the moment of execution, and because newer versions of glibc haven't been tested by the distribution's package maintainers. So you can expect an unstable system.

In general, binaries that were compiled for an older glibc version will run fine on a system with a newer glibc, as glibc is backward-compatible and handles automatically changes to its application binary interface (ABI). It achieves this wizardry by using symbol versioning, where basically to each symbol is attached a tag specifying its glibc version.

In case of semantics changes to function calls, glibc will include two versions, one for the old semantics and another for the new semantics, so each function is tagged with its version. The linker will consider both versions as two distinct functions.

This sophisticated mechanism is required since glibc is not one file but consists of many pieces (more than 200 shared libraries).

The backward-compatibility of glibc versions is under constant tracking. You may consult the ABI Laboratory report for API/ABI changes review for glibc. The report is generated by the abi-compliance-checker and abi-tracker tools.

For your question:

So if I'm overwriting with a newer libc.so.6 with additional glibc ABI versions inside it, how does it break older apps or leads system to breakage? Or doesn't it...?

Glibc compatibility is not fool-proof, but I believe that you will have to go way back to products compiled on quite old Linux versions to break it. I would also say that products may break not only because of glibc when run on versions of Linux different than where they were compiled.

So the best answer I can give is :
"It's not supposed to break anything, and there is an excellent chance that it won't".

For more information, see:

• The GNU C Library With Versioned Interface
• ABI compliance checker Notes
• ABI Policy and Guidelines

If libc.so.6 from 21.04 is not compatible with libc.so.6 from 21.10, then why isn't the libc on 21.10 called libc.so.7 instead?

libc.so is as core a library as they come. Nearly everything depends on it. One of glibc's goals is to provide backwards compatibility - a program that could run with an older libc.so.6 should (usually) also work fine with a newer release. However, if you bump the soname to libc.so.7 only because you added some new function, then all of these previously-built programs will need a rebuild for no good reason. There hasn't been a truly major break in API for glibc to warrant this yet.

I don't understand why a binary built on 21.10 is not compatible with an 21.04 system.

I don't see anybody guaranteeing forwards compatibility (which was what you expect by 21.04 being able to run something from 21.10) - why would you expect that if you don't take precautions to ensure it?

For some versions of glibc, the announcement email says which kernel versions are compatible.

glibc    Released      Requires Linux kernel version
----------------------------------------------------------

2.34     1 Aug 2021    ?   (same as 2.26?)
2.33     1 Feb 2021    ?   (same as 2.26?)
2.32     5 Aug 2020    ?   (same as 2.26?)
2.31     1 Feb 2020    ?   (same as 2.26?)
2.30     1 Aug 2019    ?   (same as 2.26?)
2.29     1 Feb 2019    ?   (same as 2.26?)
2.28     1 Aug 2018    ?   (same as 2.26?)
2.27     2 Feb 2018    ?   (same as 2.26?)
2.26     2 Aug 2017    3.2 (or higher)
2.25     5 Feb 2017    ?   (same as 2.24?)

2.24     4 Aug 2016    on i[4567]86 and x86_64  ->  2.6.32
2.24     4 Aug 2016    on other platforms       ->  3.2

Note 1:  Some minor architectures require a higher kernel version than
         listed above.

Note 2:  Your Linux distribution may vary from the above, as Glibc can
         optionally be configured at compile time to require a kernel
         version newer than the oldest supported version.

It may be possible to extend the above table by tracking down and reading more of the announcement emails for older (or newer) versions of glibc.

For reference:

Linux distribution    glibc version    Requires Linux kernel version

Debian 11 Bullseye    2.31             3.2 ?
Debian 10 Buster      2.28             3.2 ?
Debian 9 Stretch      2.24 on x86      2.6.32
Debian 9 Stretch      2.24 on other    3.2

Ubuntu 20.04          2.31             3.2
Ubuntu 18.04          2.27             3.2
Ubuntu 17.10          2.26             3.2
Ubuntu 17.04          2.24             ? 
Ubuntu 16.10          2.24             ?
Ubuntu 16.04          2.23             ?

Sources:
The GNU C Library Release Timeline
The GNU C Library (glibc)
The GNU C Library version 2.27 is now available
The GNU C Library version 2.26 is now available
The GNU C Library version 2.25 is now available
The GNU C Library version 2.24 is now available
DistroWatch.com: Ubuntu
Ubuntu glibc MIN_KERNEL_SUPPORTED

1. You can not easily (for whatever definition of the word) use newer kernel features with older versions of glibc. If you really need to, you can invoke system calls directly (using the syscall() library function) and dig whatever constant values and datastructures necessary from user-space kernel headers (the stuff which in the newer kernel is held under include/uapi). On the other hand, kernel developers usually promise not to break legacy features in newer kernels, so older glibc versions keep working as expected (well, almost).

2. Older programs still work with newer versions of glibc because glibc supports versioning of symbols (see here for some details: https://www.kernel.org/pub/software/libs/glibc/hjl/compat/). If your program is dynamically linked with newer version of glibc without special provisions (as described in the link above) you would not be able to run it with an older version of glibc libraries (dynamic linker will complain about unresolved symbols, as proper symbol versions will not be available).