I recommend getting a cross-compiler binary. On a Linux distribution it is often already packaged.

However, if you insist on building GCC from source (but beware of dependencies), read about Installing GCC and about Configuring GCC. You should build it outside of the source tree, and you want to pass some particular --target= option. I also recommend something like --program-suffix=_my

^{(I have no idea what --target is relevant for your particular configuration & board; you need to find out)}

Notice that a target is not only a CPU, but also an OS....

By default, newer GCC versions do not create correct binaries for ARMv6. Even though you pass the correct -mcpu= flag to gcc, it will create startup code for the newer ARMv7 architecture. Running them on your RasPI Zero will cause an "Illegal Instruction" exception.

this info is mentioned here https://github.com/Pro/raspi-toolchain

64-bit ARM

For 64-bit ARM, the toolchain prefix is aarch64 and usage is:

sudo apt install gcc-aarch64-linux-gnu
aarch64-linux-gnu-gcc -o main.out main.c

You can try it out on this C hello world with QEMU:

main.c

#include <stdio.h>

int main(void) {
    puts("hello");
}

and then:

sudo apt install qemu-user
qemu-aarch64 main.out

will output:

hello

Then a few fun things you can do to quickly see that ARM is actually running under the hood:

• GDB step debug it: https://stackoverflow.com/questions/20590155/how-to-single-step-arm-assembly-in-gdb-on-qemu/51310791#51310791
• log the executed ARM instructions with: qemu-aarch64 -d in_asm,out_asm main.out https://stackoverflow.com/questions/13005303/how-does-native-android-code-written-for-arm-run-on-x86/44505097#44505097

Tested in Ubuntu 19.10.

For reliability in serious applications, the disk image provider must also provide a compatible cross compiler

Although you can install a cross compiler with apt conveniently, I must warn you that this is not necessarily reliable unless explicitly supported by the image provider.

If you pick the cross compiler wrongly, the following may happen:

• the dynamic linker is at the wrong path: https://stackoverflow.com/questions/31929092/trying-to-run-a-cross-compiled-executable-on-target-device-fails-with-no-such-f/49993116#49993116
• binary incompatibility with the glibc and any other libraries you link against: https://stackoverflow.com/questions/11107263/how-compatible-are-different-versions-of-glibc

Raspberry PI cross compilation

For RPI in particular, the provided cross compilers are available at: https://github.com/raspberrypi/tools and can be used as explained at: https://raspberrypi.stackexchange.com/questions/64273/installing-raspberry-pi-cross-compiler/83215#83215

git clone https://github.com/raspberrypi/tools
export PATH="$(pwd)/tools/arm-bcm2708/gcc-linaro-arm-linux-gnueabihf-raspbian-x64/bin:${PATH}"
printf '#include <stdio.h>\nint main() { puts("hello world"); }\n' > hello_world.c
printf '#include <iostream>\nint main() { std::cout << "hello world" << std::endl; }\n' > hello_world.cpp
arm-linux-gnueabihf-gcc -std=c99 -o hello_world_c hello_world.c
arm-linux-gnueabihf-g++ -std=c++11 -o hello_world_cpp hello_world.cpp

Ubuntu cross compilation

If you want to cross compile for Ubuntu arm64, I have never been able to find a clear reference on which cross compilers support which distro version: What are the officially supported cross compilers for Ubuntu server alternative architectures like ARM?

Buildroot

My favorite alternative is to build your own image with Buildroot: https://stackoverflow.com/questions/47557262/how-to-download-the-torvalds-linux-kernel-master-recompile-it-and-boot-it-wi/49349237#49349237

This builds everything from source, including the toolchain and the image, and ensures that everything is compatible.