Think easily; when you want to debug some program, you probably compile it with -g or -ggdb and run gdb, don't you?

1. Download gdb source.

2. Compile it with -ggdb

   ./configure --prefix=<where-to-install>
   make CFLAGS="-ggdb" CXXFLAGS="-ggdb"
   make install
   

3. Debug it!

   gdb <where-to-install>/bin/gdb
   

I've never tried it (and never thought it), but it may work. (And it looks very interesting; I'm about to try it!)

Um, I've just tested it in cygwin, and figure out the problem that the debugger gdb's output and the debuggee gdb's output are mixed; I solved it by using gdbserver to debug.

# On terminal 1..
$ gdbserver localhost:1234 gdb-gdb/prefix/bin/gdb
Process gdb-gdb/prefix/bin/gdb created; pid = 972
Listening on port 1234
Remote debugging from host 127.0.0.1
GNU gdb (GDB) 7.7.1
Copyright (C) 2014 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "i686-pc-mingw32".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word".
(gdb) q

Child exited with status 0
GDBserver exiting

and

# On terminal 2..
$ gdb gdb-gdb/prefix/bin/gdb
GNU gdb (GDB) 7.8
Copyright (C) 2014 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "i686-pc-cygwin".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from gdb-gdb/prefix/bin/gdb...done.
(gdb) target remote localhost:1234
Remote debugging using localhost:1234
0x7c93120f in ntdll!DbgBreakPoint ()
   from /cygdrive/c/WINDOWS/system32/ntdll.dll
(gdb) c
Continuing.
[Inferior 1 (Remote target) exited normally]
(gdb)

help running provides some hints:

There are step and next instuctions (and also nexti and stepi).

(gdb) help next
Step program, proceeding through subroutine calls.
Usage: next [N]
Unlike "step", if the current source line calls a subroutine,
this command does not enter the subroutine, but instead steps over
the call, in effect treating it as a single source line.

So we can see that step steps into subroutines, but next will step over subroutines.

The step and stepi (and the next and nexti) are distinguishing by "line" or "instruction" increments.

step -- Step program until it reaches a different source line
stepi -- Step one instruction exactly

Related is finish:

(gdb) help finish
Execute until selected stack frame returns.
Usage: finish
Upon return, the value returned is printed and put in the value history.

A lot more useful information is at https://sourceware.org/gdb/onlinedocs/gdb/Continuing-and-Stepping.html