If you're using the example code from the book (below), at some point you should reach the "AAAAAAAA" pattern (0x41). Note that, since you're running it on a 64-bit machine that stores elements in the stack with 8 bytes each, you should run it with $ ./fmtstr "AAAAAAAA %016x %016x %016x %016x %016x %016x %016x %016x %016x %016x %016x %016x %016x" instead, or you will miss part of each element on the stack.

#include <stdlib.h>
int main(int argc, char *argv[]){
        static int canary=0;   // stores the canary value in .data section
        char temp[2048];       // string to hold large temp string
      strcpy(temp, argv[1]);   // take argv1 input and jam into temp
      printf(temp);            // print value of temp
      printf("\n");            // print carriage return
      printf("Canary at 0x%08x = 0x%08x\n", &canary, canary); //print canary
}

You should pay attention to the quote in the book that states:

The fact that the fourth item shown (from the stack) was our format string depends on the nature of the format function used and the location of the vulnerable call in the vulnerable program. To find this value, simply use brute force and keep increasing the number of %08x tokens until the beginning of the format string is found. For our simple example (fmtstr), the distance, called the offset, is defined as 4.

Remember that the parameter being parsed to printf isn't the string itself, but the address of the string. So it's position on the memory layout of the program in relation to the printf stack is what will define how further you'll have to search to find it.

what's the difference to x86 that is difficult for you?

Conversion specifiers - a, A, e, E, f, F, g, or G - for long double arguments can be used to get 64 bits at a time.

Basically, printf("%a") will get 64 bits from the stack and convert them to a Hexadecimal Floating-Point Constant.

More info about Hexadecimal Floating Point Constants - http://www.exploringbinary.com/hexadecimal-floating-point-constants/

I think that the paper provides its printf() examples in a somewhat confusing way because the examples use string literals for format strings, and those don't generally permit the type of vulnerability being described. The format string vulnerability as described here depends on the format string being provided by user input.

So the example:

printf ("\x10\x01\x48\x08_%08x.%08x.%08x.%08x.%08x|%s|");

Might better be presented as:

/* 
 * in a real program, some user input source would be copied 
 * into the `outstring` buffer 
 */
char outstring[80] = "\x10\x01\x48\x08_%08x.%08x.%08x.%08x.%08x|%s|";

printf(outstring);

Since the outstring array is an automatic, the compiler will likely put it on the stack. After copying the user input to the outstring array, it'll look like the following as 'words' on the stack (assuming little endian):

outstring[0c]               // etc...
outstring[08] 0x30252e78    // from "x.%0"
outstring[04] 0x3830255f    // from "_%08"
outstring[00] 0x08480110    // from the ""\x10\x01\x48\x08"

The compiler will put other items on the stack as it sees fit (other local variables, saved registers, whatever).

When the printf() call is about to be made, the stack might look like:

outstring[0c]               // etc...
outstring[08] 0x30252e78    // from "x.%0"
outstring[04] 0x3830255f    // from "_%08"
outstring[00] 0x08480110    // from the ""\x10\x01\x48\x08"
var1
var2
saved ECX
saved EDI

Note that I'm completely making those entries up - each compiler will use the stack in different ways (so a format string vulnerability has to be custom crafted for a particular exact scenario. In other words, you won't always use 5 dummy format specifiers like in this example - as the attacker you'd need to figure out how many dummies the particular vulnerability would need.

Now to call printf(), the argument (the address of outstring) is pushed on to the stack and printf() is called, so the argument area of the stack looks like:

outstring[0c]               // etc...
outstring[08] 0x30252e78    // from "x.%0"
outstring[04] 0x3830255f    // from "_%08"
outstring[00] 0x08480110    // from the ""\x10\x01\x48\x08"
var1
var2
var3
saved ECX
saved EDI
&outstring   // the one real argument to `printf()`

However, printf doesn't really know anything about how many arguments have been placed on the stack for it - it goes by the format specifiers it finds in the format string (the one argument it's 'sure' to get). So printf() gets the format string argument and starts processing it. When it gets to the 1st "%08x" that will correspond to the 'saved EDI' in my example, then next "%08x" will print the saved ECX' and so on. So the "%08x" format specifiers are just eating up data on the stack until it gets back to the string the attacker was able to input. Determining how many of those are needed is something an attacker would do by a kind of trial and error (probably by a test run that has a whole slew of "%08x" formats until he can 'see' where the format string starts).

Anyway, when printf() gets to processing the "%s" format specifier, it has consumed all the stack entries up to where the outstring buffer resides. The "%s" specifier treats its stack entry as a pointer, and the string that the user has put into that buffer has been carefully crafted to have a binary representation of 0x08480110, so printf() will print out whatever is at that address as an ASCIIZ string.