The reason is because hexdump by default prints out 16-bit integers, not bytes. If your system has them, hd (or hexdump -C) or xxd will provide less surprising outputs - if not, od -t x1 is a POSIX-standard way to get byte-by-byte hex output. You can use od -t x1c to show both the byte hex values and the corresponding letters.
If you have xxd (which ships with vim), you can use xxd -r to convert back from hex (from the same format xxd produces). If you just have plain hex (just the '4161', which is produced by xxd -p) you can use xxd -r -p to convert back.
The reason is because hexdump by default prints out 16-bit integers, not bytes. If your system has them, hd (or hexdump -C) or xxd will provide less surprising outputs - if not, od -t x1 is a POSIX-standard way to get byte-by-byte hex output. You can use od -t x1c to show both the byte hex values and the corresponding letters.
If you have xxd (which ships with vim), you can use xxd -r to convert back from hex (from the same format xxd produces). If you just have plain hex (just the '4161', which is produced by xxd -p) you can use xxd -r -p to convert back.
For the first part, try
echo Aa | od -t x1
It prints byte-by-byte
$ echo Aa | od -t x1
0000000 41 61 0a
0000003
The 0a is the implicit newline that echo produces.
Use echo -n or printf instead.
$ printf Aa | od -t x1
0000000 41 61
0000002
There's a printf tool that simulates the C function; normally it's at /usr/bin/printf, but a lot of shells implement built-ins for it as well. You can use %02x to get the hex representation of a character, but you need to make sure you pass a string that includes the character in single-quotes (Edit: It turns out just a single-quote at the beginning is sufficient):
printf "%%%02x\n" "'-" # Outputs %2d
You can make a shell function for convenience:
function hex() {
printf "%%%02x\n" "'$1"
}
hex - # Outputs %2d
hex _ # Outputs %5f
hex . # Outputs %2e
xxd is a hex-dump utility shipped as part of the vim-common package
It has a handy -p option which outputs a continuous 'plain/postscript' hexdump style.. This can easily be reversed via -r... -u will output upper-case Hex-Digits.
$ echo -n "M" |xxd -p # 1 ASCII char= 1 UTF-8 byte; Unicode Codepoint: U+0041
4d
$ echo -n "ใถ" |xxd -p -u # 1 CJK char= 3 UTF-8 bytes; Unicode Codepoint: U+3036
E380B6
$ echo -n "Dump a string" |xxd -p -u
44756D70206120737472696E67
$ echo -n "Dump and Revert" |xxd -p |xxd -r
Dump and Revert
bash ascii to hex - Stack Overflow
Conversion hex string into ascii in bash command line - Stack Overflow
bash - How to convert hex to ASCII characters in the Linux shell? - Stack Overflow
Convert column from hex to ascii
$ str="hello"
$ hex="$(printf '%s' "$str" | xxd -p -u)"
$ echo "$hex"
68656C6C6F
Or:
$ hex="$(printf '%s' "$str" | hexdump -ve '/1 "%02X"')"
$ echo "$hex"
68656C6C6F
Careful with the '"%X"'; it has both single quotes and double quotes.
You have several options
$ printf hello | xxd
0000000: 6865 6c6c 6f hello
See also: Ascii/Hex convert in bash
This worked for me.
$ echo -n 54657374696e672031203220330 | xxd -r -p
Testing 1 2 3$
echo -n prevents a newline from being added to the end
-r tells it to convert hex to ascii as opposed to its normal mode of doing the opposite
-p tells it to use a plain format.
This code will convert the text 0xA7.0x9B.0x46.0x8D.0x1E.0x52.0xA7.0x9B.0x7B.0x31.0xD2 into a stream of 11 bytes with equivalent values. These bytes will be written to standard out.
TESTDATA=$(echo '0xA7.0x9B.0x46.0x8D.0x1E.0x52.0xA7.0x9B.0x7B.0x31.0xD2' | tr '.' ' ')
for c in $TESTDATA; do
echo $c | xxd -r
done
As others have pointed out, this will not result in a printable ASCII string for the simple reason that the specified bytes are not ASCII. You need post more information about how you obtained this string for us to help you with that.
How it works: xxd -r translates hexadecimal data to binary (like a reverse hexdump). xxd requires that each line start off with the index number of the first character on the line (run hexdump on something and see how each line starts off with an index number). In our case we want that number to always be zero, since each execution only has one line. As luck would have it, our data already has zeros before every character as part of the 0x notation. The lower case x is ignored by xxd, so all we have to do is pipe each 0xhh character to xxd and let it do the work.
The tr translates periods to spaces so that for will split it up correctly.
I used to do this with xxd:
echo -n 5a | xxd -r -p
But then I realised that in Debian/Ubuntu, xxd is part of vim-common and hence might not be present in a minimal system. To also avoid Perl (IMHO also not part of a minimal system), I ended up using sed, xargs, and printf like this:
echo -n 5a | sed 's/\([0-9A-F]\{2\}\)/\\\\\\x\1/gI' | xargs printf
Mostly, I only want to convert a few bytes and it's okay for such tasks. The advantage of this solution over the one of ghostdog74 is, that this can convert hex strings of arbitrary lengths automatically. xargs is used because printf doesnt read from standard input.
echo -n 5a | perl -pe 's/([0-9a-f]{2})/chr hex $1/gie'
Note that this won't skip non-hex characters. If you want just the hex (no whitespace from the original string etc):
echo 5a | perl -ne 's/([0-9a-f]{2})/print chr hex $1/gie'
Also, zsh and bash support this natively in echo:
echo -e '\x5a'
In printf in the bash manual, you'll find:
Arguments to non-string format specifiers are treated as C language constants, except that a leading plus or minus sign is allowed, and if the leading character is a single or double quote, the value is the ASCII value of the following character.
(my emphasis)
These shell functions encapsulate char_to_hex and hex_to_char. Function names stolen from perl
# ord: the ascii value of a character
# $ ord "A" #=> 65
#
ord() {
printf "%d" "\"$1"
}
# chr: the character represented by the given ASCII decimal value
# $ chr 65 #=> A
#
chr() {
printf "\x$(printf "%x" "$1")"
}
Then:
printf '%02X\n' "$(ord A)"
41
Though short, @Juergen comments that this answers his question. Use the following:
printf "%X\n" \"A\"
Note, this provides the single-byte ("ASCII", UTF-8) charcter code. See the Unix & Linux StackExchange for more on char -> hex value and the converse, hex -> char, using bash shell and/or python.
Hi! I need help with something, I have a tab separated file with one column (number 6) with values in hex, I need to convert them to ascii text.
I can modify the column using awk but I can't pass that value to another command to do the convertion. Is it possible to do something like this?
Thanks!
This works in Bash, Dash (sh), ksh, zsh and ash and uses only builtins:
Edit:
Here is a version of ord that outputs in hex and chr that accepts hex input:
ordhex ()
{
printf '%x' "'$1"
}
chrhex ()
{
printf \\x"$1"
}
The original decimal versions:
ord ()
{
echo -n $(( ( 256 + $(printf '%d' "'$1"))%256 ))
}
Examples (with added newlines):
$ ord ' '
32
$ ord _
95
$ ord A
65
$ ord '*'
42
$ ord \~
126
Here is the corresponding chr:
chr ()
{
printf \\$(($1/64*100+$1%64/8*10+$1%8))
}
Examples:
$ chr 125
}
$ chr 42
*
$ chr 0 | xxd
0000000: 00 .
$ chr 255 | xxd
0000000: ff .
perl -e 'print ord("_"), "\n"'
Hex:
printf '\x4a'
Dec:
printf "\\$(printf %o 74)"
Alternative for hex :-)
xxd -r <<<'0 4a'
In general, the shell could understand hex, oct and decimal numbers in variables, provided they have been defined as integers:
$ declare -i v1 v2 v3 v4 v5 v6 v7
$ v1=0112
$ v2=74
$ v3=0x4a
$ v4=8#112
$ v5=10#74
$ v6=16#4a
$ v7=18#gg
echo "$v1 $v2 $v3 $v4 $v5 $v6 $v7"
74 74 74 74 74 74 304
Or they are the result of an "Arithmetic Expansion":
$ : $(( v1=0112, v2=74, v3=0x4a, v4=8#112, v5=10#74, v6=16#4a, v7=18#gg ))
$ echo "$v1 $v2 $v3 $v4 $v5 $v6 $v7"
74 74 74 74 74 74 304
So, you just need one way to print the character that belongs to a variable value.
But here are two possible ways:
$ var=$((0x65))
$ printf '%b\n' "\\$(printf '0%o' "$var")"
e
$ declare -i var
$ var=0x65; printf '%b\n' "\U$(printf '%08x' "$var")"
e
The two printf are needed, one to transform the value into an hexadecimal string and the second to actually print the character.
The second one will print any UNICODE point (if your console is correctly set).
For example:
$ var=0x2603; printf '%b\n' "\U$(printf '%08x' "$var")"
โ
An snow man.
The character that has an utf-8 representation as f0 9f 90 ae is 0x1F42E.
Search for cow face site:fileformat.info to get it:
$ var=0x1F42F; printf '%b\n' "\U$(printf '%08x' "$var")"
๐ฎ
Note: There is a problem with the UNICODE way in that for bash before 4.3 (corrected in that version and upwards), the characters between UNICODE points 128 and 255 (in decimal) may be incorrectly printed.
References
Fourth paragraph inside PARAMETERS in man bash:
If the variable has its integer attribute set, then value is evaluated as an arithmetic expression even if the $((...)) expansion is not used (see Arithmetic Expansion below).
Inside "ARITHMETIC EVALUATION" in man bash:
Constants with a leading 0 are interpreted as octal numbers. A leading 0x or 0X denotes hexadecimal. Otherwise, numbers take the form [base#]n, where the optional base is a decimal number between 2 and 64 representing the arithmetic base, and n is a number in that base. If base# is omitted, then base 10 is used. The digits greater than 9 are represented by the lowercase letters, the uppercase letters, @, and _, in that order. If base is less than or equal to 36, lowercase and uppercase letters may be used interchangeably to represent numbers between 10 and 35.
Hi all, freshly joined noobie here :)
I am currently working as a jr embedded software engineer, and have been struggling with data collection at runtime of the application.
I'm using a debugger that keeps sending a variable's hex value to the host pc via usb, but since this value is interpreted as ASCII, I see invalid symbols on the terminal.
As naive as it may sound, my question is: is there a way with a script to "get in between" the debugger and the terminal on the host pc to convert these hex values in their ASCII counterpart, so they are displayable "correctly"? (like, if I send 0x0123 I'd like the terminal to show "291" instead of the symbols associated with 0x01 and 0x23).
Extra question: do you have any suggestion on material I can study on to get a solid knowledge of bash scripting in general, too?
Thank you for your time and your patience, I hope I didn't sound too stupid haha.