You can use the search replace functionality of sed:
echo "00124a5b" | sed 's/../\\x&/g'
\x00\x12\x4a\x5b
The two dots search for any two characters in the stream. The \\x& replaces the match with a \x followed by the match. Adding the g on end tells sed to continue the search/replace.
I would check out this tutorial for sed: http://www.grymoire.com/Unix/Sed.html There are several other tutorials on that site for other helpful commands.
Answer from Patrick Glaser on Stack OverflowI used the -r and -p switches for xxd:
$ echo '0006303030304e43' | xxd -r -p | nc -l localhost 8181
Thanks to inspiration from @Gilles' answer, here's a Perl version:
$ echo '0006303030304e43' | perl -e 'print pack "H*", <STDIN>' | nc -l localhost 8181
Here a solution without xxd or perl:
If the echo builtin of your shell supports it (bash and zsh do, but not dash), you just need to use the right backslash escapes:
echo -ne '\x00\x06\x30\x30\x30\x30\x4e\x43' | nc -l localhost 8181
If you have /bin/echo from GNU coreutils (nearly standard on Linux systems) or from busybox you can use it, too.
With sed you can generate a escaped pattern:
$ echo '0006303030304e43' | sed -e 's/../\\x&/g'
\x00\x06\x30\x30\x30\x30\x4e\x43
Combined:
echo -ne "$(echo '0006303030304e43' | sed -e 's/../\\x&/g')" | nc -l localhost 8181
text processing - Convert hexadecimal to binary on Linux CLI - Unix & Linux Stack Exchange
shell - Linux script to convert byte data into a hex string - Stack Overflow
To Convert Hex to Byte and Then Base64 Encoding - Storage & SAN - Spiceworks Community
c - Transform hexadecimal information to binary using a Linux command - Stack Overflow
xxd -r -p
Example:
sudo apt install xxd
printf 123456789ABCDEF0 | xxd -r -p | od -A n -t x1 -v
gives:
12 34 56 78 9a bc de f0
od shows raw bytes as human readable bytes, which means that xxd -r -p did what we wanted and converted the human readable 123456789ABCDEF0 into corresponding raw bytes.
Tested on Ubuntu 19.04, xxd V1.10.
basenc --base16 -di
basenc has been available in GNU coreutils since release 8.31 (2019-03-10).
Note since release 9.5 (2024-03-28) lowercase HEX are supported.
The above command converts each sequence of 2 hexadecimal digits to the corresponding raw byte. To get a binary representation of those bytes, you can pipe to basenc --base2msbf (with most significant bit first). Example:
$ echo 00FFAA | basenc --base16 -di | basenc --base2msbf
000000001111111110101010
(add -w0 to disable line wrapping and get the whole output on one line).
It's worth noting that hex to binary depends on 2 hex digits per byte, whether it's basenc or xxd etc. doing the conversion. This may not be the case in edge cases like od producing hex without the leading 0 digit. In that case you could preprocess with something like zeropad() { sed 's/^.\(..\)*$/0&/'; } to ensure the required leading 0 is present.
Should you want only the hex strings:
$ echo '0Y0*†HÎ*†HÎ=B¬`9E>ÞÕ?ÐŽ·‹ñ6ì‰&ÉÐL_cüsyxoú¢'|od -vt x1|awk '{$1="";print}'
30 59 30 2a e2 80 a0 48 c3 8e 2a e2 80 a0 48 c3
8e 3d 42 c2 ac 60 39 45 3e c3 9e c3 95 3f c3 90
c5 bd c2 b7 e2 80 b9 c3 b1 36 c3 ac c2 ad c3 82
e2 80 b0 26 c3 89 c3 90 4c 5f 63 c3 bc 73 79 78
6f c3 ba c2 a2 0a
You can avoid the awk part by just using od -vt x1 -A n. Thanks @Stefan van den Akker.
Use the hd (hex dump) command:
$ echo '0Y0*†HÎ*†HÎ=B¬`9E>ÞÕ?ÐŽ·‹ñ6ì‰&ÉÐL_cüsyxoú¢' | hd
00000000 30 59 30 2a e2 80 a0 48 c3 8e 2a e2 80 a0 48 c3 |0Y0*...H..*...H.|
00000010 8e 3d 42 c2 ac 60 39 45 3e c3 9e c3 95 3f c3 90 |.=B..`9E>....?..|
00000020 c5 bd c2 b7 e2 80 b9 c3 b1 36 c3 ac c2 ad c3 82 |.........6......|
00000030 e2 80 b0 26 c3 89 c3 90 4c 5f 63 c3 bc 73 79 78 |...&....L_c..syx|
00000040 6f c3 ba c2 a2 0a |o.....|
00000046
Or, if you don't have hd, hexdump:
$ echo '0Y0*†HÎ*†HÎ=B¬`9E>ÞÕ?ÐŽ·‹ñ6ì‰&ÉÐL_cüsyxoú¢' | hexdump
0000000 5930 2a30 80e2 48a0 8ec3 e22a a080 c348
0000010 3d8e c242 60ac 4539 c33e c39e 3f95 90c3
0000020 bdc5 b7c2 80e2 c3b9 36b1 acc3 adc2 82c3
0000030 80e2 26b0 89c3 90c3 5f4c c363 73bc 7879
0000040 c36f c2ba 0aa2
0000046
As @user786653 suggested, use the xxd(1) program:
xxd -r -p input.txt output.bin
Python stdlib solution
If for some unfathomably enterprisey reason you can't sudo apt install xxd, it is easy to reimplement it in Python as per: How to create python bytes object from long hex string? with:
xxd2() ( python -c "import sys;import fileinput;sys.stdout.buffer.write(bytes.fromhex(''.join(fileinput.input(sys.argv[1:]))))" "$@" )
which works both with files and stdin:
printf 01ab | xxd2
printf '01 ab' | xxd2
or:
printf 01ab > myfile.hex
xxd2 myfile.hex
Here's the script with better indentation:
import sys
import fileinput
sys.stdout.buffer.write(
bytes.fromhex(
''.join(
fileinput.input(sys.argv[1:])
)
)
)
The bytes.fromhex function ignores whitespaces and newlines since Python 3.7, so it works regardless of the indentation details of the format, as per docs: https://docs.python.org/3.12/library/stdtypes.html#bytes.fromhex
Changed in version 3.7: bytes.fromhex() now skips all ASCII whitespace in the string, not just spaces.
Tested on Python 3.12.3, Ubuntu 24.04.
I came here seeing three answers thinking that I'd have nothing to add, and that this would be an exercise in how many people can post the same 1-liner in the first minute of a question being asked. But I find people using some new-fangled hexdump tool. That command is way longer than 2 letters; it alludes to some base other than The One True Base (base 8); and it's even apparent from its name what it does. Clearly this is not the Unix way.
So here's the joy of od ("octal dump").
First GNU, as you will find on your Linux Mint:
od --format=x1 --read-bytes=10 foo
Now BSD, where the irony is that it's actually the same program as hexdump:
od -t x1 -N 10 foo
Option -l <len> | -len <len> is for: stop after writing <len> octets.
Use it with a FILE like this:
xxd -l 10 FILE
or
hexdump -C -n 10 FILE
where -n <len> is the same as the -l <len> option from xxd.
Ok so i am trying to understand this concept (I have many questions)
Bytes are 1s and 0s, so binary Binary can be translated to hexadecimal So then can bytes be 1s and 0s and hexadecimal, like any of them two, (not sure about that tho)
I was watching a pentesting video on a hackthebox machine, and the guy was trying to change the so called magic bytes of a file so that the system recognized the file as a different file then it actually was, to do that he printed in plain text some hex stuff (which was the file signature in hex) like \xAb\xC3\xf3 etc etc into the file (echo \xAb\xC3\xF3 > file.txt) , so another question is when the computer read all these hex characters, did it import it in the bytes just like that, or did it recognize it has hex and then convert it to binary and then into the bytes
As you see i am very confused with bytes and how they store information, and how you can “insert” information in to bytes, and how all that would work. So if anybody could please explain like im 5 i would appreciate it a lot, as i am just starting to learn all of this stuff
Thank you!!
Ps: I hope you understand my question as I am not very good at explaining myself lol
Use hexdump(1)
$ hexdump -x /usr/bin/hexdump
0000000 feca beba 0000 0300 0001 0700 0080 0300
0000010 0000 0010 0000 5080 0000 0c00 0000 0700
0000020 0000 0300 0000 00a0 0000 b06f 0000 0c00
0000030 0000 1200 0000 0a00 0100 0010 0000 107c
0000040 0000 0c00 0000 0000 0000 0000 0000 0000
0000050 0000 0000 0000 0000 0000 0000 0000 0000
...
Another option is od:
od -t x1 FILE
sample output:
$ printf '0123456789abcdef0123456789abcdef\x00\x01\x02\x03\x04\x05\x06\x07\x08\x09\x0a\x0b\x0c\x0d\x0e\x0f' | od -t x1
0000000 30 31 32 33 34 35 36 37 38 39 61 62 63 64 65 66
*
0000040 00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e 0f
0000060
or
od -x FILE
sample output:
0000000 3130 3332 3534 3736 3938 6261 6463 6665
*
0000040 0100 0302 0504 0706 0908 0b0a 0d0c 0f0e
0000060
od has many options for finetuning.
The reason is because hexdump by default prints out 16-bit integers, not bytes. If your system has them, hd (or hexdump -C) or xxd will provide less surprising outputs - if not, od -t x1 is a POSIX-standard way to get byte-by-byte hex output. You can use od -t x1c to show both the byte hex values and the corresponding letters.
If you have xxd (which ships with vim), you can use xxd -r to convert back from hex (from the same format xxd produces). If you just have plain hex (just the '4161', which is produced by xxd -p) you can use xxd -r -p to convert back.
For the first part, try
echo Aa | od -t x1
It prints byte-by-byte
$ echo Aa | od -t x1
0000000 41 61 0a
0000003
The 0a is the implicit newline that echo produces.
Use echo -n or printf instead.
$ printf Aa | od -t x1
0000000 41 61
0000002
If you want to get the hex values of some string, then this works:
$ echo "testing some values"$'\157' | xxd
0000000: 7465 7374 696e 6720 736f 6d65 2076 616c testing some val
0000010: 7565 736f 0a ueso.
If you just need the "plain" string:
$ echo "testing some values"$'\157' | xxd -p
74657374696e6720736f6d652076616c7565736f0a
If you need to "reverse" an hex string, you do:
$ echo "74657374696e6720736f6d652076616c7565736f0a" | xxd -r -p
testing some valueso
If what you need is the character representation (not hex), you could do:
$ echo "testing:"$'\001\011\n\bend test' | od -vAn -tcx1
t e s t i n g : 001 \t \n \b e n d
74 65 73 74 69 6e 67 3a 01 09 0a 08 65 6e 64 20
t e s t \n
74 65 73 74 0a
Or:
$ echo "testing:"$'\001\011\n\bend test' | od -vAn -tax1
t e s t i n g : soh ht nl bs e n d sp
74 65 73 74 69 6e 67 3a 01 09 0a 08 65 6e 64 20
t e s t nl
74 65 73 74 0a
Looks like you want to convert hex string to bytes. xxd needs a starting point (since it is used to patch binary files and such)
string="626c610a"
echo "0: $string" | xxd -r
xxd will also silently skip any invalid hex, so check your data if you get empty output.