How do you convert a byte array to a hexadecimal string in C? - Stack Overflow
Understanding bytes and hex
arrays - How to convert hexadecimal string to bytes in Python? - Stack Overflow
c# - byte[] to hex string - Stack Overflow
printf("%02X:%02X:%02X:%02X", buf[0], buf[1], buf[2], buf[3]);
For a more generic way:
int i;
for (i = 0; i < x; i++)
{
if (i > 0) printf(":");
printf("%02X", buf[i]);
}
printf("\n");
To concatenate to a string, there are a few ways you can do this. I'd probably keep a pointer to the end of the string and use sprintf. You should also keep track of the size of the array to make sure it doesn't get larger than the space allocated:
int i;
char* buf2 = stringbuf;
char* endofbuf = stringbuf + sizeof(stringbuf);
for (i = 0; i < x; i++)
{
/* i use 5 here since we are going to add at most
3 chars, need a space for the end '\n' and need
a null terminator */
if (buf2 + 5 < endofbuf)
{
if (i > 0)
{
buf2 += sprintf(buf2, ":");
}
buf2 += sprintf(buf2, "%02X", buf[i]);
}
}
buf2 += sprintf(buf2, "\n");
For completude, you can also easily do it without calling any heavy library function (no snprintf, no strcat, not even memcpy). It can be useful, say if you are programming some microcontroller or OS kernel where libc is not available.
Nothing really fancy you can find similar code around if you google for it. Really it's not much more complicated than calling snprintf and much faster.
#include <stdio.h>
int main(){
unsigned char buf[] = {0, 1, 10, 11};
/* target buffer should be large enough */
char str[12];
unsigned char * pin = buf;
const char * hex = "0123456789ABCDEF";
char * pout = str;
int i = 0;
for(; i < sizeof(buf)-1; ++i){
*pout++ = hex[(*pin>>4)&0xF];
*pout++ = hex[(*pin++)&0xF];
*pout++ = ':';
}
*pout++ = hex[(*pin>>4)&0xF];
*pout++ = hex[(*pin)&0xF];
*pout = 0;
printf("%s\n", str);
}
Here is another slightly shorter version. It merely avoid intermediate index variable i and duplicating laste case code (but the terminating character is written two times).
#include <stdio.h>
int main(){
unsigned char buf[] = {0, 1, 10, 11};
/* target buffer should be large enough */
char str[12];
unsigned char * pin = buf;
const char * hex = "0123456789ABCDEF";
char * pout = str;
for(; pin < buf+sizeof(buf); pout+=3, pin++){
pout[0] = hex[(*pin>>4) & 0xF];
pout[1] = hex[ *pin & 0xF];
pout[2] = ':';
}
pout[-1] = 0;
printf("%s\n", str);
}
Below is yet another version to answer to a comment saying I used a "trick" to know the size of the input buffer. Actually it's not a trick but a necessary input knowledge (you need to know the size of the data that you are converting). I made this clearer by extracting the conversion code to a separate function. I also added boundary check code for target buffer, which is not really necessary if we know what we are doing.
#include <stdio.h>
void tohex(unsigned char * in, size_t insz, char * out, size_t outsz)
{
unsigned char * pin = in;
const char * hex = "0123456789ABCDEF";
char * pout = out;
for(; pin < in+insz; pout +=3, pin++){
pout[0] = hex[(*pin>>4) & 0xF];
pout[1] = hex[ *pin & 0xF];
pout[2] = ':';
if (pout + 3 - out > outsz){
/* Better to truncate output string than overflow buffer */
/* it would be still better to either return a status */
/* or ensure the target buffer is large enough and it never happen */
break;
}
}
pout[-1] = 0;
}
int main(){
enum {insz = 4, outsz = 3*insz};
unsigned char buf[] = {0, 1, 10, 11};
char str[outsz];
tohex(buf, insz, str, outsz);
printf("%s\n", str);
}
Ok so i am trying to understand this concept (I have many questions)
Bytes are 1s and 0s, so binary Binary can be translated to hexadecimal So then can bytes be 1s and 0s and hexadecimal, like any of them two, (not sure about that tho)
I was watching a pentesting video on a hackthebox machine, and the guy was trying to change the so called magic bytes of a file so that the system recognized the file as a different file then it actually was, to do that he printed in plain text some hex stuff (which was the file signature in hex) like \xAb\xC3\xf3 etc etc into the file (echo \xAb\xC3\xF3 > file.txt) , so another question is when the computer read all these hex characters, did it import it in the bytes just like that, or did it recognize it has hex and then convert it to binary and then into the bytes
As you see i am very confused with bytes and how they store information, and how you can “insert” information in to bytes, and how all that would work. So if anybody could please explain like im 5 i would appreciate it a lot, as i am just starting to learn all of this stuff
Thank you!!
Ps: I hope you understand my question as I am not very good at explaining myself lol
Suppose your hex string is something like
>>> hex_string = "deadbeef"
Convert it to a bytearray (Python 3 and 2.7):
>>> bytearray.fromhex(hex_string)
bytearray(b'\xde\xad\xbe\xef')
Convert it to a bytes object (Python 3):
>>> bytes.fromhex(hex_string)
b'\xde\xad\xbe\xef'
Note that bytes is an immutable version of bytearray.
Convert it to a string (Python ≤ 2.7):
>>> hex_data = hex_string.decode("hex")
>>> hex_data
"\xde\xad\xbe\xef"
There is a built-in function in bytearray that does what you intend.
bytearray.fromhex("de ad be ef 00")
It returns a bytearray and it reads hex strings with or without space separator.
There is a built in method for this:
byte[] data = { 1, 2, 4, 8, 16, 32 };
string hex = BitConverter.ToString(data);
Result: 01-02-04-08-10-20
If you want it without the dashes, just remove them:
string hex = BitConverter.ToString(data).Replace("-", string.Empty);
Result: 010204081020
If you want a more compact representation, you can use Base64:
string base64 = Convert.ToBase64String(data);
Result: AQIECBAg
.Net5.0 Update
Thanks to @antoninkriz's benchmark comparison we can see that Convert.ToHexString is by far the clear winner today
You'd be foolish to use anything other than Convert.ToHexString. In my opinion it clearly wins in: readability, performance, safety
The rest of this is from before Apr 10, 2012:
I thought I would attempt to compare the speed of each of the methods listed here for the hell of it. I based the speed testing code off this.
The result is that BitConverter+String.Replace seems to be faster than most other simple ways. But the speed can be improved with algorithms like Nathan Moinvaziri's ByteArrayToHexString or Kurt's ToHex.
I also found it interesting that string.Concat and string.Join are much slower than StringBuilder implementations for long strings, but similar for shorter arrays. Probably due to expanding the StringBuilder on the longer strings, so setting the initial size should negate this difference.
- Took each bit of code from an answer here:
- BitConvertRep = Answer by Guffa, BitConverter and String.Replace (I'd recommend for most cases, [edit:] where you can't use
Convert.ToHexString) - StringBuilder = Answer by Quintin Robinson, foreach char StringBuilder.Append
- LinqConcat = Answer by Michael Buen, string.Concat of Linq built array
- LinqJoin = Answer by mloskot, string.Join of Linq built array
- LinqAgg = Answer by Matthew Whited, IEnumerable.Aggregate with StringBuilder
- ToHex = Answer by Kurt, sets chars in an array, using byte values to get hex
- ByteArrayToHexString = Answer by Nathan Moinvaziri, approx same speed as the ToHex above, and is probably easier to read (I'd recommend for speed, [edit:] where you can't use
Convert.ToHexString) - ToHexFromTable = Linked in answer by Nathan Moinvaziri, for me this is near the same speed as the above 2 but requires an array of 256 strings to always exist
With:
LONG_STRING_LENGTH = 1000 * 1024;
- BitConvertRep calculation Time Elapsed 27,202 ms (fastest built in/simple)
- StringBuilder calculation Time Elapsed 75,723 ms (StringBuilder no reallocate)
- LinqConcat calculation Time Elapsed 182,094 ms
- LinqJoin calculation Time Elapsed 181,142 ms
- LinqAgg calculation Time Elapsed 93,087 ms (StringBuilder with reallocating)
- ToHex calculation Time Elapsed 19,167 ms (fastest)
With:
LONG_STRING_LENGTH = 100 * 1024;, Similar results
- BitConvertReplace calculation Time Elapsed 3431 ms
- StringBuilder calculation Time Elapsed 8289 ms
- LinqConcat calculation Time Elapsed 21512 ms
- LinqJoin calculation Time Elapsed 19433 ms
- LinqAgg calculation Time Elapsed 9230 ms
- ToHex calculation Time Elapsed 1976 ms
With:
int MANY_STRING_COUNT = 1000;int MANY_STRING_LENGTH = 1024;(Same byte count as first test but in different arrays)
- BitConvertReplace calculation Time Elapsed 25,680 ms
- StringBuilder calculation Time Elapsed 78,411 ms
- LinqConcat calculation Time Elapsed 101,233 ms
- LinqJoin calculation Time Elapsed 99,311 ms
- LinqAgg calculation Time Elapsed 84,660 ms
- ToHex calculation Time Elapsed 18,221 ms
With:
int MANY_STRING_COUNT = 2000;int MANY_STRING_LENGTH = 20;
- BitConvertReplace calculation Time Elapsed 1347 ms
- StringBuilder calculation Time Elapsed 3234 ms
- LinqConcat calculation Time Elapsed 5013 ms
- LinqJoin calculation Time Elapsed 4826 ms
- LinqAgg calculation Time Elapsed 3589 ms
- ToHex calculation Time Elapsed 772 ms
Testing code I used:
void Main()
{
int LONG_STRING_LENGTH = 100 * 1024;
int MANY_STRING_COUNT = 1024;
int MANY_STRING_LENGTH = 100;
var source = GetRandomBytes(LONG_STRING_LENGTH);
List<byte[]> manyString = new List<byte[]>(MANY_STRING_COUNT);
for (int i = 0; i < MANY_STRING_COUNT; ++i)
{
manyString.Add(GetRandomBytes(MANY_STRING_LENGTH));
}
var algorithms = new Dictionary<string,Func<byte[], string>>();
algorithms["BitConvertReplace"] = BitConv;
algorithms["StringBuilder"] = StringBuilderTest;
algorithms["LinqConcat"] = LinqConcat;
algorithms["LinqJoin"] = LinqJoin;
algorithms["LinqAgg"] = LinqAgg;
algorithms["ToHex"] = ToHex;
algorithms["ByteArrayToHexString"] = ByteArrayToHexString;
Console.WriteLine(" === Long string test");
foreach (var pair in algorithms) {
TimeAction(pair.Key + " calculation", 500, () =>
{
pair.Value(source);
});
}
Console.WriteLine(" === Many string test");
foreach (var pair in algorithms) {
TimeAction(pair.Key + " calculation", 500, () =>
{
foreach (var str in manyString)
{
pair.Value(str);
}
});
}
}
// Define other methods and classes here
static void TimeAction(string description, int iterations, Action func) {
var watch = new Stopwatch();
watch.Start();
for (int i = 0; i < iterations; i++) {
func();
}
watch.Stop();
Console.Write(description);
Console.WriteLine(" Time Elapsed {0} ms", watch.ElapsedMilliseconds);
}
//static byte[] GetRandomBytes(int count) {
// var bytes = new byte[count];
// (new Random()).NextBytes(bytes);
// return bytes;
//}
static Random rand = new Random();
static byte[] GetRandomBytes(int count) {
var bytes = new byte[count];
rand.NextBytes(bytes);
return bytes;
}
static string BitConv(byte[] data)
{
return BitConverter.ToString(data).Replace("-", string.Empty);
}
static string StringBuilderTest(byte[] data)
{
StringBuilder sb = new StringBuilder(data.Length*2);
foreach (byte b in data)
sb.Append(b.ToString("X2"));
return sb.ToString();
}
static string LinqConcat(byte[] data)
{
return string.Concat(data.Select(b => b.ToString("X2")).ToArray());
}
static string LinqJoin(byte[] data)
{
return string.Join("",
data.Select(
bin => bin.ToString("X2")
).ToArray());
}
static string LinqAgg(byte[] data)
{
return data.Aggregate(new StringBuilder(),
(sb,v)=>sb.Append(v.ToString("X2"))
).ToString();
}
static string ToHex(byte[] bytes)
{
char[] c = new char[bytes.Length * 2];
byte b;
for(int bx = 0, cx = 0; bx < bytes.Length; ++bx, ++cx)
{
b = ((byte)(bytes[bx] >> 4));
c[cx] = (char)(b > 9 ? b - 10 + 'A' : b + '0');
b = ((byte)(bytes[bx] & 0x0F));
c[++cx] = (char)(b > 9 ? b - 10 + 'A' : b + '0');
}
return new string(c);
}
public static string ByteArrayToHexString(byte[] Bytes)
{
StringBuilder Result = new StringBuilder(Bytes.Length*2);
string HexAlphabet = "0123456789ABCDEF";
foreach (byte B in Bytes)
{
Result.Append(HexAlphabet[(int)(B >> 4)]);
Result.Append(HexAlphabet[(int)(B & 0xF)]);
}
return Result.ToString();
}
Also another answer with a similar process, I haven't compared our results yet.
What I'm trying to do is convert a string like "abcd" into bytes that represent the hexadecimal of the string oxabcd. Python keeps telling me OverflowError: int too big to convert
The code is the following:
a = "abcd" print(int(a, 16).to_bytes())
The error in full is:
Traceback (most recent call last):
File "/home/repl109/main.py", line 6, in <module>
print(int(a, 16).to_bytes())
^^^^^^^^^^^^^^^^^^^^^
OverflowError: int too big to convertAnyone know a way to do this for large integers?