Since Python 3.5 this is finally no longer awkward:
>>> b'\xde\xad\xbe\xef'.hex()
'deadbeef'
and reverse:
>>> bytes.fromhex('deadbeef')
b'\xde\xad\xbe\xef'
works also with the mutable bytearray type.
Reference: https://docs.python.org/3/library/stdtypes.html#bytes.hex
Answer from Felix Weis on Stack OverflowSince Python 3.5 this is finally no longer awkward:
>>> b'\xde\xad\xbe\xef'.hex()
'deadbeef'
and reverse:
>>> bytes.fromhex('deadbeef')
b'\xde\xad\xbe\xef'
works also with the mutable bytearray type.
Reference: https://docs.python.org/3/library/stdtypes.html#bytes.hex
Use the binascii module:
>>> import binascii
>>> binascii.hexlify('foo'.encode('utf8'))
b'666f6f'
>>> binascii.unhexlify(_).decode('utf8')
'foo'
See this answer: Python 3 string to hex
There is a built in method for this:
byte[] data = { 1, 2, 4, 8, 16, 32 };
string hex = BitConverter.ToString(data);
Result: 01-02-04-08-10-20
If you want it without the dashes, just remove them:
string hex = BitConverter.ToString(data).Replace("-", string.Empty);
Result: 010204081020
If you want a more compact representation, you can use Base64:
string base64 = Convert.ToBase64String(data);
Result: AQIECBAg
.Net5.0 Update
Thanks to @antoninkriz's benchmark comparison we can see that Convert.ToHexString is by far the clear winner today
You'd be foolish to use anything other than Convert.ToHexString. In my opinion it clearly wins in: readability, performance, safety
The rest of this is from before Apr 10, 2012:
I thought I would attempt to compare the speed of each of the methods listed here for the hell of it. I based the speed testing code off this.
The result is that BitConverter+String.Replace seems to be faster than most other simple ways. But the speed can be improved with algorithms like Nathan Moinvaziri's ByteArrayToHexString or Kurt's ToHex.
I also found it interesting that string.Concat and string.Join are much slower than StringBuilder implementations for long strings, but similar for shorter arrays. Probably due to expanding the StringBuilder on the longer strings, so setting the initial size should negate this difference.
- Took each bit of code from an answer here:
- BitConvertRep = Answer by Guffa, BitConverter and String.Replace (I'd recommend for most cases, [edit:] where you can't use
Convert.ToHexString) - StringBuilder = Answer by Quintin Robinson, foreach char StringBuilder.Append
- LinqConcat = Answer by Michael Buen, string.Concat of Linq built array
- LinqJoin = Answer by mloskot, string.Join of Linq built array
- LinqAgg = Answer by Matthew Whited, IEnumerable.Aggregate with StringBuilder
- ToHex = Answer by Kurt, sets chars in an array, using byte values to get hex
- ByteArrayToHexString = Answer by Nathan Moinvaziri, approx same speed as the ToHex above, and is probably easier to read (I'd recommend for speed, [edit:] where you can't use
Convert.ToHexString) - ToHexFromTable = Linked in answer by Nathan Moinvaziri, for me this is near the same speed as the above 2 but requires an array of 256 strings to always exist
With:
LONG_STRING_LENGTH = 1000 * 1024;
- BitConvertRep calculation Time Elapsed 27,202 ms (fastest built in/simple)
- StringBuilder calculation Time Elapsed 75,723 ms (StringBuilder no reallocate)
- LinqConcat calculation Time Elapsed 182,094 ms
- LinqJoin calculation Time Elapsed 181,142 ms
- LinqAgg calculation Time Elapsed 93,087 ms (StringBuilder with reallocating)
- ToHex calculation Time Elapsed 19,167 ms (fastest)
With:
LONG_STRING_LENGTH = 100 * 1024;, Similar results
- BitConvertReplace calculation Time Elapsed 3431 ms
- StringBuilder calculation Time Elapsed 8289 ms
- LinqConcat calculation Time Elapsed 21512 ms
- LinqJoin calculation Time Elapsed 19433 ms
- LinqAgg calculation Time Elapsed 9230 ms
- ToHex calculation Time Elapsed 1976 ms
With:
int MANY_STRING_COUNT = 1000;int MANY_STRING_LENGTH = 1024;(Same byte count as first test but in different arrays)
- BitConvertReplace calculation Time Elapsed 25,680 ms
- StringBuilder calculation Time Elapsed 78,411 ms
- LinqConcat calculation Time Elapsed 101,233 ms
- LinqJoin calculation Time Elapsed 99,311 ms
- LinqAgg calculation Time Elapsed 84,660 ms
- ToHex calculation Time Elapsed 18,221 ms
With:
int MANY_STRING_COUNT = 2000;int MANY_STRING_LENGTH = 20;
- BitConvertReplace calculation Time Elapsed 1347 ms
- StringBuilder calculation Time Elapsed 3234 ms
- LinqConcat calculation Time Elapsed 5013 ms
- LinqJoin calculation Time Elapsed 4826 ms
- LinqAgg calculation Time Elapsed 3589 ms
- ToHex calculation Time Elapsed 772 ms
Testing code I used:
void Main()
{
int LONG_STRING_LENGTH = 100 * 1024;
int MANY_STRING_COUNT = 1024;
int MANY_STRING_LENGTH = 100;
var source = GetRandomBytes(LONG_STRING_LENGTH);
List<byte[]> manyString = new List<byte[]>(MANY_STRING_COUNT);
for (int i = 0; i < MANY_STRING_COUNT; ++i)
{
manyString.Add(GetRandomBytes(MANY_STRING_LENGTH));
}
var algorithms = new Dictionary<string,Func<byte[], string>>();
algorithms["BitConvertReplace"] = BitConv;
algorithms["StringBuilder"] = StringBuilderTest;
algorithms["LinqConcat"] = LinqConcat;
algorithms["LinqJoin"] = LinqJoin;
algorithms["LinqAgg"] = LinqAgg;
algorithms["ToHex"] = ToHex;
algorithms["ByteArrayToHexString"] = ByteArrayToHexString;
Console.WriteLine(" === Long string test");
foreach (var pair in algorithms) {
TimeAction(pair.Key + " calculation", 500, () =>
{
pair.Value(source);
});
}
Console.WriteLine(" === Many string test");
foreach (var pair in algorithms) {
TimeAction(pair.Key + " calculation", 500, () =>
{
foreach (var str in manyString)
{
pair.Value(str);
}
});
}
}
// Define other methods and classes here
static void TimeAction(string description, int iterations, Action func) {
var watch = new Stopwatch();
watch.Start();
for (int i = 0; i < iterations; i++) {
func();
}
watch.Stop();
Console.Write(description);
Console.WriteLine(" Time Elapsed {0} ms", watch.ElapsedMilliseconds);
}
//static byte[] GetRandomBytes(int count) {
// var bytes = new byte[count];
// (new Random()).NextBytes(bytes);
// return bytes;
//}
static Random rand = new Random();
static byte[] GetRandomBytes(int count) {
var bytes = new byte[count];
rand.NextBytes(bytes);
return bytes;
}
static string BitConv(byte[] data)
{
return BitConverter.ToString(data).Replace("-", string.Empty);
}
static string StringBuilderTest(byte[] data)
{
StringBuilder sb = new StringBuilder(data.Length*2);
foreach (byte b in data)
sb.Append(b.ToString("X2"));
return sb.ToString();
}
static string LinqConcat(byte[] data)
{
return string.Concat(data.Select(b => b.ToString("X2")).ToArray());
}
static string LinqJoin(byte[] data)
{
return string.Join("",
data.Select(
bin => bin.ToString("X2")
).ToArray());
}
static string LinqAgg(byte[] data)
{
return data.Aggregate(new StringBuilder(),
(sb,v)=>sb.Append(v.ToString("X2"))
).ToString();
}
static string ToHex(byte[] bytes)
{
char[] c = new char[bytes.Length * 2];
byte b;
for(int bx = 0, cx = 0; bx < bytes.Length; ++bx, ++cx)
{
b = ((byte)(bytes[bx] >> 4));
c[cx] = (char)(b > 9 ? b - 10 + 'A' : b + '0');
b = ((byte)(bytes[bx] & 0x0F));
c[++cx] = (char)(b > 9 ? b - 10 + 'A' : b + '0');
}
return new string(c);
}
public static string ByteArrayToHexString(byte[] Bytes)
{
StringBuilder Result = new StringBuilder(Bytes.Length*2);
string HexAlphabet = "0123456789ABCDEF";
foreach (byte B in Bytes)
{
Result.Append(HexAlphabet[(int)(B >> 4)]);
Result.Append(HexAlphabet[(int)(B & 0xF)]);
}
return Result.ToString();
}
Also another answer with a similar process, I haven't compared our results yet.
You can use Convert.ToHexString starting with .NET 5.
There's also a method for the reverse operation: Convert.FromHexString.
For older versions of .NET you can either use:
public static string ByteArrayToString(byte[] ba)
{
StringBuilder hex = new StringBuilder(ba.Length * 2);
foreach (byte b in ba)
hex.AppendFormat("{0:x2}", b);
return hex.ToString();
}
or:
public static string ByteArrayToString(byte[] ba)
{
return BitConverter.ToString(ba).Replace("-","");
}
There are even more variants of doing it, for example here.
The reverse conversion would go like this:
public static byte[] StringToByteArray(String hex)
{
int NumberChars = hex.Length;
byte[] bytes = new byte[NumberChars / 2];
for (int i = 0; i < NumberChars; i += 2)
bytes[i / 2] = Convert.ToByte(hex.Substring(i, 2), 16);
return bytes;
}
Using Substring is the best option in combination with Convert.ToByte. See this answer for more information. If you need better performance, you must avoid Convert.ToByte before you can drop SubString.
Performance Analysis
Note: new leader as of 2015-08-20.
I ran each of the various conversion methods through some crude Stopwatch performance testing, a run with a random sentence (n=61, 1000 iterations) and a run with a Project Gutenburg text (n=1,238,957, 150 iterations). Here are the results, roughly from fastest to slowest. All measurements are in ticks (10,000 ticks = 1 ms) and all relative notes are compared to the [slowest] StringBuilder implementation. For the code used, see below or the test framework repo where I now maintain the code for running this.
Disclaimer
WARNING: Do not rely on these stats for anything concrete; they are simply a sample run of sample data. If you really need top-notch performance, please test these methods in an environment representative of your production needs with data representative of what you will use.
Results
- Lookup by byte
unsafe(via CodesInChaos) (added to test repo by airbreather)- Text: 4,727.85 (105.2X)
- Sentence: 0.28 (99.7X)
- Lookup by byte (via CodesInChaos)
- Text: 10,853.96 (45.8X faster)
- Sentence: 0.65 (42.7X faster)
- Byte Manipulation 2 (via CodesInChaos)
- Text: 12,967.69 (38.4X faster)
- Sentence: 0.73 (37.9X faster)
- Byte Manipulation (via Waleed Eissa)
- Text: 16,856.64 (29.5X faster)
- Sentence: 0.70 (39.5X faster)
- Lookup/Shift (via Nathan Moinvaziri)
- Text: 23,201.23 (21.4X faster)
- Sentence: 1.24 (22.3X faster)
- Lookup by nibble (via Brian Lambert)
- Text: 23,879.41 (20.8X faster)
- Sentence: 1.15 (23.9X faster)
BitConverter(via Tomalak)- Text: 113,269.34 (4.4X faster)
- Sentence: 9.98 (2.8X faster)
{SoapHexBinary}.ToString(via Mykroft)- Text: 178,601.39 (2.8X faster)
- Sentence: 10.68 (2.6X faster)
{byte}.ToString("X2")(usingforeach) (derived from Will Dean's answer)- Text: 308,805.38 (2.4X faster)
- Sentence: 16.89 (2.4X faster)
{byte}.ToString("X2")(using{IEnumerable}.Aggregate, requires System.Linq) (via Mark)- Text: 352,828.20 (2.1X faster)
- Sentence: 16.87 (2.4X faster)
Array.ConvertAll(usingstring.Join) (via Will Dean)- Text: 675,451.57 (1.1X faster)
- Sentence: 17.95 (2.2X faster)
Array.ConvertAll(usingstring.Concat, requires .NET 4.0) (via Will Dean)- Text: 752,078.70 (1.0X faster)
- Sentence: 18.28 (2.2X faster)
{StringBuilder}.AppendFormat(usingforeach) (via Tomalak)- Text: 672,115.77 (1.1X faster)
- Sentence: 36.82 (1.1X faster)
{StringBuilder}.AppendFormat(using{IEnumerable}.Aggregate, requires System.Linq) (derived from Tomalak's answer)- Text: 718,380.63 (1.0X faster)
- Sentence: 39.71 (1.0X faster)
Lookup tables have taken the lead over byte manipulation. Basically, there is some form of precomputing what any given nibble or byte will be in hex. Then, as you rip through the data, you simply look up the next portion to see what hex string it would be. That value is then added to the resulting string output in some fashion. For a long time byte manipulation, potentially harder to read by some developers, was the top-performing approach.
Your best bet is still going to be finding some representative data and trying it out in a production-like environment. If you have different memory constraints, you may prefer a method with fewer allocations to one that would be faster but consume more memory.
Testing Code
Feel free to play with the testing code I used. A version is included here but feel free to clone the repo and add your own methods. Please submit a pull request if you find anything interesting or want to help improve the testing framework it uses.
- Add the new static method (
Func<byte[], string>) to /Tests/ConvertByteArrayToHexString/Test.cs. - Add that method's name to the
TestCandidatesreturn value in that same class. - Make sure you are running the input version you want, sentence or text, by toggling the comments in
GenerateTestInputin that same class. - Hit F5 and wait for the output (an HTML dump is also generated in the /bin folder).
static string ByteArrayToHexStringViaStringJoinArrayConvertAll(byte[] bytes) {
return string.Join(string.Empty, Array.ConvertAll(bytes, b => b.ToString("X2")));
}
static string ByteArrayToHexStringViaStringConcatArrayConvertAll(byte[] bytes) {
return string.Concat(Array.ConvertAll(bytes, b => b.ToString("X2")));
}
static string ByteArrayToHexStringViaBitConverter(byte[] bytes) {
string hex = BitConverter.ToString(bytes);
return hex.Replace("-", "");
}
static string ByteArrayToHexStringViaStringBuilderAggregateByteToString(byte[] bytes) {
return bytes.Aggregate(new StringBuilder(bytes.Length * 2), (sb, b) => sb.Append(b.ToString("X2"))).ToString();
}
static string ByteArrayToHexStringViaStringBuilderForEachByteToString(byte[] bytes) {
StringBuilder hex = new StringBuilder(bytes.Length * 2);
foreach (byte b in bytes)
hex.Append(b.ToString("X2"));
return hex.ToString();
}
static string ByteArrayToHexStringViaStringBuilderAggregateAppendFormat(byte[] bytes) {
return bytes.Aggregate(new StringBuilder(bytes.Length * 2), (sb, b) => sb.AppendFormat("{0:X2}", b)).ToString();
}
static string ByteArrayToHexStringViaStringBuilderForEachAppendFormat(byte[] bytes) {
StringBuilder hex = new StringBuilder(bytes.Length * 2);
foreach (byte b in bytes)
hex.AppendFormat("{0:X2}", b);
return hex.ToString();
}
static string ByteArrayToHexViaByteManipulation(byte[] bytes) {
char[] c = new char[bytes.Length * 2];
byte b;
for (int i = 0; i < bytes.Length; i++) {
b = ((byte)(bytes[i] >> 4));
c[i * 2] = (char)(b > 9 ? b + 0x37 : b + 0x30);
b = ((byte)(bytes[i] & 0xF));
c[i * 2 + 1] = (char)(b > 9 ? b + 0x37 : b + 0x30);
}
return new string(c);
}
static string ByteArrayToHexViaByteManipulation2(byte[] bytes) {
char[] c = new char[bytes.Length * 2];
int b;
for (int i = 0; i < bytes.Length; i++) {
b = bytes[i] >> 4;
c[i * 2] = (char)(55 + b + (((b - 10) >> 31) & -7));
b = bytes[i] & 0xF;
c[i * 2 + 1] = (char)(55 + b + (((b - 10) >> 31) & -7));
}
return new string(c);
}
static string ByteArrayToHexViaSoapHexBinary(byte[] bytes) {
SoapHexBinary soapHexBinary = new SoapHexBinary(bytes);
return soapHexBinary.ToString();
}
static string ByteArrayToHexViaLookupAndShift(byte[] bytes) {
StringBuilder result = new StringBuilder(bytes.Length * 2);
string hexAlphabet = "0123456789ABCDEF";
foreach (byte b in bytes) {
result.Append(hexAlphabet[(int)(b >> 4)]);
result.Append(hexAlphabet[(int)(b & 0xF)]);
}
return result.ToString();
}
static readonly uint* _lookup32UnsafeP = (uint*)GCHandle.Alloc(_Lookup32, GCHandleType.Pinned).AddrOfPinnedObject();
static string ByteArrayToHexViaLookup32UnsafeDirect(byte[] bytes) {
var lookupP = _lookup32UnsafeP;
var result = new string((char)0, bytes.Length * 2);
fixed (byte* bytesP = bytes)
fixed (char* resultP = result) {
uint* resultP2 = (uint*)resultP;
for (int i = 0; i < bytes.Length; i++) {
resultP2[i] = lookupP[bytesP[i]];
}
}
return result;
}
static uint[] _Lookup32 = Enumerable.Range(0, 255).Select(i => {
string s = i.ToString("X2");
return ((uint)s[0]) + ((uint)s[1] << 16);
}).ToArray();
static string ByteArrayToHexViaLookupPerByte(byte[] bytes) {
var result = new char[bytes.Length * 2];
for (int i = 0; i < bytes.Length; i++)
{
var val = _Lookup32[bytes[i]];
result[2*i] = (char)val;
result[2*i + 1] = (char) (val >> 16);
}
return new string(result);
}
static string ByteArrayToHexViaLookup(byte[] bytes) {
string[] hexStringTable = new string[] {
"00", "01", "02", "03", "04", "05", "06", "07", "08", "09", "0A", "0B", "0C", "0D", "0E", "0F",
"10", "11", "12", "13", "14", "15", "16", "17", "18", "19", "1A", "1B", "1C", "1D", "1E", "1F",
"20", "21", "22", "23", "24", "25", "26", "27", "28", "29", "2A", "2B", "2C", "2D", "2E", "2F",
"30", "31", "32", "33", "34", "35", "36", "37", "38", "39", "3A", "3B", "3C", "3D", "3E", "3F",
"40", "41", "42", "43", "44", "45", "46", "47", "48", "49", "4A", "4B", "4C", "4D", "4E", "4F",
"50", "51", "52", "53", "54", "55", "56", "57", "58", "59", "5A", "5B", "5C", "5D", "5E", "5F",
"60", "61", "62", "63", "64", "65", "66", "67", "68", "69", "6A", "6B", "6C", "6D", "6E", "6F",
"70", "71", "72", "73", "74", "75", "76", "77", "78", "79", "7A", "7B", "7C", "7D", "7E", "7F",
"80", "81", "82", "83", "84", "85", "86", "87", "88", "89", "8A", "8B", "8C", "8D", "8E", "8F",
"90", "91", "92", "93", "94", "95", "96", "97", "98", "99", "9A", "9B", "9C", "9D", "9E", "9F",
"A0", "A1", "A2", "A3", "A4", "A5", "A6", "A7", "A8", "A9", "AA", "AB", "AC", "AD", "AE", "AF",
"B0", "B1", "B2", "B3", "B4", "B5", "B6", "B7", "B8", "B9", "BA", "BB", "BC", "BD", "BE", "BF",
"C0", "C1", "C2", "C3", "C4", "C5", "C6", "C7", "C8", "C9", "CA", "CB", "CC", "CD", "CE", "CF",
"D0", "D1", "D2", "D3", "D4", "D5", "D6", "D7", "D8", "D9", "DA", "DB", "DC", "DD", "DE", "DF",
"E0", "E1", "E2", "E3", "E4", "E5", "E6", "E7", "E8", "E9", "EA", "EB", "EC", "ED", "EE", "EF",
"F0", "F1", "F2", "F3", "F4", "F5", "F6", "F7", "F8", "F9", "FA", "FB", "FC", "FD", "FE", "FF",
};
StringBuilder result = new StringBuilder(bytes.Length * 2);
foreach (byte b in bytes) {
result.Append(hexStringTable[b]);
}
return result.ToString();
}
Update (2010-01-13)
Added Waleed's answer to analysis. Quite fast.
Update (2011-10-05)
Added string.Concat Array.ConvertAll variant for completeness (requires .NET 4.0). On par with string.Join version.
Update (2012-02-05)
Test repo includes more variants such as StringBuilder.Append(b.ToString("X2")). None upset the results any. foreach is faster than {IEnumerable}.Aggregate, for instance, but BitConverter still wins.
Update (2012-04-03)
Added Mykroft's SoapHexBinary answer to analysis, which took over third place.
Update (2013-01-15)
Added CodesInChaos's byte manipulation answer, which took over first place (by a large margin on large blocks of text).
Update (2013-05-23)
Added Nathan Moinvaziri's lookup answer and the variant from Bri…