from bitarray import bitarray
d=bitarray('0'*30)
d[5]=1
i = 0
for bit in d:
i = (i << 1) | bit
print i
output: 16777216.
Answer from Serenity on Stack Overflowfrom bitarray import bitarray
d=bitarray('0'*30)
d[5]=1
i = 0
for bit in d:
i = (i << 1) | bit
print i
output: 16777216.
To convert a bitarray to its integer form you can use the struct module:
Code:
from bitarray import bitarray
import struct
d = bitarray('0' * 30, endian='little')
d[5] = 1
print(struct.unpack("<L", d)[0])
d[6] = 1
print(struct.unpack("<L", d)[0])
Outputs:
32
96
arrays - Bits list to integer in Python - Stack Overflow
python - Converting a bytearray into an integer - Code Review Stack Exchange
python - Integer to bitfield as a list - Stack Overflow
Convert python byte "array" to int "array - Stack Overflow
You can use bitshifting:
out = 0
for bit in bitlist:
out = (out << 1) | bit
This easily beats the "int cast" method proposed by A. R. S., or the modified cast with lookup proposed by Steven Rumbalski:
>>> def intcaststr(bitlist):
... return int("".join(str(i) for i in bitlist), 2)
...
>>> def intcastlookup(bitlist):
... return int(''.join('01'[i] for i in bitlist), 2)
...
>>> def shifting(bitlist):
... out = 0
... for bit in bitlist:
... out = (out << 1) | bit
... return out
...
>>> timeit.timeit('convert([1,0,0,0,0,0,0,0])', 'from __main__ import intcaststr as convert', number=100000)
0.5659139156341553
>>> timeit.timeit('convert([1,0,0,0,0,0,0,0])', 'from __main__ import intcastlookup as convert', number=100000)
0.4642159938812256
>>> timeit.timeit('convert([1,0,0,0,0,0,0,0])', 'from __main__ import shifting as convert', number=100000)
0.1406559944152832
...or using the bitstring module
>>> from bitstring import BitArray
>>> bitlist=[1,0,0,0,0,0,0,0]
>>> b = BitArray(bitlist)
>>> b.uint
128
How about this:
def bitfield(n):
return [int(digit) for digit in bin(n)[2:]] # [2:] to chop off the "0b" part
This gives you
>>> bitfield(123)
[1, 1, 1, 1, 0, 1, 1]
>>> bitfield(255)
[1, 1, 1, 1, 1, 1, 1, 1]
>>> bitfield(1234567)
[1, 0, 0, 1, 0, 1, 1, 0, 1, 0, 1, 1, 0, 1, 0, 0, 0, 0, 1, 1, 1]
This only works for positive integers, though.
EDIT:
Conversion to int using int() is a bit overkill here. This is a lot faster:
def bitfield(n):
return [1 if digit=='1' else 0 for digit in bin(n)[2:]]
See the timings:
>>> import timeit
>>> timeit.timeit("[int(digit) for digit in bin(123)[2:]]")
7.895014818543946
>>> timeit.timeit("[123 >> i & 1 for i in range(7,-1,-1)]")
2.966295244250407
>>> timeit.timeit("[1 if digit=='1' else 0 for digit in bin(123)[2:]]")
1.7918431924733795
This doesn't use bin:
b = [n >> i & 1 for i in range(7,-1,-1)]
and this is how to handle any integer this way:
b = [n >> i & 1 for i in range(n.bit_length() - 1,-1,-1)]
See bit_length.
If you want index 0 of the list to correspond to the lsb of the int, change the range order, i.e.
b = [n >> i & 1 for i in range(0, n.bit_length()-1)]
Note also that using n.bit_length() can be a point of failure if you're trying to represent fixed length binary values. It returns the minimum number of bits to represent n.
Note: this solution only works for Python 3. For Python 2 solution please see Scott Hunter's answer.
You could do this using a list comprehension:
In [1]: frame = b'\xff\xff\xff\xffXabccc'
In [2]: int_values = [x for x in frame]
In [3]: print(int_values)
[255, 255, 255, 255, 88, 97, 98, 99, 99, 99]
To confirm that these are indeed stored as integer values which you can work with:
In [4]: print([type(x) for x in int_values])
[<class 'int'>, <class 'int'>, <class 'int'>, <class 'int'>, <class 'int'>,
<class 'int'>, <class 'int'>, <class 'int'>, <class 'int'>, <class 'int'>]
The simplest, and most obvious way to do it, is to pass the frame object to the list constructor. No list comprehension is needed for this:
>>> frame = b'\xff\xff\xff\xffXabccc'
>>> list(frame)
[255, 255, 255, 255, 88, 97, 98, 99, 99, 99]
Python doesn't traditionally have much use for "numbers in big-endian C layout" that are too big for C. (If you're dealing with 2-byte, 4-byte, or 8-byte numbers, then struct.unpack is the answer.)
But enough people got sick of there not being one obvious way to do this that Python 3.2 added a method int.from_bytes that does exactly what you want:
int.from_bytes(b, byteorder='big', signed=False)
Unfortunately, if you're using an older version of Python, you don't have this. So, what options do you have? (Besides the obvious one: update to 3.2, or, better, 3.4…)
First, there's your code. I think binascii.hexlify is a better way to spell it than .encode('hex'), because "encode" has always seemed a little weird for a method on byte strings (as opposed to Unicode strings), and it's in fact been banished in Python 3. But otherwise, it seems pretty readable and obvious to me. And it should be pretty fast—yes, it has to create an intermediate string, but it's doing all the looping and arithmetic in C (at least in CPython), which is generally an order of magnitude or two faster than in Python. Unless your bytearray is so big that allocating the string will itself be costly, I wouldn't worry about performance here.
Alternatively, you could do it in a loop. But that's going to be more verbose and, at least in CPython, a lot slower.
You could try to eliminate the explicit loop for an implicit one, but the obvious function to do that is reduce, which is considered un-Pythonic by part of the community—and of course it's going to require calling a function for each byte.
You could unroll the loop or reduce by breaking it into chunks of 8 bytes and looping over struct.unpack_from, or by just doing a big struct.unpack('Q'*len(b)//8 + 'B' * len(b)%8) and looping over that, but that makes it a lot less readable and probably not that much faster.
You could use NumPy… but if you're going bigger than either 64 or maybe 128 bits, it's going to end up converting everything to Python objects anyway.
So, I think your answer is the best option.
Here are some timings comparing it to the most obvious manual conversion:
import binascii
import functools
import numpy as np
def hexint(b):
return int(binascii.hexlify(b), 16)
def loop1(b):
def f(x, y): return (x<<8)|y
return functools.reduce(f, b, 0)
def loop2(b):
x = 0
for c in b:
x <<= 8
x |= c
return x
def numpily(b):
n = np.array(list(b))
p = 1 << np.arange(len(b)-1, -1, -1, dtype=object)
return np.sum(n * p)
In [226]: b = bytearray(range(256))
In [227]: %timeit hexint(b)
1000000 loops, best of 3: 1.8 µs per loop
In [228]: %timeit loop1(b)
10000 loops, best of 3: 57.7 µs per loop
In [229]: %timeit loop2(b)
10000 loops, best of 3: 46.4 µs per loop
In [283]: %timeit numpily(b)
10000 loops, best of 3: 88.5 µs per loop
For comparison in Python 3.4:
In [17]: %timeit hexint(b)
1000000 loops, best of 3: 1.69 µs per loop
In [17]: %timeit int.from_bytes(b, byteorder='big', signed=False)
1000000 loops, best of 3: 1.42 µs per loop
So, your method is still pretty fast…
Function struct.unpack(...) does what you need.
Assuming you're on at least 3.2, there's a built in for this:
int.from_bytes(bytes,byteorder, *,signed=False)...
The argument
bytesmust either be a bytes-like object or an iterable producing bytes.The
byteorderargument determines the byte order used to represent the integer. Ifbyteorderis"big", the most significant byte is at the beginning of the byte array. Ifbyteorderis"little", the most significant byte is at the end of the byte array. To request the native byte order of the host system, usesys.byteorderas the byte order value.The
signedargument indicates whether two’s complement is used to represent the integer.
## Examples:
int.from_bytes(b'\x00\x01', "big") # 1
int.from_bytes(b'\x00\x01', "little") # 256
int.from_bytes(b'\x00\x10', byteorder='little') # 4096
int.from_bytes(b'\xfc\x00', byteorder='big', signed=True) #-1024
Lists of bytes are subscriptable (at least in Python 3.6). This way you can retrieve the decimal value of each byte individually.
>>> intlist = [64, 4, 26, 163, 255]
>>> bytelist = bytes(intlist) # b'@\x04\x1a\xa3\xff'
>>> for b in bytelist:
... print(b) # 64 4 26 163 255
>>> [b for b in bytelist] # [64, 4, 26, 163, 255]
>>> bytelist[2] # 26
Using numpy:
>>> import numpy as np
>>> a = np.frombuffer(b'this is my test data', dtype=np.int32)
>>> a
array([1936287860, 544434464, 1948285293, 544502629, 1635017060], dtype=int32)
>>> a.tobytes()
b'this is my test data'
Use '<i4' or similar as dtype for endianness that's portable between machines.
I'm assuming that you can keep your initial data as bytes rather than unicode, because you really should try hard to do that.
As long as your string is an integer multiple of 4, you can use NumPy in a very efficient way:
import numpy as np
data = np.fromstring(textdata, dtype='>u4')
# array([1952999795, 543781664, 1836654708, 1702065184, 1684108385])
'>u4' means 'big-endian unsigned 4-byte integer'.
Edit: If you use NumPy >= 1.14, then np.fromstring is deprecated, and the right way to process your text is by calling np.frombuffer(textdata.encode(), dtype='>u4').