Dtypes don't work like they look at first glance. np.uint16 isn't a dtype object. It's just convertible to one. np.uint16 is a type object representing the type of array scalars of uint16 dtype.

x.dtype is an actual dtype object, and dtype objects implement == in a weird way that's non-transitive and inconsistent with hash. dtype == other is basically implemented as dtype == np.dtype(other) when other isn't already a dtype. You can see the details in the source.

Particularly, x.dtype compares equal to np.uint16, but it doesn't have the same hash, so the dict lookup doesn't find it.

You can convert a string to a 32-bit signed integer with the int function:

string = "1234"
i = int(string)  # i is a 32-bit integer

If the string does not represent an integer, you'll get a ValueError exception. Note, however, that if the string does represent an integer, but that integer does not fit into a 32-bit signed int, then you'll actually get an object of type long instead.

You can then convert it to other widths and signednesses with some simple math:

s8 = (i + 2**7) % 2**8 - 2**7      # convert to signed 8-bit
u8 = i % 2**8                      # convert to unsigned 8-bit
s16 = (i + 2**15) % 2**16 - 2**15  # convert to signed 16-bit
u16 = i % 2**16                    # convert to unsigned 16-bit
s32 = (i + 2**31) % 2**32 - 2**31  # convert to signed 32-bit
u32 = i % 2**32                    # convert to unsigned 32-bit
s64 = (i + 2**63) % 2**64 - 2**63  # convert to signed 64-bit
u64 = i % 2**64                    # convert to unsigned 64-bit

You can convert strings to floating point with the float function:

f = float("3.14159")

Python floats are what other languages refer to as double, i.e. they are 64-bits. There are no 32-bit floats in Python.

An easy way is to convert to uint8 and then use tobytes():

>>> bayer.astype(dtype=np.uint8).tobytes()
b'gbrg'

Another alternative:

>>> bytes(bayer.ravel().tolist())
b'gbrg'

Or even:

>>> bayer.tobytes()[::2]
b'gbrg'

I found a solution by NumPy.

uint16_list.**view**(np.float16)

The default dtype for integers in a Pandas Series is int64 -- a signed 64-bit integer.

In [82]: pd.Series([-2692]).dtype
Out[82]: dtype('int64')

If you use astype to convert the dtype to uint16 -- an unsigned 16-bit integer -- then int64 values which are outside the range of ints representable as uint16s get cast to uint16 values. For example, the negative int64 -2692 gets mapped to 62844 as a uint16:

In [80]: np.array([-2692], dtype='int64').astype('uint16')
Out[80]: array([62844], dtype=uint16)

Here is the range of ints representable as int64s:

In [83]: np.iinfo('int64')
Out[83]: iinfo(min=-9223372036854775808, max=9223372036854775807, dtype=int64)

And here is the range of ints representable as uint16s:

In [84]: np.iinfo('uint16')
Out[84]: iinfo(min=0, max=65535, dtype=uint16)

To debug problems like this it is useful to isolate a toy example which exhibits the problem. For example, if you run

for i in range(0,originalN):
    monthstoadd = all_treatments.iloc[i,emcolix].astype('uint16')
    if monthstoadd == 62844:
        print(all_treatments.iloc[i,emcolix])
        print(all_treatments.iloc[i,emcolix].dtype)
        break

then you would see the value of all_treatments.iloc[i,emcolix] before calling astype, and also the dtype. This would be a good starting point to discover the source of the problem.

Sounds like you need to use a different byte ordering (note 'h' is the format specifier for 16bit ints).

>>> import struct
>>> struct.unpack('h', b'\x30\x00') # default byte ordering
(48,)
>>> struct.unpack('h', b'\x00\x30') # you get the right result if you swap the bytes in the file
(12288,)
>>> struct.unpack('<h', b'\x30\x00') # little endian
(48,)
>>> struct.unpack('>h', b'\x30\x00') # big endian <- this is likely what you want
(12288,)

In numpy you can use something like

>>> import numpy
>>> numpy.dtype('>h')
dtype('>i2')

You need to use shorts or unsigned shorts:)

>>> import struct
>>> data_read
b'\x00\x00\x01\x00\x02\x00\x03\x00'
>>> struct.unpack("hhhh" , data_read)
(0, 1, 2, 3)
>>> struct.unpack("HHHH" , data_read)
(0, 1, 2, 3)

use int() to convert the string to an integer. Python doesn't have different fixed-width integers so you'll just get one type of thing out.

Then use struct to pack the integer into a fixed width:

res = struct.pack("=B",i) ## uint8_t
res = struct.pack("=b",i) ## int8_t

res = struct.pack("=H",i) ## uint16_t
res = struct.pack("=h",i) ## int16_t

res = struct.pack("=I",i) ## uint32_t
res = struct.pack("=i",i) ## int32_t

res = struct.pack("=Q",i) ## uint64_t
res = struct.pack("=q",i) ## int64_t

res = struct.pack("=f",i) ## float
res = struct.pack("=d",i) ## double

struct produces a byte-string containing the number in binary.

EDIT: From the comments it sounds like you just want to convert the string (of decimal digits) into an integer. Just use int() for that, however you won't get all the complicated overflow/underflow semantics of the specified types. You can't reproduce that in python, at least not without writing a whole lot of code.

I think if you want any more help you'll have to be more precise about what you want to achieve.

This should work:

import struct
dataStr = b''
for uint_16t in data:
    dataStr += struct.pack('>I', uint_16t)
print dataStr

Output based on your supplied list:

  {"idx":1430,"nvalue":0,"svalue":"-58.00"

Not sure why it is missing the closing } though...

Edit: To remove that weird whitespace you can do:

for i in dataStr:
    if ord(i) != 0:
        newDataStr += i
print newDataStr

You can convert using the binascii and struct modules from the standard library:

>>> import binascii
>>> import struct
>>> import sys

# Check our system's byte order
>>> sys.byteorder
'little'
>>> hx = '8a01'
# convert hex to bytes
>>> bs = binascii.unhexlify(hx)
>>> bs
b'\x8a\x01'
# struct module expects ints to be four bytes long, so pad to make up the length
>>> padded = bs + b'\x00\x00'
# Ask struct to unpack a little-endian unsigned int.
>>> i = struct.unpack('<I', padded)
>>> i
(394,)

Update

This question has been closed as a duplicate. The duplicate solution doesn't produce the required result:

>>> int('8a01', 16)
35329

However it works as expected if the order of bytes is reversed:

>>> int('018a', 16)
394

This is because the int builtin function assumes that the hexstring is ordered in the same way we order base 10 numbers on paper, that is the leftmost values are the most significant. The initial value 0x8a01 has the least significant values on the left, hence using int to convert from base 16 produces the wrong result.

However in Python3 we can still use int to produce a simpler solution, using int.from_bytes.

>>> hx = '8a01'
>>> bs = binascii.unhexlify(hx)
>>> bs
b'\x8a\x01'
>>> int.from_bytes(bs, byteorder=sys.byteorder)
394