Dtypes don't work like they look at first glance. np.uint16 isn't a dtype object. It's just convertible to one. np.uint16 is a type object representing the type of array scalars of uint16 dtype.

x.dtype is an actual dtype object, and dtype objects implement == in a weird way that's non-transitive and inconsistent with hash. dtype == other is basically implemented as dtype == np.dtype(other) when other isn't already a dtype. You can see the details in the source.

Particularly, x.dtype compares equal to np.uint16, but it doesn't have the same hash, so the dict lookup doesn't find it.

Your data corruption is caused by integer overflow.

Once a numpy integer goes outside the range of a datatype (0-255 for uint8), the value overflows and wraps around to 0 again.

For example:

np.uint16(255).astype(np.uint8) == 255
np.uint16(256).astype(np.uint8) == 0
np.uint16(258).astype(np.uint8) == 2
np.uint16(260).astype(np.uint8) == 4
np.uint16(511).astype(np.uint8) == 255
np.uint16(512).astype(np.uint8) == 0
np.uint16(514).astype(np.uint8) == 2
np.uint16(516).astype(np.uint8) == 4

A simple rescale will work, e.g.

data_8bit = ((data - data.min()) / (data.max() - data.min()) * 255).astype(np.uint8)

But you'll get a much nicer looking image if you rescale per band and even better with a per band percentile stretch similar to what most GIS software does, such as [2, 98] or [1, 99]. E.g.

import rasterio
import numpy as np

input_tiff = "test_uint16.tif"
output_tiff = "test_uint8.tif"

percentiles = [1, 99]

with rasterio.open(input_tiff) as src:
    profile = src.profile
    profile.update(dtype=rasterio.uint8)

    with rasterio.open(output_tiff, 'w', **profile) as dst:
        for i in range(src.count):
            band = i+1
            data = src.read(band, masked=True)

            # data.compressed() returns an array of non-masked pixel values
            pmin, pmax = np.percentile(data.compressed(), percentiles)

            # rescale 0-1
            data = (data - pmin) / (pmax - pmin)

            # Ensure data is >= pmin <= pmax
            data[data<0] = 0
            data[data>1] = 1

            # Make it 0-255 8bit unsigned.
            data_int8 = (data * 255).astype(np.uint8)

            dst.write(data_int8, band)

The default dtype for integers in a Pandas Series is int64 -- a signed 64-bit integer.

In [82]: pd.Series([-2692]).dtype
Out[82]: dtype('int64')

If you use astype to convert the dtype to uint16 -- an unsigned 16-bit integer -- then int64 values which are outside the range of ints representable as uint16s get cast to uint16 values. For example, the negative int64 -2692 gets mapped to 62844 as a uint16:

In [80]: np.array([-2692], dtype='int64').astype('uint16')
Out[80]: array([62844], dtype=uint16)

Here is the range of ints representable as int64s:

In [83]: np.iinfo('int64')
Out[83]: iinfo(min=-9223372036854775808, max=9223372036854775807, dtype=int64)

And here is the range of ints representable as uint16s:

In [84]: np.iinfo('uint16')
Out[84]: iinfo(min=0, max=65535, dtype=uint16)

To debug problems like this it is useful to isolate a toy example which exhibits the problem. For example, if you run

for i in range(0,originalN):
    monthstoadd = all_treatments.iloc[i,emcolix].astype('uint16')
    if monthstoadd == 62844:
        print(all_treatments.iloc[i,emcolix])
        print(all_treatments.iloc[i,emcolix].dtype)
        break

then you would see the value of all_treatments.iloc[i,emcolix] before calling astype, and also the dtype. This would be a good starting point to discover the source of the problem.

You want to use "H" (unsigned short) instead of "I" (unsigned int). In C, int can be 2 or 4 bytes depending on architecture and its usually 4. You could check someArray.itemsize to verify on your machine.

Probably every machine Python supports has 8 bits per byte, so a 16-bit integer uses 2 bytes. We want the size to be the same on every machine, so we look at the standard size column. The format for an unsigned integer with a standard size of 2 bytes or 16 bits is H.

For the standard size to be relevant, the unpack pattern must start with <, >, ! or = depending on endianness. ELF supports both little- and big-endian values depending on the byte at offset 0x05 of the file, so your pattern would start with either < or > depending on what the endianness of the file.

• If the byte at offset 0x05 is 1, it's a little-endian file, so your pattern must start with <.
  

  LE uint16_t 0x3456 = 13398 is b'\x56\x34'

  >>> x = b'\x56\x34'
  >>> struct.unpack('<H', x)
  (13398,)
  

• If the byte at offset 0x05 is 2, it's a big-endian file, so your pattern must start with >.
  

  BE uint16_t 0x3456 = 13398 is b'\x34\x56'

  >>> x = b'\x34\x56'
  >>> struct.unpack('>H', x)
  (13398,)
  

How about ctypes.c_int16? Also supports ctypes.c_uint16 if you are looking for that.

Documentation:
https://docs.python.org/2/library/ctypes.html#ctypes.c_int16 and
https://docs.python.org/2/library/ctypes.html#ctypes.c_uint16

Sounds like you need to use a different byte ordering (note 'h' is the format specifier for 16bit ints).

>>> import struct
>>> struct.unpack('h', b'\x30\x00') # default byte ordering
(48,)
>>> struct.unpack('h', b'\x00\x30') # you get the right result if you swap the bytes in the file
(12288,)
>>> struct.unpack('<h', b'\x30\x00') # little endian
(48,)
>>> struct.unpack('>h', b'\x30\x00') # big endian <- this is likely what you want
(12288,)

In numpy you can use something like

>>> import numpy
>>> numpy.dtype('>h')
dtype('>i2')