The default dtype for integers in a Pandas Series is int64 -- a signed 64-bit integer.

In [82]: pd.Series([-2692]).dtype
Out[82]: dtype('int64')

If you use astype to convert the dtype to uint16 -- an unsigned 16-bit integer -- then int64 values which are outside the range of ints representable as uint16s get cast to uint16 values. For example, the negative int64 -2692 gets mapped to 62844 as a uint16:

In [80]: np.array([-2692], dtype='int64').astype('uint16')
Out[80]: array([62844], dtype=uint16)

Here is the range of ints representable as int64s:

In [83]: np.iinfo('int64')
Out[83]: iinfo(min=-9223372036854775808, max=9223372036854775807, dtype=int64)

And here is the range of ints representable as uint16s:

In [84]: np.iinfo('uint16')
Out[84]: iinfo(min=0, max=65535, dtype=uint16)

To debug problems like this it is useful to isolate a toy example which exhibits the problem. For example, if you run

for i in range(0,originalN):
    monthstoadd = all_treatments.iloc[i,emcolix].astype('uint16')
    if monthstoadd == 62844:
        print(all_treatments.iloc[i,emcolix])
        print(all_treatments.iloc[i,emcolix].dtype)
        break

then you would see the value of all_treatments.iloc[i,emcolix] before calling astype, and also the dtype. This would be a good starting point to discover the source of the problem.

Your data corruption is caused by integer overflow.

Once a numpy integer goes outside the range of a datatype (0-255 for uint8), the value overflows and wraps around to 0 again.

For example:

np.uint16(255).astype(np.uint8) == 255
np.uint16(256).astype(np.uint8) == 0
np.uint16(258).astype(np.uint8) == 2
np.uint16(260).astype(np.uint8) == 4
np.uint16(511).astype(np.uint8) == 255
np.uint16(512).astype(np.uint8) == 0
np.uint16(514).astype(np.uint8) == 2
np.uint16(516).astype(np.uint8) == 4

A simple rescale will work, e.g.

data_8bit = ((data - data.min()) / (data.max() - data.min()) * 255).astype(np.uint8)

But you'll get a much nicer looking image if you rescale per band and even better with a per band percentile stretch similar to what most GIS software does, such as [2, 98] or [1, 99]. E.g.

import rasterio
import numpy as np

input_tiff = "test_uint16.tif"
output_tiff = "test_uint8.tif"

percentiles = [1, 99]

with rasterio.open(input_tiff) as src:
    profile = src.profile
    profile.update(dtype=rasterio.uint8)

    with rasterio.open(output_tiff, 'w', **profile) as dst:
        for i in range(src.count):
            band = i+1
            data = src.read(band, masked=True)

            # data.compressed() returns an array of non-masked pixel values
            pmin, pmax = np.percentile(data.compressed(), percentiles)

            # rescale 0-1
            data = (data - pmin) / (pmax - pmin)

            # Ensure data is >= pmin <= pmax
            data[data<0] = 0
            data[data>1] = 1

            # Make it 0-255 8bit unsigned.
            data_int8 = (data * 255).astype(np.uint8)

            dst.write(data_int8, band)

All of these do different things.

np.uint8 considers only the lowest byte of your number. It's like doing value & 0xff.

>>> img = np.array([2000, -150, 11], dtype=np.int16)
>>> np.uint8(img)
array([208, 106,  11], dtype=uint8)

cv2.normalize with the cv2.NORM_MINMAX norm type normalises your values according to the normalisation function

img_new = (img - img.min()) * ((max_new - min_new) / (img.max() - img.min())) + min_new

It effectively changes one range to another and all the values in the between are scaled accordingly. By definition the original min/max values become the targetted min/max values.

>>> cv2.normalize(img, out, 0, 255, cv2.NORM_MINMAX)
array([255,   0,  19], dtype=int16)

uint8 in Matlab simply saturates your values. Everything above 255 becomes 255 and everything below 0 becomes 0.

>> uint8([2000 -150 11])

ans =

  255    0   11

If you want to replicate Matlab's functionality, you can do

>>> img[img > 255] = 255
>>> img[img < 0] = 0

Which one you want to use depends on what you're trying to do. If your int16 covers the range of your pixel values and you want to rescale those to uint8, then cv2.normalize is the answer.

Use:

integer_array = (img5.astype(np.float32)/255.0*65535.0-32768.0).astype(np.int16)

That will scale properly your 0,255 range (uint8) to the -32768,32767 (int16) range.

You could discard the bottom 16 bits:

n=(array_int32>>16).astype(np.int16)                          

which will give you this:

array([ 485, 1054, 2531], dtype=int16

You can use numpy.clip() before converting:

import numpy as np

# Example array
array_float64 = np.array([-1, 0, 100000, 65535, 65536], dtype=np.float64)


# Clip values
necessary_type = np.uint16
min_value = np.iinfo(necessary_type).min # 0
max_value = np.iinfo(necessary_type).max # 65535
array_clipped = np.clip(array_float64, min_value, max_value)


# Convert to uint16
array_uint16 = array_clipped.astype(necessary_type)
array_uint16
# array([    0,     0, 65535, 65535, 65535], dtype=uint16)

This ensures values above 65535 are set to 65535 and values below 0 are set to zero

Here is a doc and here is a related question

I maybe got it.

list = [3,103,3,103]
nlist = np.asarray(list, dtype='<B')
nlist = nlist.view(np.uint16)
print nlist, type(nlist[0])

output:

[26371 26371] <type 'numpy.uint16'>

NumPy functions are designed to have consistent return dtypes, based only on the dtypes of their arguments. This is an essential feature for avoiding bugs based on strange input values, and lets you accelerate numpy functions with tools like Numba.

Not every signed int can be safely cast into a unsigned int of the same size (e.g., 2 ** 63 is a valid 64-bit unsigned int but not a valid signed int). Likewise, not every signed int can be cast into an unsigned int (consider negative numbers). Hence, NumPy uses the next largest size dtype that can hold both values. 64-bits are the largest size integers supported by numpy, so it uses float64 instead. In contrast, as shown in the other comment, uint16 + int16 can fit in int32.

In contrast, np.arraycoerces dtypes of input arrays. This means it always succeeds.... even if the values cannot be safely represented:

In [31]: np.array(np.arange(2 ** 40, 2 ** 40 + 3), dtype=np.int16)
Out[31]: array([0, 1, 2], dtype=int16)

Use ctypes.c_ushort:

>>> import ctypes
>>> word.insert(0, ctypes.c_ushort(0x19c6acc6).value)
>>> word
array('H', [44230])

If NumPy is available then:

>>> numpy.ushort(0x19c6acc6)
44230

You can use cv2.convertScaleAbs for this problem. See the Documentation.

Check out the command terminal demo below :

>>> img = np.empty((100,100,1),dtype = np.uint16)
>>> image = cv2.cvtColor(img,cv2.COLOR_GRAY2BGR)

>>> cvuint8 = cv2.convertScaleAbs(image)

>>> cvuint8.dtype
dtype('uint8')

Hope it helps!!!

you can achieve your goal by using np.astype()

EXAMPLE:

import numpy as np
import cv2
        
image1 = cv2.imread("82.png",0)
image_uint16=image1.astype("uint16")

Dtypes don't work like they look at first glance. np.uint16 isn't a dtype object. It's just convertible to one. np.uint16 is a type object representing the type of array scalars of uint16 dtype.

x.dtype is an actual dtype object, and dtype objects implement == in a weird way that's non-transitive and inconsistent with hash. dtype == other is basically implemented as dtype == np.dtype(other) when other isn't already a dtype. You can see the details in the source.

Particularly, x.dtype compares equal to np.uint16, but it doesn't have the same hash, so the dict lookup doesn't find it.