Generally your idea of trying to apply astype to each column is fine.

In [590]: X[:,0].astype(int)
Out[590]: array([1, 2, 3, 4, 5])

But you have to collect the results in a separate list. You can't just put them back in X. That list can then be concatenated.

In [601]: numlist=[]; obj_ind=[]

In [602]: for ind in range(X.shape[1]):
   .....:     try:
   .....:         x = X[:,ind].astype(np.float32)
   .....:         numlist.append(x)
   .....:     except:
   .....:         obj_ind.append(ind)

In [603]: numlist
Out[603]: [array([ 3.,  4.,  5.,  6.,  7.], dtype=float32)]

In [604]: np.column_stack(numlist)
Out[604]: 
array([[ 3.],
       [ 4.],
       [ 5.],
       [ 6.],
       [ 7.]], dtype=float32)

In [606]: obj_ind
Out[606]: [1]

X is a numpy array with dtype object:

In [582]: X
Out[582]: 
array([[1, 'A'],
       [2, 'A'],
       [3, 'C'],
       [4, 'D'],
       [5, 'B']], dtype=object)

You could use the same conversion logic to create a structured array with a mix of int and object fields.

In [616]: ytype=[]

In [617]: for ind in range(X.shape[1]):
    try:                        
        x = X[:,ind].astype(np.float32)
        ytype.append('i4')
    except:
        ytype.append('O')       

In [618]: ytype
Out[618]: ['i4', 'O']

In [620]: Y=np.zeros(X.shape[0],dtype=','.join(ytype))

In [621]: for i in range(X.shape[1]):
    Y[Y.dtype.names[i]] = X[:,i]

In [622]: Y
Out[622]: 
array([(3, 'A'), (4, 'A'), (5, 'C'), (6, 'D'), (7, 'B')], 
      dtype=[('f0', '<i4'), ('f1', 'O')])

Y['f0'] gives the the numeric field.

The IEEE 754-2008 16-bit base 2 format, aka binary16, doesn't give you a lot of precision. What do you expect from 16 bits? :) 1 bit is the sign bit, 5 bits are used for the exponent, and that leaves 10 bits to store the normalised 11 bit mantissa, so anything > 2**11 == 2048 has to be quantized.

According to Wikipedia, integers between 4097 and 8192 round to a multiple of 4, and integers between 8193 and 16384 round to a multiple of 8.