Try this:

import pandas as pd

file = open("your.csv", "r")

data = pd.read_csv(file, sep = ",")

gender = {'male': 1,'female': 0}

data.Gender = [gender[item] for item in data.Gender]
print(data)

Or

data.Gender[data.Gender == 'male'] = 1
data.Gender[data.Gender == 'female'] = 0
print(data)

In [6]: (df['number'] < 15).astype(int)
Out[6]:
0    1
1    0
2    1
3    0
4    0
5    1
6    0
7    1
8    0
Name: number, dtype: int32

In [7]: df['binary'] = (df['number'] < 15).astype(int)

In [8]: df
Out[8]:
   number  binary
0      12       1
1      89       0
2      12       1
3      56       0
4      62       0
5       2       1
6     657       0
7       5       1
8      73       0

If performance is important, use numpy with this solution:

d = df['Col_B'].values
m = 2
df[['Col_C','Col_D']]  = pd.DataFrame((((d[:,None] & (1 << np.arange(m)))) > 0).astype(int))
print (df)
  Col_A  Col_B  Col_C  Col_D
0     a      1      1      0
1     b      2      0      1
2     c      0      0      0

Performance (about 1000 times faster):

df = pd.DataFrame([['a', 1], ['b', 2], ['c', 0]], columns=["Col_A", "Col_B"])


df = pd.concat([df] * 1000, ignore_index=True)

In [162]: %%timeit
     ...: df[['Col_C','Col_D']] = df['Col_B'].apply(lambda x: pd.Series(list(bin(x)[2:].zfill(2))))
     ...: 
609 ms ± 14.5 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

In [163]: %%timeit
     ...: d = df['Col_B'].values
     ...: m = 2
     ...: df[['Col_C','Col_D']]  = pd.DataFrame((((d[:,None] & (1 << np.arange(m)))) > 0).astype(int))
     ...: 
618 µs ± 26.2 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

Well, one way i like to handle this problem (which is a common problem, at least in daily job life) is to convert each possibility in a column with binary value. Let me elaborate a bit. Let's say you have your column animals with 3 possibilities : dog, cat, and horse. You explode your column in 3 differents columns : colDog, colCat and colHorse. And you fill your new columns based on the value of the column animals. For example : if you have dog in the first row, you put 1 in the column colDog, etc.

The problem with handling categorical data with numerical value instead of binary is that you create a hierarchical order between your values. If dog is 1, cat is 2 and horse is 3, then horse will have more impact than cat and dog. Or i think you just want to represent your categories.

df.round

>>> df.round()

np.round

>>> np.round(df)

astype

>>> df.ge(0.5).astype(int)

All which yield

     0    1    2
0  0.0  0.0  1.0
1  0.0  1.0  1.0
2  1.0  0.0  1.0
3  0.0  1.0  0.0

Note: round works here because it automatically sets the threshold for .5 between two integers. For custom thresholds, use the 3rd solution

You can also work with numpy, much faster than pandas.

Edit: faster numpy using view Couple of tricks here:

• Work only with the column of interest
• Convert the underlaying array to uint16, to ensure compatibility with any integer input
• Swapbytes to have a proper H,L order (at least on my architecture)
• Split H,L without actually moving any data with view
• Run unpackbits and reshape accordingly

My machine requires a byteswap to have the bytes of the uint16 in the proper place. Note that this aproach requires to have the data as int16/uint16, while the other one would work for int64 as well.

import pandas as pd
import numpy as np

df = pd.DataFrame({'EVENT_ID': [ 4162, 4161, 4160, 4159,4158, 4157, 4156, 4155, 4154]}, dtype='uint16')

zz=np.unpackbits(df.EVENT_ID.values.astype('uint16').byteswap().view('uint8')).reshape(-1,16)
df3 = pd.concat([df,pd.DataFrame(zz)],axis=1)

print(f"{df3 =}")

df3 =   EVENT_ID  0  1  2  3  4  5  6  7  8  9  10  11  12  13  14  15
0      4162  0  0  0  1  0  0  0  0  0  1   0   0   0   0   1   0
1      4161  0  0  0  1  0  0  0  0  0  1   0   0   0   0   0   1
2      4160  0  0  0  1  0  0  0  0  0  1   0   0   0   0   0   0
3      4159  0  0  0  1  0  0  0  0  0  0   1   1   1   1   1   1
4      4158  0  0  0  1  0  0  0  0  0  0   1   1   1   1   1   0
5      4157  0  0  0  1  0  0  0  0  0  0   1   1   1   1   0   1
6      4156  0  0  0  1  0  0  0  0  0  0   1   1   1   1   0   0
7      4155  0  0  0  1  0  0  0  0  0  0   1   1   1   0   1   1
8      4154  0  0  0  1  0  0  0  0  0  0   1   1   1   0   1   0

older proposed method:

lh = np.unpackbits((df.values & 0xFF).astype('uint8')).reshape(-1,8)
uh = np.unpackbits((df.values >> 8).astype('uint8')).reshape(-1,8)

df2 = pd.concat([df, pd.DataFrame(np.concatenate([uh,lh],axis=1),index=df.index)],axis=1)

Benchmark: numpy is orders of magnitude faster than pandas" For 1M points:

• numpy view: 35ms for 1million uint64 points
• numpy low/high: 50ms
• pandas list bin: 1.78s
• pandas apply format + list: 1.97s
• pandas apply lambda: 6.08s

df = pd.DataFrame({'EVENT_ID': (np.random.random(int(1e6))*65000).astype('uint16')})

pandas apply format list

In [13]: %timeit df2 = df.join(pd.DataFrame(df['EVENT_ID'].apply('{0:b}'.format).apply(list).tolist()))
1.97 s ± 42.5 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

pandas list bin

In [10]: %%timeit
    ...: binary_values = pd.DataFrame([list(bin(x)[2:]) for x in df['EVENT_ID']])
    ...: df2 = df.join(binary_values)
    ...:
    ...:
1.78 s ± 53.9 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

pandas3 apply lambda

In [5]: %%timeit
   ...: for i in range(16):
   ...:     df[f"bit{i}"] = df["EVENT_ID"].apply(lambda x: x & 1 << i).astype(bool).astype(int)
   ...:
6.08 s ± 65.8 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

numpy

In [14]: %%timeit
    ...: lh = np.unpackbits((df.values & 0xFF).astype('uint8')).reshape(-1,8)
    ...: uh = np.unpackbits((df.values >> 8).astype('uint8')).reshape(-1,8)
    ...: df3=pd.concat([df, pd.DataFrame(np.concatenate([uh,lh],axis=1),index=df.index)],axis=1)
    ...:
    ...:
49.9 ms ± 232 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)

you can use map -

a = {'education' : 1,'cinema' : 0}
train_docs['Class'] = train_docs['Class'].map(a)

As the contents of the columns are integers rather than string, the int(i, 2) function cannot be used directly on the integer i, or else it will throw an error. E.g.

int(11, 2)
TypeError: int() can't convert non-string with explicit base

You have to firstly convert the columns to string and then convert string to decimal values. For example, assuming column 'val', use the code:

df['val'] = df['val'].astype(str).map(lambda x: int(x, 2))

1. astype(str) convert the column type to string, then

2. use the int() function within map() to apply to each element in the column

Test codes and output:

data = { 'val': [11, 101, 1001, 1101]}
df = pd.DataFrame(data)
print(df)
                
Output:  (before conversion)    

                 val
            0     11
            1    101
            2   1001
            3   1101
        
df['val'] = df['val'].astype(str).map(lambda x: int(x, 2))
print(df)
        
Output:  (after conversion)
        
            val
        0     3
        1     5
        2     9
        3    13

You can use pandas.get_dummies and concatenate the result with the original data frame:

pd.concat([df, pd.get_dummies(df.ph_level).rename(columns = "{}_binary".format)], axis = 1)

We can do this by a chain of actions:

1. first we convert the hexadecimal number to an int with .apply(int, base=16);
2. next we convert this to binary data, with .apply(bin);
3. next we chunk off the first two characters with .str[2:];
4. then we obtain the last three characters with .str[-3:]; and
5. finally we again interpret these as ints, with .apply(int, base=2).

So:

>>> df.Data.apply(int, base=16).apply(bin).str[2:].str[-3:].apply(int, base=2)
0    2
1    3
2    3
3    7
4    7
5    0
6    3
Name: Data, dtype: int64

We can however use another strategy here:

1. we first convert the hexadecimal number to an int; and
2. then we apply a bitwise and with 0b111.

for example:

>>> df.Data.apply(int, base=16) & 0b111
0    2
1    3
2    3
3    7
4    7
5    0
6    3
Name: Data, dtype: int64

The second attempt is not only simpler, but faster as well, approximately by 66%:

>>> timeit(first_strategy, number=10000)
6.962630775000434
>>> timeit(second_strategy, number=10000)
2.330652763019316

for a dataframe that repeats the sample data 100 times, we get:

>>> timeit(first_strategy, number=10000)
17.603060900000855
>>> timeit(second_strategy, number=10000)
5.901462858979357

this is again 66% faster.