df.round

>>> df.round()

np.round

>>> np.round(df)

astype

>>> df.ge(0.5).astype(int)

All which yield

     0    1    2
0  0.0  0.0  1.0
1  0.0  1.0  1.0
2  1.0  0.0  1.0
3  0.0  1.0  0.0

---

Note: round works here because it automatically sets the threshold for .5 between two integers. For custom thresholds, use the 3rd solution

the next generation of pandas will use Arrow. you can use it today as well https://arrow.apache.org/docs/python/pandas.html

Try this:

import pandas as pd

file = open("your.csv", "r")

data = pd.read_csv(file, sep = ",")

gender = {'male': 1,'female': 0}

data.Gender = [gender[item] for item in data.Gender]
print(data)

Or

data.Gender[data.Gender == 'male'] = 1
data.Gender[data.Gender == 'female'] = 0
print(data)

A Panda's DataFrame is not an excel spreadsheet on it's own. You have to first create the excel file, turn it into binary and then you can store it in your Binary field `file`. However, if you want to edit an already existing excel spreadsheet on your computer you can simply do this: import pandas as pd # Create a Pandas dataframe from the data. df = pd . DataFrame ({ 'Data' : [ 10 , 20 , 30 , 20 , 15 , 30 , 45 ]}) # Create a Pandas Excel writer using XlsxWriter as the engine. write

Hera are alternatives for binary:

crm['Reconciled'] = (crm['Reconciled'] > 0).astype(int)
crm['Reconciled'] = (crm['Reconciled'] > 0).view('i1')
crm['Reconciled'] = np.where(crm['Reconciled'] > 0, 1, 0)

If performance is important, use numpy with this solution:

d = df['Col_B'].values
m = 2
df[['Col_C','Col_D']]  = pd.DataFrame((((d[:,None] & (1 << np.arange(m)))) > 0).astype(int))
print (df)
  Col_A  Col_B  Col_C  Col_D
0     a      1      1      0
1     b      2      0      1
2     c      0      0      0

Performance (about 1000 times faster):

df = pd.DataFrame([['a', 1], ['b', 2], ['c', 0]], columns=["Col_A", "Col_B"])


df = pd.concat([df] * 1000, ignore_index=True)

In [162]: %%timeit
     ...: df[['Col_C','Col_D']] = df['Col_B'].apply(lambda x: pd.Series(list(bin(x)[2:].zfill(2))))
     ...: 
609 ms ± 14.5 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

In [163]: %%timeit
     ...: d = df['Col_B'].values
     ...: m = 2
     ...: df[['Col_C','Col_D']]  = pd.DataFrame((((d[:,None] & (1 << np.arange(m)))) > 0).astype(int))
     ...: 
618 µs ± 26.2 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

You mean "one-hot" encoding?

Say you have the following dataset:

import pandas as pd
df = pd.DataFrame([
            ['green', 1, 10.1, 0], 
            ['red', 2, 13.5, 1], 
            ['blue', 3, 15.3, 0]])

df.columns = ['color', 'size', 'prize', 'class label']
df

Now, you have multiple options ...

A) The Tedious Approach

color_mapping = {
           'green': (0,0,1),
           'red': (0,1,0),
           'blue': (1,0,0)}

df['color'] = df['color'].map(color_mapping)
df

import numpy as np
y = df['class label'].values
X = df.iloc[:, :-1].values
X = np.apply_along_axis(func1d= lambda x: np.array(list(x[0]) + list(x[1:])), axis=1, arr=X)

print('Class labels:', y)
print('\nFeatures:\n', X)

Yielding:

Class labels: [0 1 0]

Features:
 [[  0.    0.    1.    1.   10.1]
 [  0.    1.    0.    2.   13.5]
 [  1.    0.    0.    3.   15.3]]

B) Scikit-learn's DictVectorizer

from sklearn.feature_extraction import DictVectorizer
dvec = DictVectorizer(sparse=False)

X = dvec.fit_transform(df.transpose().to_dict().values())
X

Yielding:

array([[  0. ,   0. ,   1. ,   0. ,  10.1,   1. ],
       [  1. ,   0. ,   0. ,   1. ,  13.5,   2. ],
       [  0. ,   1. ,   0. ,   0. ,  15.3,   3. ]])

C) Pandas' get_dummies

pd.get_dummies(df)

Pandas now offers a wide variety of formats:

Format Type Data Description     Reader         Writer
text        CSV                  read_csv       to_csv
text        JSON                 read_json      to_json
text        HTML                 read_html      to_html
text        Local clipboard      read_clipboard to_clipboard
binary      MS Excel             read_excel     to_excel
binary      HDF5 Format          read_hdf       to_hdf
binary      Feather Format       read_feather   to_feather
binary      Parquet Format       read_parquet   to_parquet
binary      Msgpack              read_msgpack   to_msgpack
binary      Stata                read_stata     to_stata
binary      SAS                  read_sas    
binary      Python Pickle Format read_pickle    to_pickle
SQL         SQL                  read_sql       to_sql
SQL         Google Big Query     read_gbq       to_gbq

For small to medium sized files, I prefer CSV, as properly-formatted CSV can store arbitrary string data, is human readable, and is as dirt-simple as any format can be while achieving the previous two goals.

Unfortunately, for more complex problems, the choice is harder.

If I were on Amazon AWS, I would consider using parquet. However, I do not have any experience with this format.

I can no longer favor the pickle format. Although the pickle format claims long term stability, it allows arbitrary code execution. And even if no code is actually stored in the pickle, ALL pickles execute code just to unpickle them, which is what lead the folks at huggingface to recommend first pickle-tools for scanning pickles and then safetensors as an alternative to pickles. But for saving and loading pandas tables, safetensors is not an option, as they are designed for large multidimensional arrays of floating-point values ("tensors") not for tabular data.

I do not recommend using tofile(). tofile() is best for quick file storage where you do not expect the file to be used on a different machine where the data may have a different endianness (big-/little-endian).

I no longer favor the HDF5 format. It has serious risks for long-term archival since it is fairly complex. It has a 150 page specification, and only one 300,000 line C implementation.

If you have advice on stable, secure, binary formats for saving your own pandas data, please share them! In the meantime, I think CSV for small data and zipped CSVs to save disk space or network bandwidth when needed may be the best way to go. I personally try to avoid any pandas format that requires more than plain-text to store, if possible.

not sure how scalable this is, but here's one idea:

df = pd.DataFrame([["0,1", "10", "10,300"], ["1", "5", "300,0"], ["10,1,0", "", "300,10"]], columns = ["x", "y", "z"])

bin_dict_x = {'0': 100, '1': 10, '10': 1}
bin_dict_y = {'5': 10, '10': 1}
bin_dict_z = {'0': 100, '10': 10, '300': 1}

def to_bin(dct, entry):
    out = 0
    for i in entry.split(','):
       if len(i) > 0:
           out+= dct[i]  
    return str(out).zfill(len(dct))

df['x_bin'] = df.apply(lambda x: to_bin(bin_dict_x, x['x']), axis = 1)
df['y_bin'] = df.apply(lambda x: to_bin(bin_dict_y, x['y']), axis = 1)
df['z_bin'] = df.apply(lambda x: to_bin(bin_dict_y, x['z']), axis = 1)

df[['x_bin','y_bin','z_bin']]

  x_bin y_bin z_bin
0   110    01   011
1   010    10   101
2   111    00   011