You you are almost there... Like you said you can add all the columns you want to encode in fit_transform directly.

ohe = OneHotEncoder(categories='auto')
feature_arr = ohe.fit_transform(df[['phone','city']]).toarray()
feature_labels = ohe.categories_

And then you just need to do the following:

feature_labels = np.array(feature_labels).ravel()

Which enables you to name your columns like you wanted:

features = pd.DataFrame(feature_arr, columns=feature_labels)

LabelEncoder is not made to transform the data but the target (also known as labels) as explained here. If you want to encode the data you should use OrdinalEncoder.

If you really need to do it this way:

categorical_cols = ['a', 'b', 'c', 'd'] 

from sklearn.preprocessing import LabelEncoder
# instantiate labelencoder object
le = LabelEncoder()

# apply le on categorical feature columns
data[categorical_cols] = data[categorical_cols].apply(lambda col: le.fit_transform(col))    
from sklearn.preprocessing import OneHotEncoder
ohe = OneHotEncoder()

#One-hot-encode the categorical columns.
#Unfortunately outputs an array instead of dataframe.
array_hot_encoded = ohe.fit_transform(data[categorical_cols])

#Convert it to df
data_hot_encoded = pd.DataFrame(array_hot_encoded, index=data.index)

#Extract only the columns that didnt need to be encoded
data_other_cols = data.drop(columns=categorical_cols)

#Concatenate the two dataframes : 
data_out = pd.concat([data_hot_encoded, data_other_cols], axis=1)

Otherwise:

I suggest you to use pandas.get_dummies if you want to achieve one-hot-encoding from raw data (without having to use OrdinalEncoder before) :

#categorical data
categorical_cols = ['a', 'b', 'c', 'd'] 

#import pandas as pd
df = pd.get_dummies(data, columns = categorical_cols)

You can also use drop_first argument to remove one of the one-hot-encoded columns, as some models require.

import pandas as pd
df = pd.DataFrame({'name': ['Manie', 'Joyce', 'Ami'],
                   'Org':  ['ABC2', 'ABC1', 'NSV2'],
                   'Dept': ['Finance', 'HR', 'HR']        
        })


df_2 = pd.get_dummies(df,drop_first=True)

test:

print(df_2)
   Dept_HR  Org_ABC2  Org_NSV2  name_Joyce  name_Manie
0        0         1         0           0           1
1        1         0         0           1           0
2        1         0         1           0           0 

UPDATE regarding your error with pd.get_dummies(X, columns =[1:]:

Per the documentation page, the columns parameter takes "Column Names". So the following code would work:

df_2 = pd.get_dummies(df, columns=['Org', 'Dept'], drop_first=True)

output:

    name  Org_ABC2  Org_NSV2  Dept_HR
0  Manie         1         0        0
1  Joyce         0         0        1
2    Ami         0         1        1

If you really want to define your columns positionally, you could do it this way:

column_names_for_onehot = df.columns[1:]
df_2 = pd.get_dummies(df, columns=column_names_for_onehot, drop_first=True)

Update

I want to obtain only four... one for each 'a', 'b', 'c' and 'd' categories distributed in my columns

You can use value_counts applied on columns axis:

>>> df1.apply(pd.value_counts, axis="columns").fillna(0).astype(int)
   a  b  c  d
0  1  1  1  0
1  1  1  0  1
2  1  0  1  1
3  1  1  0  1

Old answer

Some explanations about how encoding working:

>>> df1
   W  X  Y  Z
0  a  b  c  c
1  b  a  a  b
2  a  c  a  b

>>> df1.nunique()
W    2  # [a, b]
X    3  # [a, b, c]
Y    2  # [a, c]
Z    2  # [b, c]

For column W, there are two different values [a, b] so you need 2 columns to encode them: For instance:

   a  b
a  1  0
b  0  1

For column X, there are three different values [a, b, c] so you need 3 columns to encode them. For instance:

   a  b  c
a  1  0  0
b  0  1  0
c  0  0  1

Note the identity matrix.

Let's use pd.get_dummies rather than OneHotEncoder to a better understanding:

>>> pd.get_dummies(df1)
   W_a  W_b  X_a  X_b  X_c  Y_a  Y_c  Z_b  Z_c
0    1    0    0    1    0    0    1    0    1
1    0    1    1    0    0    1    0    1    0
2    1    0    0    0    1    1    0    1    0

The question is why you want to get only 3 columns?

Starting from sklearn==0.20 OneHotEncoder has categories parameter where you can provide a list of lists with all the possible values for a given column.

import pandas as pd
df = pd.DataFrame([[0, 1, 2, 3, 1],
 [3, 1, 0, 1, 1],
 [0, 1, 0, 3, 0],
 [0, 2, 0, 1, 2],
 [0, 2, 0, 3, 2]], columns=['color1', 'color2', 'color3', 'color4', 'color5'])

from sklearn.preprocessing import OrdinalEncoder, OneHotEncoder

# Get all the unique values if we don't have them
unique_values = pd.unique(df.values.ravel()) 

ohe = OneHotEncoder(categories=[unique_values]*df.shape[1], sparse=False)
encoded = pd.DataFrame(ohe.fit_transform(
    df), columns=ohe.get_feature_names(df.columns))
>>> encoded

   color1_0  color1_1  color1_2  color1_3  color2_0  color2_1    0
0       1.0       0.0       0.0       0.0       0.0       1.0  ...
1       0.0       0.0       0.0       1.0       0.0       1.0  ...
2       1.0       0.0       0.0       0.0       0.0       1.0  ...
3       1.0       0.0       0.0       0.0       0.0       0.0  ...
4       1.0       0.0       0.0       0.0       0.0       0.0  ...

To get back original classes you can do inverse_transform:

>>> ohe.inverse_transform(encoded) 
array([[0, 1, 2, 3, 1],
       [3, 1, 0, 1, 1],
       [0, 1, 0, 3, 0],
       [0, 2, 0, 1, 2],
       [0, 2, 0, 3, 2]], dtype=int64)