Firstly, when you want to encode categorical variables, which is not ordinal (meaning: there is no inherent ordering between the values of the variable/column. ex- cat, dog), you must use one hot encoding.

import pandas as pd
from sklearn.preprocessing import OneHotEncoder 

df = pd.DataFrame({'pets': ['cat', 'dog', 'cat', 'monkey', 'dog', 'meo'], 
                   'owner': ['Champ', 'Ron', 'Brick', 'Champ', 'Veronica', 'Ron'], 
                   'location': ['San_Diego', 'New_York', 'New_York', 'San_Diego', 'San_Diego', 
             'New_York']})

enc = [['cat','dog','monkey'],
       ['Brick', 'Champ', 'Ron', 'Veronica'],
       ['New_York', 'San_Diego']]
ohe = OneHotEncoder(categories=enc, handle_unknown='ignore', sparse=False)

Here, I have modified your enc in a way that can be fed into the OneHotEncoder.

Now comes the point of how can we going to handle the unseen labels?

when you handle_unknown as False, the unseen values will have zeros in all the dummy variables, which in a way would help the model to understand its a unknown value.

colnames= ['{}_{}'.format(col,val) for col,unique_values in zip(df.columns,ohe.categories_) \
                                       for val in unique_values]
pd.DataFrame(ohe.fit_transform(df), columns=colnames) 

Update:

If you are fine with ordinal endocing, the following change could help.

df2.apply(lambda row: [transform_dict[val].get(col,0) \
                                    for val,col in row.items()], 
          axis=1,
          result_type='expand')

#1000 loops, best of 3: 1.17 ms per loop

IIUC, your user is empty and everything is on name. If that's the case, you can

pd.pivot_table(df, index=df.name.str[0], columns=df.name.str[1:].values, aggfunc='count').fillna(0)

Generally, the logic of the categorical count transformation lies in the fact that features with similar frequencies tend to behave similarly. Have words in a corpus as an example, common words share little or no real information whereas uncommon words share more information with an algorithm.

Specifically, certain algorithms (tree-based methods) could even yield rules given an unspecified category from an event count. Say, for example, we have an unknown category whose count is 4. The algorithm may give a rule:

If Column Count is < 5 and N is > 3 = X

That will be exactly the same as if an algorithm took a One-Hot encoded column and gave a rule:

If One-Hot-Encoded-Column is > 0 = X

In that case, a tree-based algorithm will make several rules from many categories using the same count column. But how I said at the beginning, algorithms generalize among populations of similar counts so most likely you will find rules like:

If Column Counts is < 10 and N is > 3 = X

Which will often contain different categories that behave similarly. Just check the models and look for parameters/ importance of the column in question to see this for yourself.

Secondly, I was wondering if anyone may be willing to share other techniques of dealing with categorical variables.

Feature hashing became really popular there not so long ago. Hashing has very nice properties and it's a whole topic learned at schools but the main principle is that if you have a category with high cardinality you decide a minimum number of reduced categories (hashes) that all the categories will have to share. if two categories share the same hash or bucket, that is called a hash collision. Feature hashing doesn't deal with hash collisions because according to some authors (I don't have the reference here) may improve accuracy by forcing the algorithm to pick more carefully the features.

There are many ways we can encode these categorical variables as numbers and use them in the algorithm.

1) One Hot Encoding
2) Label Encoding
3) Ordinal Encoding
4) Helmert Encoding
5) Binary Encoding
6) Frequency Encoding
7) Mean Encoding
8) Weight of Evidence Encoding
9) Probability Ratio Encoding
10) Hashing Encoding
11) Backward Difference Encoding
12) Leave One Out Encoding
13) James-Stein Encoding
14) M-estimator Encoding

Find the below cheatsheet

More Info