Comparing map, applymap and apply: Context Matters

The major differences are:

Definition

• map is defined on Series only
• applymap is defined on DataFrames only
• apply is defined on both

Input argument

• map accepts dict, Series, or callable
• applymap and apply accept callable only

Behavior

• map is elementwise for Series
• applymap is elementwise for DataFrames
• apply also works elementwise but is suited to more complex operations and aggregation. The behaviour and return value depends on the function.

Use case (the most important difference)

• map is meant for mapping values from one domain to another, so is optimised for performance, e.g.,

  df['A'].map({1:'a', 2:'b', 3:'c'})
  

• applymap is good for elementwise transformations across multiple rows/columns, e.g.,

  df[['A', 'B', 'C']].applymap(str.strip)
  

• apply is for applying any function that cannot be vectorised, e.g.,

  df['sentences'].apply(nltk.sent_tokenize)
  

Also see When should I (not) want to use pandas apply() in my code? for a writeup I made a while back on the most appropriate scenarios for using apply. (Note that there aren't many, but there are a few— apply is generally slow.)

Summarising

Footnotes

1. map when passed a dictionary/Series will map elements based on the keys in that dictionary/Series. Missing values will be recorded as NaN in the output.

2. applymap in more recent versions has been optimised for some operations. You will find applymap slightly faster than apply in some cases. My suggestion is to test them both and use whatever works better.

3. map is optimised for elementwise mappings and transformation. Operations that involve dictionaries or Series will enable pandas to use faster code paths for better performance.

4. Series.apply returns a scalar for aggregating operations, Series otherwise. Similarly for DataFrame.apply. Note that apply also has fastpaths when called with certain NumPy functions such as mean, sum, etc.

What's meant by .transform() returns a like-indexed DataFrame stated in the documentation?

That means .transform() applies a function to every value (or a group, once preceded by groupby) in the DataFrame and returns another DataFrame with the same length as the input, so to emphasize: it keeps the input index labels in the output.

Is there any use case where one of applymap/transform and the other doesn't?

Sure. Here are some examples:

1) applymap Vs transform

Since applymap performs on all elements of a DataFrame, you can't perform applymap on a Series:

df['Quantity'].transform(lambda x: x+10) # successful
df['Quantity'].apply(lambda x: x+10) # successful
df['Quantity'].applymap(lambda x: x+10) # gives AttributeError: 'Series' object has no attribute 'applymap'

# unless you cast it to DataFrame:
pd.DataFrame(df['Quantity']).applymap(lambda x: x+10) # successful

Another important difference is that despite .applymap() which operates element-wise, .transform() can perform group-wise operations, referred to in the next part.

Moreover, applymap cannot be preceded by groupby.

2) apply Vs transform

apply and transform can be interchangeable as long as you perform them on DataFrame column(s). Here is a simple example:

# imagine the following DataFrame
df = pd.DataFrame({'Label': ['A', 'B', 'C', 'A', 'C'],
                   'Values': [0,1,2,3,4],
                   'Quantity': [5,6,7,8,9]}, index = list('VWXYZ'))


    Label   Quantity   Values
---------------------------------
V    A         5         0
W    B         6         1
X    C         7         2
Y    A         8         3
Z    C         9         4


df.loc[:, ['Quantity', 'Values']].apply(lambda x: x+10)
df.loc[:, ['Quantity', 'Values']].transform(lambda x: x+10)
# both of them give the following same result:

    Quantity   Values
-------------------------
V    15          10
W    16          11
X    17          12
Y    18          13
Z    19          14

The main difference emerges once they follow a groupby operation. For instance:

label_grouping = df.groupby('Label')
label_grouping.apply(lambda x: x.mean())
# output:

      Quantity   Values
Label
-----------------------
A       6.5       1.5
B       6.0       1.0
C       8.0       3.0

label_grouping.transform(lambda x: x.mean())
# see how `transform` could manage to keeps the input index labels in the output
# output:

    Quantity   Values
------------------------
V     6.5       1.5
W     6.0       1.0
X     8.0       3.0
Y     6.5       1.5
Z     8.0       3.0

The above example clearly shows how transform can retain input DataFrame indexes; So to get the better of this exclusive feature, the following short example tries to clarify how to benefit from this alignment of the indexes between the input and output of the transform operation, by calculating the percentage of order total that each product represents:

df_sales = pd.DataFrame({'OrderID': [1001,1001,1001,1002,1002],
                         'Product': ['p1','p2','p3','p1','p4'],
                         'Quantity': [30,20,70,160,40]})


    OrderID   Product   Quantity
-----------------------------------
0    1001       p1        30
1    1001       p2        20
2    1001       p3        70
3    1002       p1        160
4    1002       p4        40


df_sales['total_per_order'] = df_sales.groupby(['OrderID'])['Quantity'].transform(lambda x: x.sum()) 
df_sales['pct_of_order'] = df_sales['Quantity'] / df_sales['total_per_order']


    OrderID   Product   Quantity   total_per_order   pct_of_order
----------------------------------------------------------------------
0    1001       p1        30           120             0.250000
1    1001       p2        20           120             0.166667
2    1001       p3        70           120             0.583333
3    1002       p1        160          200             0.800000
4    1002       p4        40           200             0.200000

It's highly advised to follow this link for a more detailed example: https://pbpython.com/pandas_transform.html

Many aggregation functions are built in directly to the groupby object to save you some typing. specifically some of the common ones to benefit from are (prefixed by gb):

• gb.apply
• gb.transform
• gb.filter
• gb.agg
• gb.count
• gb.comsum
• gb.fillna
• ...

Hope this helped :)