The following should work, here we mask the df where the condition is met, this will set NaN to the rows where the condition isn't met so we call fillna on the new col:

In [67]:
df = pd.DataFrame(np.random.randn(5,3), columns=list('ABC'))
df

Out[67]:
          A         B         C
0  0.197334  0.707852 -0.443475
1 -1.063765 -0.914877  1.585882
2  0.899477  1.064308  1.426789
3 -0.556486 -0.150080 -0.149494
4 -0.035858  0.777523 -0.453747

In [73]:    
df['total'] = df.loc[df['A'] > 0,['A','B']].sum(axis=1)
df['total'].fillna(0, inplace=True)
df

Out[73]:
          A         B         C     total
0  0.197334  0.707852 -0.443475  0.905186
1 -1.063765 -0.914877  1.585882  0.000000
2  0.899477  1.064308  1.426789  1.963785
3 -0.556486 -0.150080 -0.149494  0.000000
4 -0.035858  0.777523 -0.453747  0.000000

Another approach is to call where on the sum result, this takes a value param to return when the condition isn't met:

In [75]:
df['total'] = df[['A','B']].sum(axis=1).where(df['A'] > 0, 0)
df

Out[75]:
          A         B         C     total
0  0.197334  0.707852 -0.443475  0.905186
1 -1.063765 -0.914877  1.585882  0.000000
2  0.899477  1.064308  1.426789  1.963785
3 -0.556486 -0.150080 -0.149494  0.000000
4 -0.035858  0.777523 -0.453747  0.000000
Answer from EdChum on Stack Overflow
🌐
Statology
statology.org › home › pandas: how to sum columns based on a condition
Pandas: How to Sum Columns Based on a Condition
January 18, 2021 - You can use the following syntax to sum the values of a column in a pandas DataFrame based on a condition:
🌐
Reddit
reddit.com › r/learnpython › pandas sum row values based on condition
r/learnpython on Reddit: Pandas sum row values based on condition
July 23, 2021 -

Hi friends - I am sure this is very simple but I have googled my heart out and can't figure out how to do this. I am trying to append a new column to a pandas dataframe which sums all values in existing columns only if they are even.

odd_lst = [1, 3, 5, 7, 9]
even_lst = [0, 2, 4, 6, 8]

df = pd.DataFrame()

df['Odd'] = odd_lst
df['Even'] = even_lst

df['Odd Sum'] = df.apply(some lambda function or something which only executes if values are odd, axis = 1)

Am I supposed to use lambda here? I don't know why I am struggling so much with this. I want to go row by row and only sum the values if they are odd and append this as a new column on the dataframe.

Discussions

python - How do I sum values in a column that match a given condition using pandas? - Stack Overflow
Suppose I have a dataframe like so: a b 1 5 1 7 2 3 1 3 2 5 I want to sum up the values for b where a = 1, for example. This would give me 5 + 7 + 3 = 15. How do I do this in pandas? More on stackoverflow.com
🌐 stackoverflow.com
pandas - python sum a column's value with condition - Stack Overflow
I have the below dataframe. I would like to return a second column that is the sum of every item in the column with a condition: only those larger than -1. input Price 0 12 1 14 2 15 3 ... More on stackoverflow.com
🌐 stackoverflow.com
python - How to sum over some columns based on condition in pandas - Stack Overflow
Timeit results for df with 30000 rows: method_1 ran in 0.008165133331203833 seconds using 3 iterations method_2 ran in 13.408894366662329 seconds using 3 iterations method_3 ran in 0.007688766665523872 seconds using 3 iterations method_4 ran in 0.006326200003968552 seconds using 3 iterations · Time performance results: Method #4 (numpy multiply/sum) is about 20% faster than the runners-up. Methods #1 and #3 (pandas ... More on stackoverflow.com
🌐 stackoverflow.com
pandas - How to groupby and sum values of only one column based on value of another column - Data Science Stack Exchange
I have a dataset that has the following columns: Category, Product, Launch_Year, and columns named 2010, 2011 and 2012. These 13 columns contain sales of the product in that year. The goal is to cr... More on datascience.stackexchange.com
🌐 datascience.stackexchange.com
June 2, 2022
Top answer
1 of 2
36

The following should work, here we mask the df where the condition is met, this will set NaN to the rows where the condition isn't met so we call fillna on the new col:

In [67]:
df = pd.DataFrame(np.random.randn(5,3), columns=list('ABC'))
df

Out[67]:
          A         B         C
0  0.197334  0.707852 -0.443475
1 -1.063765 -0.914877  1.585882
2  0.899477  1.064308  1.426789
3 -0.556486 -0.150080 -0.149494
4 -0.035858  0.777523 -0.453747

In [73]:    
df['total'] = df.loc[df['A'] > 0,['A','B']].sum(axis=1)
df['total'].fillna(0, inplace=True)
df

Out[73]:
          A         B         C     total
0  0.197334  0.707852 -0.443475  0.905186
1 -1.063765 -0.914877  1.585882  0.000000
2  0.899477  1.064308  1.426789  1.963785
3 -0.556486 -0.150080 -0.149494  0.000000
4 -0.035858  0.777523 -0.453747  0.000000

Another approach is to call where on the sum result, this takes a value param to return when the condition isn't met:

In [75]:
df['total'] = df[['A','B']].sum(axis=1).where(df['A'] > 0, 0)
df

Out[75]:
          A         B         C     total
0  0.197334  0.707852 -0.443475  0.905186
1 -1.063765 -0.914877  1.585882  0.000000
2  0.899477  1.064308  1.426789  1.963785
3 -0.556486 -0.150080 -0.149494  0.000000
4 -0.035858  0.777523 -0.453747  0.000000
2 of 2
1

Another approach is to use numpy.where() method to select values. It returns elements chosen from the sum result if the condition is met, 0 otherwise. Due to a lower overhead, numpy methods are usually faster than their pandas cousins. Barring numba-jitted or Cython loops, this is the fastest approach for this specific task.

import numpy as np
df['Total'] = np.where(df['A'] < 0.78, df[['A','B','C']].sum(axis=1), 0)

or

df['total'] = np.where(df['dog'] == 'dog2', df[['A','B','C']].sum(axis=1), 0)
Top answer
1 of 3
182

The essential idea here is to select the data you want to sum, and then sum them. This selection of data can be done in several different ways, a few of which are shown below.

Boolean indexing

Arguably the most common way to select the values is to use Boolean indexing.

With this method, you find out where column 'a' is equal to 1 and then sum the corresponding rows of column 'b'. You can use loc to handle the indexing of rows and columns:

>>> df.loc[df['a'] == 1, 'b'].sum()
15

The Boolean indexing can be extended to other columns. For example if df also contained a column 'c' and we wanted to sum the rows in 'b' where 'a' was 1 and 'c' was 2, we'd write:

df.loc[(df['a'] == 1) & (df['c'] == 2), 'b'].sum()

Query

Another way to select the data is to use query to filter the rows you're interested in, select column 'b' and then sum:

>>> df.query("a == 1")['b'].sum()
15

Again, the method can be extended to make more complicated selections of the data:

df.query("a == 1 and c == 2")['b'].sum()

Note this is a little more concise than the Boolean indexing approach.

Groupby

The alternative approach is to use groupby to split the DataFrame into parts according to the value in column 'a'. You can then sum each part and pull out the value that the 1s added up to:

>>> df.groupby('a')['b'].sum()[1]
15

This approach is likely to be slower than using Boolean indexing, but it is useful if you want check the sums for other values in column a:

>>> df.groupby('a')['b'].sum()
a
1    15
2     8
2 of 3
11

You can also do this without using groupby or loc. By simply including the condition in code. Let the name of dataframe be df. Then you can try :

df[df['a']==1]['b'].sum()

or you can also try :

sum(df[df['a']==1]['b'])

Another way could be to use the numpy library of python :

import numpy as np
print(np.where(df['a']==1, df['b'],0).sum())
🌐
Bobby Hadz
bobbyhadz.com › blog › pandas-sum-values-in-column-with-condition
Pandas: Sum the values in a Column that match a Condition | bobbyhadz
You can use boolean indexing to sum the values in a column in a Pandas DataFrame that match a condition.
🌐
IncludeHelp
includehelp.com › python › how-to-sum-values-in-a-column-that-matches-a-given-condition-using-pandas.aspx
How to sum values in a column that matches a given condition using Pandas?
Suppose we have a column with integer ... is 5. To find the sum value in a column that matches a given condition, we will use pandas.DataFrame.loc property and sum() method, first, we will check the condition if the value of 1st column matches a specific condition, then we will ...
🌐
Vultr Docs
docs.vultr.com › python › third party › pandas › dataframe › sum()
Python Pandas DataFrame sum() - Sum Column Values
December 25, 2024 - This code snippet sums the values in column 'A' where the values are greater than 1. The result is a conditional sum, demonstrating how flexible the sum() function can be in practice.
🌐
Trymito
trymito.io › excel-to-python › functions › math › SUMIF
Excel to Python: SUMIF Function - A Complete Guide | Mito
To use SUMIF in Python, you can use conditional expressions, like 'df['A'] > 5', combined with the `sum` method to sum the values that match the condition. In pandas, you can sum the values of a column based on a condition from another column using a simple comparison and the `sum` method.
Find elsewhere
🌐
datagy
datagy.io › home › pandas tutorials › data analysis in pandas › pandas sum: add dataframe columns and rows
Pandas Sum: Add Dataframe Columns and Rows • datagy
December 15, 2022 - We can see that the we’ve created a new column that stores the sum of two of our columns. A great thing about this operation is that its vectorized, meaning that its very fast and takes advantage of the power of Pandas. In the next section, you’ll learn how to add dataframe columns conditionally. There may be times when you want to add multiple columns in a dataframe, but not all of them. We can do this by adding Pandas columns conditionally, with the help of a list comprehension.
🌐
Arab Psychology
scales.arabpsychology.com › psychological scales › how to sum columns based on a condition
How To Sum Columns Based On A Condition
November 15, 2023 - The range is the range of cells to be evaluated, the criteria is the condition that must be met to be included in the sum, and the sum_range is the range of cells to be summed. This function allows you to quickly sum a column based on a given condition. You can use the following syntax to sum the values of a column in a pandas DataFrame based on a condition:
🌐
Delft Stack
delftstack.com › home › howto › python pandas › how to get the sum of pandas column
How to Get the Sum of Pandas Column | Delft Stack
February 2, 2024 - We will introduce how to get the sum of pandas dataframe column. It includes methods like calculating cumulative sum with groupby, and dataframe sum of columns based on conditional of other column values.
🌐
thisPointer
thispointer.com › home › dataframe › pandas: get sum of column values in a dataframe
Pandas: Get sum of column values in a Dataframe - thisPointer
April 30, 2023 - Using loc[] we selected the column ‘Score’ but for only those rows where column ‘City’ has value ‘Delhi’. Then we called the sum() function on the series object to get the sum of scores of students from ‘Delhi’. So, basically ...
🌐
Easy Tweaks
easytweaks.com › python-pandas-sum-columns-dataframe
How to sum specific multiple rows in a Pandas DataFrame?
August 26, 2021 - Master meetings, chats, channels and online collaboration · Go beyond the basics in Word, Excel, PowerPoint and Outlook
Top answer
1 of 2
4

You can use mask. The idea is to create a boolean mask with the w columns, and use it to filter the relevant w columns and sum:

df['top_p'] = df.filter(like='p').mask(df.filter(like='w').isin(['CUSTOM_MASK','CUSTOM_UNKNOWN']).to_numpy()).sum(axis=1)

Output:

    p1   p2    p3    p4   p5      w1    w2           w3              w4           w5  top_p
0  0.1  0.2  0.10  0.11  0.3  cancel  good       thanks     CUSTOM_MASK  CUSTOM_MASK   0.40
1  0.2  0.1  0.90  0.20  0.1   hello   bad  CUSTOM_MASK  CUSTOM_UNKNOWN  CUSTOM_MASK   0.30
2  0.3  0.3  0.01  0.40  0.5      hi  ugly        great          trible          job   1.51

Before summing, the output of mask looks like:

    p1   p2    p3   p4   p5
0  0.1  0.2  0.10  NaN  NaN
1  0.2  0.1   NaN  NaN  NaN
2  0.3  0.3  0.01  0.4  0.5
2 of 2
1

Here's a way to do this using np.dot():

pCols, wCols = ['p'+str(i + 1) for i in range(5)], ['w'+str(i + 1)for i in range(5)]
mydf['top_p'] = mydf.apply(lambda x: np.dot(x[pCols], ~(x[wCols].isin(['CUSTOM_MASK','CUSTOM_UNKNOWN']))), axis=1)

We first prepare the two sets of column names p1,...,p5 and w1,...,w5.

Then we use apply() to take the dot product of the values in the pN columns with the filtering criteria based on the wN columns (namely include only contributions from pN column values whose corresponding wN column value is not in the list of excluded strings).

Output:

    p1   p2    p3    p4   p5      w1    w2           w3              w4           w5  top_p
0  0.1  0.2  0.10  0.11  0.3  cancel  good       thanks     CUSTOM_MASK  CUSTOM_MASK   0.40
1  0.2  0.1  0.90  0.20  0.1   hello   bad  CUSTOM_MASK  CUSTOM_UNKNOWN  CUSTOM_MASK   0.30
2  0.3  0.3  0.01  0.40  0.5      hi  ugly        great          trible          job   1.51

Alternatively, element-wise multiplication and sum across columns can be used like this:

pCols, wCols = [[c for c in mydf.columns if c[0] == char] for char in 'pw']
colMap = {wCols[i] : pCols[i] for i in range(len(pCols))}
mydf['top_p'] = (mydf[pCols] * ~mydf[wCols].rename(columns=colMap).isin(['CUSTOM_MASK','CUSTOM_UNKNOWN'])).sum(axis=1)

Here, we needed to rename the columns of one of the 5-column DataFrames to ensure that * (DataFrame.multiply()) can do the element-wise multiplication.

UPDATE: Here are a few timing comparisons on various possible methods for solving this question:

#1. Pandas mask and sum (see answer by @enke):

df['top_p'] = df.filter(like='p').mask(df.filter(like='w').isin(['CUSTOM_MASK','CUSTOM_UNKNOWN']).to_numpy()).sum(axis=1)

#2. Pandas apply with Numpy dot solution:

pCols, wCols = ['p'+str(i + 1) for i in range(5)], ['w'+str(i + 1)for i in range(5)]
df['top_p'] = df.apply(lambda x: np.dot(x[pCols], ~(x[wCols].isin(['CUSTOM_MASK','CUSTOM_UNKNOWN']))), axis=1)

#3. Pandas element-wise multiply and sum:

pCols, wCols = [[c for c in df.columns if c[0] == char] for char in 'pw']
colMap = {wCols[i] : pCols[i] for i in range(len(pCols))}
df['top_p'] = (df[pCols] * ~df[wCols].rename(columns=colMap).isin(['CUSTOM_MASK','CUSTOM_UNKNOWN'])).sum(axis=1)

#4. Numpy element-wise multiply and sum:

pCols, wCols = [[c for c in df.columns if c[0] == char] for char in 'pw']
df['top_p'] = (df[pCols].to_numpy() * ~df[wCols].isin(['CUSTOM_MASK','CUSTOM_UNKNOWN']).to_numpy()).sum(axis=1)

Timing results:

Timeit results for df with 30000 rows:
method_1 ran in 0.008165133331203833 seconds using 3 iterations
method_2 ran in 13.408894366662329 seconds using 3 iterations
method_3 ran in 0.007688766665523872 seconds using 3 iterations
method_4 ran in 0.006326200003968552 seconds using 3 iterations

Time performance results: Method #4 (numpy multiply/sum) is about 20% faster than the runners-up. Methods #1 and #3 (pandas mask/sum vs multiply/sum) are neck-and-neck in second place. Method #2 (pandas apply/numpy dot) is frightfully slow.

Here's the timeit() test code in case it's of interest:

import pandas as pd
import numpy as np
nListReps = 10000
df = pd.DataFrame({'p1':[0.1, 0.2, 0.3]*nListReps, 'p2':[0.2, 0.1,0.3]*nListReps, 'p3':[0.1,0.9, 0.01]*nListReps, 'p4':[0.11, 0.2, 0.4]*nListReps, 'p5':[0.3, 0.1,0.5]*nListReps,
        'w1':['cancel','hello', 'hi']*nListReps, 'w2':['good','bad','ugly']*nListReps, 'w3':['thanks','CUSTOM_MASK','great']*nListReps,
        'w4':['CUSTOM_MASK','CUSTOM_UNKNOWN', 'trible']*nListReps,'w5':['CUSTOM_MASK','CUSTOM_MASK','job']*nListReps})

from timeit import timeit

def foo_1(df):
    df['top_p'] = df.filter(like='p').mask(df.filter(like='w').isin(['CUSTOM_MASK','CUSTOM_UNKNOWN']).to_numpy()).sum(axis=1)
    return df

def foo_2(df):
    pCols, wCols = ['p'+str(i + 1) for i in range(5)], ['w'+str(i + 1)for i in range(5)]
    df['top_p'] = df.apply(lambda x: np.dot(x[pCols], ~(x[wCols].isin(['CUSTOM_MASK','CUSTOM_UNKNOWN']))), axis=1)
    return df

def foo_3(df):
    pCols, wCols = [[c for c in df.columns if c[0] == char] for char in 'pw']
    colMap = {wCols[i] : pCols[i] for i in range(len(pCols))}
    df['top_p'] = (df[pCols] * ~df[wCols].rename(columns=colMap).isin(['CUSTOM_MASK','CUSTOM_UNKNOWN'])).sum(axis=1)
    return df

def foo_4(df):
    pCols, wCols = [[c for c in df.columns if c[0] == char] for char in 'pw']
    df['top_p'] = (df[pCols].to_numpy() * ~df[wCols].isin(['CUSTOM_MASK','CUSTOM_UNKNOWN']).to_numpy()).sum(axis=1)
    return df

n = 3
print(f'Timeit results for df with {len(df.index)} rows:')
for foo in ['foo_'+str(i + 1) for i in range(4)]:
    t = timeit(f"{foo}(df.copy())", setup=f"from __main__ import df, {foo}", number=n) / n
    print(f'{foo} ran in {t} seconds using {n} iterations')

Conclusion: The absolute fastest of these four approaches seems to be Numpy element-wise multiply and sum. However, @enke's Pandas mask and sum is pretty close in performance and is arguably the most aesthetically pleasing of the four candidates.

Perhaps this hybrid of the two (which runs about as fast as #4 above) is worth considering:

df['top_p'] = (df.filter(like='p').to_numpy() * ~df.filter(like='w').isin(['CUSTOM_MASK','CUSTOM_UNKNOWN']).to_numpy()).sum(axis=1)
🌐
Pandas
pandas.pydata.org › docs › reference › api › pandas.DataFrame.sum.html
pandas.DataFrame.sum — pandas 3.0.6 documentation
The behavior of DataFrame.sum with axis=None is deprecated, in a future version this will reduce over both axes and return a scalar To retain the old behavior, pass axis=0 (or do not pass axis). Added in version 2.0.0. ... Exclude NA/null values when computing the result. ... Include only float, int, boolean columns...
🌐
Arab Psychology
scales.arabpsychology.com › home › can columns be summed in pandas based on a condition?
Can Columns Be Summed In Pandas Based On A Condition?
April 24, 2024 - This means that users can specify a certain criteria or condition, and Pandas will only sum the values in the selected columns that meet that condition. This feature allows for efficient and targeted data analysis, as it eliminates the need ...
🌐
IncludeHelp
includehelp.com › python › pandas-conditional-sum-with-groupby.aspx
Python - Pandas: Conditional Sum with Groupby
We need to find out the sum of a column where the grouped column is course and we need to apply a condition that only those values will be added where the course is equal to a specific value. For this purpose, we will first group the required column, We will then apply a lambda function on ...