If you look at the documentation for pd.DataFrame.append

Append rows of other to the end of this frame, returning a new object. Columns not in this frame are added as new columns.

(emphasis mine).

Try

df_res = df_res.append(res)

Incidentally, note that pandas isn't that efficient for creating a DataFrame by successive concatenations. You might try this, instead:

all_res = []
for df in df_all:
    for i in substr:
        res = df[df['url'].str.contains(i)]
        all_res.append(res)

df_res = pd.concat(all_res)

This first creates a list of all the parts, then creates a DataFrame from all of them once at the end.

Answer from Ami Tavory on Stack Overflow
🌐
GeeksforGeeks
geeksforgeeks.org › pandas › python-pandas-dataframe-append
Python - Pandas dataframe.append() - GeeksforGeeks
1 month ago - import pandas as pd a = pd.DataFrame({"a":[1,2], "b":[3,4]}) b = pd.DataFrame({"a":[5], "b":[6]}) r = a.append(b, ignore_index=True) print(r) ... Explanation: ignore_index=True resets the index in the resulting DataFrame.
Top answer
1 of 3
100

If you look at the documentation for pd.DataFrame.append

Append rows of other to the end of this frame, returning a new object. Columns not in this frame are added as new columns.

(emphasis mine).

Try

df_res = df_res.append(res)

Incidentally, note that pandas isn't that efficient for creating a DataFrame by successive concatenations. You might try this, instead:

all_res = []
for df in df_all:
    for i in substr:
        res = df[df['url'].str.contains(i)]
        all_res.append(res)

df_res = pd.concat(all_res)

This first creates a list of all the parts, then creates a DataFrame from all of them once at the end.

2 of 3
50

Why am I getting "AttributeError: 'DataFrame' object has no attribute 'append'?

pandas >= 2.0 append has been removed, use pd.concat instead1

Starting from pandas 2.0, append has been removed from the API. It was previously deprecated in version 1.4. See the docs on Deprecations as well as this github issue that originally proposed its deprecation.

The rationale for its removal was to discourage iteratively growing DataFrames in a loop (which is what people typically use append for). This is because append makes a new copy at each stage, resulting in quadratic complexity in memory.

1. This assume you're appending one DataFrame to another. If you're appending a row to a DataFrame, the solution is slightly different - see below.


The idiomatic way to append DataFrames is to collect all your smaller DataFrames into a list, and then make one single call to pd.concat. Here's a(n oversimplified) example

df_list = []
for df in some_function_that_yields_dfs():
    df_list.append(df)

final_df = pd.concat(df_list)

Note that if you are trying to append one row at a time rather than one DataFrame at a time, the solution is even simpler.

data = []
for a, b, c from some_function_that_yields_data():
    data.append([a, b, c])

df = pd.DataFrame(data, columns=['a', 'b', 'c'])

More information in Creating an empty Pandas DataFrame, and then filling it?

Discussions

python - Pandas merge two dataframes with different columns - Stack Overflow
I'm surely missing something simple here. Trying to merge two dataframes in pandas that have mostly the same column names, but the right dataframe has some columns that the left doesn't have, and v... More on stackoverflow.com
🌐 stackoverflow.com
python - How to append two pandas.DataFrame with different numbers of columns - Stack Overflow
Based on the fact that directly append two dataframe with different numbers of columns, an error would occur as pandas.io.common.CParserError: Error tokenizing data. C error: Expected 4 fields in l... More on stackoverflow.com
🌐 stackoverflow.com
data cleaning - Append Existing Columns to another Column in Pandas Dataframe - Data Science Stack Exchange
I have a data that looks like this: The T2M indicates the temperature, and the followed row is the year. I want to append all the similar parameters columns under a single column having all the ye... More on datascience.stackexchange.com
🌐 datascience.stackexchange.com
python - Append column to pandas dataframe - Stack Overflow
This is probably easy, but I have the following data: In data frame 1: index dat1 0 9 1 5 In data frame 2: index dat2 0 7 1 6 I want a data frame with the following form: index ... More on stackoverflow.com
🌐 stackoverflow.com
🌐
Pandas
pandas.pydata.org › docs › user_guide › merging.html
Merge, join, concatenate and compare — pandas 3.0.2 documentation
The merge suffixes argument takes a tuple or list of strings to append to overlapping column names in the input DataFrame to disambiguate the result columns:
🌐
TutorialsPoint
tutorialspoint.com › how-to-append-two-dataframes-in-pandas
How to append two DataFrames in Pandas?
August 22, 2023 - Input DataFrame 1 is: x y z 0 5 4 9 1 2 7 3 Input DataFrame 2 is: x y z 0 1 1 29 1 3 9 30 After appending, DataFrame is: x y z 0 5 4 9 1 2 7 3 2 1 1 29 3 3 9 30 · When DataFrames have different column names, missing values are filled with NaN ? import pandas as pd df1 = pd.DataFrame({"x": [5, 2], "y": [4, 7], "z": [9, 3]}) df2 = pd.DataFrame({"a": [1, 3], "b": [1, 9], "c": [29, 30]}) print("Input DataFrame 1 is:") print(df1) print("\nInput DataFrame 2 is:") print(df2) df3 = df1.append(df2) print("\nAfter appending, DataFrame is:") print(df3)
🌐
Python Examples
pythonexamples.org › pandas-append-dataframe
Append DataFrame in Pandas - Python Examples
Now, let us take two DataFrames with different columns and append the DataFrames. import pandas as pd # Initialize a DataFrame df_1 = pd.DataFrame( [['Somu', 68, 84, 78, 96], ['Kiku', 74, 56, 88, 85], ['Ajit', 77, 73, 82, 87]], columns=['name', 'physics', 'chemistry','algebra','calculus']) # Initialize another DataFrame df_2 = pd.DataFrame( [['Amol', 72, 67, 91, 83], ['Lini', 78, 69, 87, 92]], columns=['name', 'physics', 'chemistry','science','calculus']) # Append DataFrames df = df_1.append(df_2, ignore_index=True, sort=False) # Print DataFrames print("df_1\n------\n",df_1) print("\ndf_2\n------\n",df_2) print("\ndf\n--------\n",df)
🌐
Saturn Cloud
saturncloud.io › blog › append-dataframes-with-different-column-names-in-pandas
Append DataFrames with Different Column Names in Pandas | Saturn Cloud Blog
December 5, 2023 - In Pandas, you can use the append() function to append dataframes. ... However, this operation assumes that the dataframes have the same column names. If the column names are different, the append() function will result in NaN values for the ...
Top answer
1 of 3
187

I think in this case concat is what you want:

In [12]:

pd.concat([df,df1], axis=0, ignore_index=True)
Out[12]:
   attr_1  attr_2  attr_3  id  quantity
0       0       1     NaN   1        20
1       1       1     NaN   2        23
2       1       1     NaN   3        19
3       0       0     NaN   4        19
4       1     NaN       0   5         8
5       0     NaN       1   6        13
6       1     NaN       1   7        20
7       1     NaN       1   8        25

by passing axis=0 here you are stacking the df's on top of each other which I believe is what you want then producing NaN value where they are absent from their respective dfs.

2 of 3
16

The accepted answer will break if there are duplicate headers:

InvalidIndexError: Reindexing only valid with uniquely valued Index objects.

For example, here A has 3x trial columns, which prevents concat:

A = pd.DataFrame([[3, 1, 4, 1]], columns=['id', 'trial', 'trial', 'trial'])
#    id  trial  trial  trial
# 0   3      1      4      1

B = pd.DataFrame([[5, 9], [2, 6]], columns=['id', 'trial'])
#    id  trial
# 0   5      9
# 1   2      6

A_B = pd.concat([A, B], ignore_index=True)
# InvalidIndexError: Reindexing only valid with uniquely valued Index objects

To fix this, deduplicate the column names before you concat:

  • pandas 2.0+

    for df in [A, B]:
        df.columns = pd.io.common.dedup_names(df.columns, is_potential_multiindex=False)
    
    A_B = pd.concat([A, B], ignore_index=True)
    #    id  trial  trial.1  trial.2
    # 0   3      1        4        1
    # 1   5      9      NaN      NaN
    # 2   2      6      NaN      NaN
    
  • pandas < 2.0

    parser = pd.io.parsers.base_parser.ParserBase({'usecols': None})
    
    for df in [A, B]:
        df.columns = parser._maybe_dedup_names(df.columns) 
    
    A_B = pd.concat([A, B], ignore_index=True)
    #    id  trial  trial.1  trial.2
    # 0   3      1        4        1
    # 1   5      9      NaN      NaN
    # 2   2      6      NaN      NaN
    
  • pandas < 1.3

    parser = pd.io.parsers.ParserBase({})
    
    for df in [A, B]:
        df.columns = parser._maybe_dedup_names(df.columns) 
    
    A_B = pd.concat([A, B], ignore_index=True)
    #    id  trial  trial.1  trial.2
    # 0   3      1        4        1
    # 1   5      9      NaN      NaN
    # 2   2      6      NaN      NaN
    
Find elsewhere
🌐
Saturn Cloud
saturncloud.io › blog › pandas-append-dataframe-to-another-dataframe
Joining a DataFrame to Another DataFrame Using Pandas Concat | Saturn Cloud Blog
October 25, 2023 - import pandas as pd # create df1 ... 'Name': ['Dave', 'Eve'], 'Age': [40, 45], 'City': ['Houston', 'Miami'] }) We can use the concat() function to combine df1 and df2 as follows: # append df2 to df1 new_df = pd.concat([df1, df2]) ...
🌐
Replit
replit.com › home › discover › how to append a dataframe in python
How to append a dataframe in Python | Replit
March 3, 2026 - The pd.concat() function is strict—it only accepts a list of DataFrames. If you include another object type, like a raw NumPy array, Python will raise a TypeError. The code below demonstrates what happens when this rule is broken. import pandas ...
🌐
Spark By {Examples}
sparkbyexamples.com › home › pandas › pandas append() usage by examples
Pandas append() Usage by Examples - Spark By {Examples}
June 17, 2025 - pandas.DataFrame.append() method is used to append one DataFrame row(s) and column(s) with another, it can also be used to append multiple (three or more)
🌐
Pandas
pandas.pydata.org › pandas-docs › version › 1.5 › reference › api › pandas.DataFrame.append.html
pandas.DataFrame.append — pandas 1.5.3 documentation
A better solution is to append those rows to a list and then concatenate the list with the original DataFrame all at once. ... >>> df = pd.DataFrame([[1, 2], [3, 4]], columns=list('AB'), index=['x', 'y']) >>> df A B x 1 2 y 3 4 >>> df2 = pd.DataFrame([[5, 6], [7, 8]], columns=list('AB'), ...
🌐
Medium
medium.com › @amit25173 › understanding-pandas-dataframe-append-1ebb21a36b37
Understanding pandas.DataFrame.append() | by Amit Yadav | Medium
March 6, 2025 - NaN means “Not a Number,” and it’s pandas’ way of saying, “Hey, there’s no data here.” · This is handy when merging data from different sources. ... You don’t always need a whole DataFrame to append data. Sometimes, you just want to add a single row. Here’s how you do it with a dictionary. new_row = {'A': 7, 'B': 9} result = df1.append(new_row, ignore_index=True) print(result) ... The keys in the dictionary ('A' and 'B') match the column names.
🌐
KDnuggets
kdnuggets.com › 2022 › 08 › 3-ways-append-rows-pandas-dataframes.html
3 Ways to Append Rows to Pandas DataFrames - KDnuggets
To concat two dataframe or series, we will use the pandas concat() function. It provides advanced features such as appending columns using an inner or outer join.
🌐
W3Schools
w3schools.com › python › pandas › ref_df_append.asp
Pandas DataFrame append() Method
Pandas Editor Pandas Quiz Pandas Exercises Pandas Syllabus Pandas Study Plan Pandas Certificate · DataFrames Reference · ❮ DataFrame Reference · Append a DataFrame at the end of another DataFrame: import pandas as pd data1 = { "age": [16, 14, 10], "qualified": [True, True, True] } df1 = pd.DataFrame(data1) data2 = { "age": [55, 40], "qualified": [True, False] } df2 = pd.DataFrame(data2) newdf = df1.append(df2) Try it Yourself » ·
🌐
Saturn Cloud
saturncloud.io › blog › how-to-append-two-data-frames-with-pandas
How to Append Two Data Frames with Pandas | Saturn Cloud Blog
November 10, 2023 - The append method is a convenient way to append one DataFrame to another, similar to how we might append a row to a list. import pandas as pd # Creating two sample DataFrames df1 = pd.DataFrame({'A': [1, 2, 3], 'B': [4, 5, 6]}) df2 = ...
🌐
Codepointtech
codepointtech.com › home › how to append multiple pandas dataframes efficiently
How to Append Multiple Pandas DataFrames Efficiently - codepointtech.com
January 17, 2026 - The most common use case is appending DataFrames one below the other. This is done by setting the axis parameter to 0 (the default). You pass a list of DataFrames to be concatenated. Let’s start with a simple scenario where all DataFrames have the same columns. import pandas as pd # Create ...
🌐
Pickl
pickl.ai › home › python › understanding the basics of pandas dataframe.append()
Understanding the Basics of Pandas Dataframe.append()
August 1, 2024 - This method allows you to append rows from another DataFrame or Series to the calling DataFrame. ... other: This parameter represents the DataFrame or Series you want to append. It can be a single DataFrame, a Series, or even a list of DataFrames. When appending a Series, ensure it has the same column labels as the DataFrame you are appending to.