If you look at the documentation for pd.DataFrame.append
Append rows of other to the end of this frame, returning a new object. Columns not in this frame are added as new columns.
(emphasis mine).
Try
df_res = df_res.append(res)
Incidentally, note that pandas isn't that efficient for creating a DataFrame by successive concatenations. You might try this, instead:
all_res = []
for df in df_all:
for i in substr:
res = df[df['url'].str.contains(i)]
all_res.append(res)
df_res = pd.concat(all_res)
This first creates a list of all the parts, then creates a DataFrame from all of them once at the end.
Answer from Ami Tavory on Stack OverflowIf you look at the documentation for pd.DataFrame.append
Append rows of other to the end of this frame, returning a new object. Columns not in this frame are added as new columns.
(emphasis mine).
Try
df_res = df_res.append(res)
Incidentally, note that pandas isn't that efficient for creating a DataFrame by successive concatenations. You might try this, instead:
all_res = []
for df in df_all:
for i in substr:
res = df[df['url'].str.contains(i)]
all_res.append(res)
df_res = pd.concat(all_res)
This first creates a list of all the parts, then creates a DataFrame from all of them once at the end.
Why am I getting "AttributeError: 'DataFrame' object has no attribute 'append'?
pandas >= 2.0 append has been removed, use pd.concat instead1
Starting from pandas 2.0, append has been removed from the API. It was previously deprecated in version 1.4. See the docs on Deprecations as well as this github issue that originally proposed its deprecation.
The rationale for its removal was to discourage iteratively growing DataFrames in a loop (which is what people typically use append for). This is because append makes a new copy at each stage, resulting in quadratic complexity in memory.
1. This assume you're appending one DataFrame to another. If you're appending a row to a DataFrame, the solution is slightly different - see below.
The idiomatic way to append DataFrames is to collect all your smaller DataFrames into a list, and then make one single call to pd.concat. Here's a(n oversimplified) example
df_list = []
for df in some_function_that_yields_dfs():
df_list.append(df)
final_df = pd.concat(df_list)
Note that if you are trying to append one row at a time rather than one DataFrame at a time, the solution is even simpler.
data = []
for a, b, c from some_function_that_yields_data():
data.append([a, b, c])
df = pd.DataFrame(data, columns=['a', 'b', 'c'])
More information in Creating an empty Pandas DataFrame, and then filling it?
Appending Data to DataFrames
Don't append rows to a pandas DataFrame
I need to append a dataframe to another and I must ensure that the order of the rows stays the same
inserting a pandas dataframe into another dataframe
Videos
One thing that has always made me kinda mad at Pandas is the lack of an easy method to append a row to a dataframe.
Like if I create a dataframe like this.
df = pd.DataFrame(columns=['Name', 'Age', 'Job'])
then I want to add a row, I have to do something rediculous like
df.loc[len(df)] = ["Velocibadgery", 35, "Programmer"]
instead of just
df.append([row data])
So I decided to do something about it. I first started to create my own wrapper class for pandas, and it was a pain. To build in the same amount of functionality I basically just had to expose the dataframe outside the class, which is fine, but just hurts my OOP sensibilities. I like to keep data in my class contained in my class.
So I looked into maybe inheriting from pandas instead. So I dug into the documentation and I found that you can just extend pandas instead and tack on your own methods. So I came up with the following code
@pd.api.extensions.register_dataframe_accessor("easy")
class EasyFrame:
def __init__(self, pandas_obj):
self._obj = pandas_obj
def append(self, data):
if isinstance(data[0], list):
for row in data:
self._obj.loc[len(self._obj)] = row
else:
self._obj.loc[len(self._obj)] = dataIt is super simple and yet does everything I want. Now I can create a dataframe and append data super easily.
df = pd.DataFrame(columns=['Name', 'Age', 'Job'])
and append data like this
df.easy.append(['Velocibadgery', 35, 'Programmer'])
It is great. I hope this helps some of you out.
Here is a Google Colab Notebook if you want to try it out
https://colab.research.google.com/drive/1sQdIPV9K9G5lqh__N0-xEAC4IFJgdtS8?usp=sharing
Edit:
Update code to allow assignment with a list of lists.