Indeed, you’ve astutely identified that the solution with a for loop is not going to be very efficient especially for large dataframes like the one you’re using, due to the quadratic complexity of the search as well as the inherent efficiency of using a for-loop rather than built-in vectorized panda… Answer from CAM-Gerlach on discuss.python.org
Top answer
1 of 12
834

TL;DR version:

For the simple case of:

  • I have a text column with a delimiter and I want two columns

The simplest solution is:

df[['A', 'B']] = df['AB'].str.split(' ', n=1, expand=True)

You must use expand=True if your strings have a non-uniform number of splits and you want None to replace the missing values.

Notice how, in either case, the .tolist() method is not necessary. Neither is zip().

In detail:

Andy Hayden's solution is most excellent in demonstrating the power of the str.extract() method.

But for a simple split over a known separator (like, splitting by dashes, or splitting by whitespace), the .str.split() method is enough1. It operates on a column (Series) of strings, and returns a column (Series) of lists:

>>> import pandas as pd
>>> df = pd.DataFrame({'AB': ['A1-B1', 'A2-B2']})
>>> df

      AB
0  A1-B1
1  A2-B2
>>> df['AB_split'] = df['AB'].str.split('-')
>>> df

      AB  AB_split
0  A1-B1  [A1, B1]
1  A2-B2  [A2, B2]

1: If you're unsure what the first two parameters of .str.split() do, I recommend the docs for the plain Python version of the method.

But how do you go from:

  • a column containing two-element lists

to:

  • two columns, each containing the respective element of the lists?

Well, we need to take a closer look at the .str attribute of a column.

It's a magical object that is used to collect methods that treat each element in a column as a string, and then apply the respective method in each element as efficient as possible:

>>> upper_lower_df = pd.DataFrame({"U": ["A", "B", "C"]})
>>> upper_lower_df

   U
0  A
1  B
2  C
>>> upper_lower_df["L"] = upper_lower_df["U"].str.lower()
>>> upper_lower_df

   U  L
0  A  a
1  B  b
2  C  c

But it also has an "indexing" interface for getting each element of a string by its index:

>>> df['AB'].str[0]

0    A
1    A
Name: AB, dtype: object

>>> df['AB'].str[1]

0    1
1    2
Name: AB, dtype: object

Of course, this indexing interface of .str doesn't really care if each element it's indexing is actually a string, as long as it can be indexed, so:

>>> df['AB'].str.split('-', 1).str[0]

0    A1
1    A2
Name: AB, dtype: object

>>> df['AB'].str.split('-', 1).str[1]

0    B1
1    B2
Name: AB, dtype: object

Then, it's a simple matter of taking advantage of the Python tuple unpacking of iterables to do

>>> df['A'], df['B'] = df['AB'].str.split('-', n=1).str
>>> df

      AB  AB_split   A   B
0  A1-B1  [A1, B1]  A1  B1
1  A2-B2  [A2, B2]  A2  B2

Of course, getting a DataFrame out of splitting a column of strings is so useful that the .str.split() method can do it for you with the expand=True parameter:

>>> df['AB'].str.split('-', n=1, expand=True)

    0   1
0  A1  B1
1  A2  B2

So, another way of accomplishing what we wanted is to do:

>>> df = df[['AB']]
>>> df

      AB
0  A1-B1
1  A2-B2

>>> df.join(df['AB'].str.split('-', n=1, expand=True).rename(columns={0:'A', 1:'B'}))

      AB   A   B
0  A1-B1  A1  B1
1  A2-B2  A2  B2

The expand=True version, although longer, has a distinct advantage over the tuple unpacking method. Tuple unpacking doesn't deal well with splits of different lengths:

>>> df = pd.DataFrame({'AB': ['A1-B1', 'A2-B2', 'A3-B3-C3']})
>>> df
         AB
0     A1-B1
1     A2-B2
2  A3-B3-C3
>>> df['A'], df['B'], df['C'] = df['AB'].str.split('-')
Traceback (most recent call last):
  [...]    
ValueError: Length of values does not match length of index
>>> 

But expand=True handles it nicely by placing None in the columns for which there aren't enough "splits":

>>> df.join(
...     df['AB'].str.split('-', expand=True).rename(
...         columns={0:'A', 1:'B', 2:'C'}
...     )
... )
         AB   A   B     C
0     A1-B1  A1  B1  None
1     A2-B2  A2  B2  None
2  A3-B3-C3  A3  B3    C3
2 of 12
188

There might be a better way, but this here's one approach:

                            row
    0       00000 UNITED STATES
    1             01000 ALABAMA
    2  01001 Autauga County, AL
    3  01003 Baldwin County, AL
    4  01005 Barbour County, AL
df = pd.DataFrame(df.row.str.split(' ',1).tolist(),
                                 columns = ['fips','row'])
   fips                 row
0  00000       UNITED STATES
1  01000             ALABAMA
2  01001  Autauga County, AL
3  01003  Baldwin County, AL
4  01005  Barbour County, AL
Discussions

python - Splitting dataframe into multiple dataframes - Stack Overflow
Find the answer to your question by asking. Ask question ... See similar questions with these tags. ... Making the OWASP top ten in the vibe code... ... Help Shape the 2026 Developer Survey! ... 38 Python - splitting dataframe into multiple dataframes based on column values and naming them with those values · 0 Finding values in a column that are same and making separate dataframes of them · -1 Create a new Pandas ... More on stackoverflow.com
🌐 stackoverflow.com
Pandas split dataframe vertically
Are your column names actually that simple? t_cols = [c for c in df.columns if 'target' in c] etc seems like a good way to isolate the column names. df[t_cols] is then just a frame with those columns. Are they actually split left and right? df.iloc would settle that pretty nicely. As always, refer to the docs for more information. More on reddit.com
🌐 r/learnpython
5
1
October 10, 2016
Noob question: Split pandas dataframe based on unique column values
Groupby creates dataframes based on the unique values. grouped = Cost_Per.groupby(by='Sector') Then you could create a function that takes a dataframe and process the data e.g. makes and saves a chart. And make the function return None (on mobile so will test later) grouped.apply(make_chart)# this will run the make_chart twice with the first group grouped.aggregate(make_chart) The make_chart function should create a new chart, save it and then clear it (or delete) def make_chart(df): plt.plot(df.iloc[:, 0], df.iloc[:, 1]) # make plot with first and second column group_id = df['Sector'].iloc[0] plt.savefig("my_plot_{}".format(group_id), dpi=300, bbox_inches='tight') plt.close() # if more than one figure is used -> plt.close('all') return None # this line can be omitted, because python will return None if return is not given edit. ok, did some testing (apply vs aggregate, make_chart func) More on reddit.com
🌐 r/learnpython
2
1
June 18, 2016
Help splitting a pandas dataframe column into two?
After playing around with it a bit, here's what I came up with. Pretty ugly though. In [3]: df = pd.DataFrame({'years': ['1900-2000', '1920-2020', '1930-2030']}) In [4]: df Out[4]: years 0 1900-2000 1 1920-2020 2 1930-2030 In [14]: df['start'], df['end'] = zip(*df['years'].map(lambda x: x.split('-'))) In [15]: df Out[15]: years start end 0 1900-2000 1900 2000 1 1920-2020 1920 2020 2 1930-2030 1930 2030 More on reddit.com
🌐 r/learnpython
5
5
February 8, 2016
🌐
Spark By {Examples}
sparkbyexamples.com › home › pandas › split pandas dataframe by column value
Split Pandas DataFrame by Column Value - Spark By {Examples}
June 27, 2025 - The Pandas DataFrame can be split into smaller DataFrames based on either single or multiple-column values. Pandas provide various features and functions
🌐
GeeksforGeeks
geeksforgeeks.org › python › split-pandas-dataframe-by-column-index
Split Pandas Dataframe by Column Index - GeeksforGeeks
July 15, 2025 - import pandas as pd dataset = ... columns (toothed, hair, breathes, legs) for this we are going to make use of the iloc[rows, columns] method offered by pandas....
🌐
GeeksforGeeks
geeksforgeeks.org › python › split-pandas-dataframe-by-column-value
Split Pandas Dataframe by column value - GeeksforGeeks
July 15, 2025 - In the above example, the data frame 'df' is split into 2 parts 'df1' and 'df2' on the basis of values of column 'Age'. ... # importing pandas library import pandas as pd # Initializing the nested list with Data-set player_list = [['M.S.Dhoni', 36, 75, 5428000], ['A.B.D Villiers', 38, 74, 3428000], ['V.Kholi', 31, 70, 8428000], ['S.Smith', 34, 80, 4428000], ['C.Gayle', 40, 100, 4528000], ['J.Root', 33, 72, 7028000], ['K.Peterson', 42, 85, 2528000]] # creating a pandas dataframe df = pd.DataFrame(player_list, columns = ['Name', 'Age', 'Weight', 'Salary']) # splitting the dataframe into 2 parts # on basis of 'Weight' column values mask = df['Weight'] >= 80 df1 = df[mask] # invert the boolean values df2 = df[~mask] # printing df1 df1
🌐
Statology
statology.org › home › pandas: how to split dataframe by column value
Pandas: How to Split DataFrame By Column Value
November 29, 2021 - We can use the following code to split the DataFrame into two DataFrames where the first contains the rows where ‘points’ is greater than or equal to 20 and the second contains the rows where ‘points’ is less than 20: #define value to split on x = 20 #define df1 as DataFrame where 'points' is >= 20 df1 = df[df['points'] >= x] print(df1) team points rebounds 0 A 22 11 1 B 24 8 5 F 29 5 6 G 31 9 #define df2 as DataFrame where 'points' is < 20 df2 = df[df['points'] < x] print(df2) team points rebounds 2 C 19 10 3 D 18 6 4 E 14 6 7 H 16 12
Find elsewhere
🌐
Pandas
pandas.pydata.org › docs › reference › api › pandas.Series.str.split.html
pandas.Series.str.split — pandas 3.0.4 documentation - PyData |
If NaN is present, it is propagated throughout the columns during the split. >>> s.str.split(expand=True) 0 1 2 3 4 0 this is a regular sentence 1 https://docs.python.org/3/tutorial/index.html NaN NaN NaN NaN 2 NaN NaN NaN NaN NaN · For slightly more complex use cases like splitting the html document name from a url, a combination of parameter settings can be used.
🌐
datagy
datagy.io › home › pandas tutorials › pandas dataframes › python: split a pandas dataframe
Python: Split a Pandas Dataframe • datagy
December 16, 2022 - Learn how to split a Pandas dataframe in Python. Split a dataframe by column value, by position, and by random values.
🌐
Spark By {Examples}
sparkbyexamples.com › home › pandas › how to split pandas dataframe?
How to Split Pandas DataFrame? - Spark By {Examples}
December 6, 2024 - You can split the Pandas DataFrame based on rows or columns by using Pandas.DataFrame.iloc[] attribute, groupby().get_group(), sample() functions. It
Top answer
1 of 13
113

Can I ask why not just do it by slicing the data frame. Something like

#create some data with Names column
data = pd.DataFrame({'Names': ['Joe', 'John', 'Jasper', 'Jez'] *4, 'Ob1' : np.random.rand(16), 'Ob2' : np.random.rand(16)})

#create unique list of names
UniqueNames = data.Names.unique()

#create a data frame dictionary to store your data frames
DataFrameDict = {elem : pd.DataFrame() for elem in UniqueNames}

for key in DataFrameDict.keys():
    DataFrameDict[key] = data[:][data.Names == key]

Hey presto you have a dictionary of data frames just as (I think) you want them. Need to access one? Just enter

DataFrameDict['Joe']
2 of 13
79

Firstly your approach is inefficient because the appending to the list on a row by basis will be slow as it has to periodically grow the list when there is insufficient space for the new entry, list comprehensions are better in this respect as the size is determined up front and allocated once.

However, I think fundamentally your approach is a little wasteful as you have a dataframe already so why create a new one for each of these users?

I would sort the dataframe by column 'name', set the index to be this and if required not drop the column.

Then generate a list of all the unique entries and then you can perform a lookup using these entries and crucially if you only querying the data, use the selection criteria to return a view on the dataframe without incurring a costly data copy.

Use pandas.DataFrame.sort_values and pandas.DataFrame.set_index:

# sort the dataframe
df.sort_values(by='name', axis=1, inplace=True)

# set the index to be this and don't drop
df.set_index(keys=['name'], drop=False,inplace=True)

# get a list of names
names=df['name'].unique().tolist()

# now we can perform a lookup on a 'view' of the dataframe
joe = df.loc[df.name=='joe']

# now you can query all 'joes'
🌐
GeeksforGeeks
geeksforgeeks.org › python › split-dataframe-in-pandas-based-on-values-in-multiple-columns
Split dataframe in Pandas based on values in multiple columns - GeeksforGeeks
July 23, 2025 - In this article, we are going to see how to divide a dataframe by various methods and based on various parameters using Python. To divide a dataframe into two or more separate dataframes based on the values present in the column we first create a data frame. ... # importing pandas as pd import pandas as pd # dictionary of lists dict = {'First_Name': ["Aparna", "Pankaj", "Sudhir", "Geeku", "Anuj", "Aman", "Madhav", "Raj", "Shruti"], 'Last_Name': ["Pandey", "Gupta", "Mishra", "Chopra", "Mishra", "Verma", "Sen", "Roy", "Agarwal"], 'Email_ID': ["apandey@gmail.com", "pankaj@gmail.com", "sumishra23@
🌐
Skytowner
skytowner.com › explore › splitting_pandas_dataframe_based_on_column_values
Splitting Pandas DataFrame based on column values
To split a Pandas DataFrame based on column values, first build a mask of booleans that indicate rows where condition is satisfied. Next, use df[mask] and df[~mask] to obtain two separate DataFrames.
🌐
Spark By {Examples}
sparkbyexamples.com › home › pandas › pandas split column into two columns
Pandas Split Column into Two Columns - Spark By {Examples}
November 13, 2024 - Pandas Series.str.the split() function is used to split the one-string column value into two columns based on a specified separator or delimiter. This
🌐
Arab Psychology
scales.arabpsychology.com › home › how can i split a pandas dataframe based on a specific column value?
How Can I Split A Pandas DataFrame Based On A Specific Column Value?
July 2, 2024 - The process of splitting a Pandas ... on the unique values in the specified column. This can be achieved by using the “groupby” function in Pandas, which allows for grouping the data by a specific column....
🌐
Arab Psychology
scales.arabpsychology.com › psychological scales › how to split a pandas dataframe by column value ?
How To Split A Pandas DataFrame By Column Value ?
November 15, 2023 - It can be easily done by using the groupby() function combined with the appropriate column name. This will group the DataFrame by the given column, and then we can apply an aggregate function to each group to obtain the desired result.
🌐
Hackers and Slackers
hackersandslackers.com › splitting-columns-with-pandas
Splitting Columns With Pandas
November 19, 2019 - The full name of the department and the department's acronym is currently in the same column, which could become a problem. For instance, we might have an output from a database table with department info that we'd like to merge it with, and it just uses the acronym as its foreign key. And let's also say that we're ingesting this as part of a long function chain and don't want to break our stride - so let's put that all in one convenient function. def assign_split_col(df: pd.DataFrame, col: str, name_list: List[str], pat: str=None): df = df.copy() split_col = df[col].str.split(pat, expand=True) return df.assign( **dict( zip(name_list, [split_col.iloc[:, x] for x in range(split_col.shape[1])]) ) )
🌐
Finxter
blog.finxter.com › home › learn python blog › 5 best ways to split pandas dataframe column values
5 Best Ways to Split Pandas DataFrame Column Values - Be on the Right Side of Change
February 19, 2024 - This approach provides more flexibility because any kind of Python function can be applied to each value in the column, which is particularly useful for more complex splitting logic. ... import pandas as pd # Sample DataFrame df = pd.DataFrame({'Name': ['John Doe', 'Jane Smith']}) # Split the 'Name' column into two new columns df[['First Name', 'Last Name']] = df.apply(lambda row: pd.Series(row['Name'].split(' ')), axis=1) print(df)
🌐
Saturn Cloud
saturncloud.io › blog › how-to-split-pandas-dataframe-column-values-in-python
How to Split Pandas Dataframe Column Values in Python | Saturn Cloud Blog
May 1, 2026 - Pandas dataframe columns can contain different types of data such as text, numbers, and dates. Each column can have a specific data type, such as string, integer, float, or datetime. The data type determines how the column values are stored and how operations can be performed on the column. For example, a column with string values can be manipulated using string methods such as split(), strip(), and replace().
🌐
Saturn Cloud
saturncloud.io › blog › how-to-split-one-column-into-multiple-columns-in-pandas-dataframe
How to Split One Column into Multiple Columns in Pandas DataFrame | Saturn Cloud Blog
May 1, 2026 - We want to split this column into two separate columns, one for first names and one for last names. One way to split a column into multiple columns is by using the str.split() method in Pandas.