I managed to solve it via iterating through the rows:

for index,row in test_df.iterrows():
    test_df.loc[index, "Score"] =  fuzz.token_set_ratio("amsterdam",test_df.loc[index,"City"])

The result is:

        City Country Code  Score
0  Amsterdam           NL    100
1  Amsterdam           NL    100
2  Rotterdam           NL     67
3     Zurich           NL     13
4     Vienna           NL     13
Answer from Dániel Molnár on Stack Overflow
🌐
Medium
medium.com › analytics-vidhya › matching-messy-pandas-columns-with-fuzzywuzzy-4adda6c7994f
Matching Messy Pandas columns with FuzzyWuzzy | by Khalid El Mouloudi | Analytics Vidhya | Medium
February 1, 2024 - In this article, I’m going to show you how to use the Python package FuzzyWuzzy to match two Pandas dataframe columns based on string similarity; the intended outcome is to have each value of column A matched with the closest corresponding value in column B, which is then put in the same row.
Discussions

python - Vectorizing or Speeding up Fuzzywuzzy String Matching on PANDAS Column - Stack Overflow
Is there a quicker way to generate ... list or PANDAS column? ... So for each row, you want to use the org_name column of that row as your query and then use the entire list of organization names from the complete org_name column as your match options? ... Yes, that is correct. ... Given your task your comparing 70k strings with each other using fuzz.WRatio, so your having a total of 4,900,000,000 comparisions, with each of these comparisions using the levenshtein distance inside fuzzywuzzy which is a ... More on stackoverflow.com
🌐 stackoverflow.com
pandas - Fuzzy Wuzzy String Matching on 2 Large Data Sets Based on a Condition - python - Stack Overflow
When I try merging these two DFs outright using pandas.merge on the address field, I get a paltry number of match compared to the number of rows. So I thought I would try to fuzzy string match to see if it improves the number of output matches. I approached this by trying to create a new column in DF1 (20K rows) that was the result of applying the fuzzywuzzy ... More on stackoverflow.com
🌐 stackoverflow.com
Can i use fuzzy wuzzy library to match string in a column in pandas and print its corresponding row?
With pandas it’s best to create a new column with the output. The apply function might be useful here. Though you are processing it for all rows before printing anything. And afterwards then you can select a subset . This allows you to shortlist the rows you want to print. More on reddit.com
🌐 r/learnpython
1
1
April 29, 2021
python - create new column in dataframe using fuzzywuzzy - Stack Overflow
I have a dataframe in pandas where I am using fuzzywuzzy package in python to match first column in the dataframe with second column. I have defined a function to create an output with first column, More on stackoverflow.com
🌐 stackoverflow.com
March 21, 2016
🌐
HAR Data Extractor
jonathansoma.com › lede › algorithms-2017 › classes › fuzziness-matplotlib › fuzzing-matching-in-pandas-with-fuzzywuzzy
Fuzzing matching in pandas with fuzzywuzzy
Fuzzy matches are incomplete or inexact matches. The Python package fuzzywuzzy has a few functions that can help you, although they’re a little bit confusing!
🌐
Bryan Ross
rossbj92.github.io › Fuzzy-Matching
Fuzzy String Matching with Pandas and fuzzywuzzy - Bryan Ross
February 8, 2020 - In actual experimental conditions where there are hundreds or thousands of participants, though, this is not so easy. Pandas has many native string methods that make cleaning text data easier (e.g., str.lower() can easily handle Oneida Cadogan and Oneida cadogan) - but instances like Ernie Grajeda and Ernie grajda are a bit more difficult.
🌐
GeeksforGeeks
geeksforgeeks.org › how-to-do-fuzzy-matching-on-pandas-dataframe-column-using-python
How to do Fuzzy Matching on Pandas Dataframe Column Using Python? - GeeksforGeeks
May 30, 2021 - Then we will convert it into pandas data frames and create two empty lists for storing the matches later than as shown below: ... from fuzzywuzzy import fuzz from fuzzywuzzy import process import pandas dict1 = {'name': ["aparna", "pankaj", "sudhir", "Geeku"]} dict2 = {'name': ["aparn", "arup", "Pankaj", "sudhir c", "Geek", "abc"]} # converting to pandas dataframes dframe1 = pd.DataFrame(dict1) dframe2 = pd.DataFrame(dict2) # empty lists for storing the # matches later mat1 = [] mat2 = [] # printing the pandas dataframes dframe1.show() dframe2.show()
Top answer
1 of 4
48

Given your task your comparing 70k strings with each other using fuzz.WRatio, so your having a total of 4,900,000,000 comparisions, with each of these comparisions using the levenshtein distance inside fuzzywuzzy which is a O(N*M) operation. fuzz.WRatio is a combination of multiple different string matching ratios that have different weights. It then selects the best ratio among them. Therefore it even has to calculate the Levenshtein distance multiple times. So one goal should be to reduce the search space by excluding some possibilities using a way faster matching algorithm. Another issue is that the strings are preprocessed to remove punctuation and to lowercase the strings. While this is required for the matching (so e.g. a uppercased word becomes equal to a lowercased one) we can do this ahead of time. So we only have to preprocess the 70k strings once. I will use RapidFuzz instead of FuzzyWuzzy here, since it is quite a bit faster (I am the author).

The following version performs more than 10 times as fast as your previous solution in my experiments and applies the following improvements:

  1. it preprocesses the strings ahead of time

  2. it passes a score_cutoff to extractOne so it can skip calculations where it already knows they can not reach this ratio

import pandas as pd, numpy as np
from rapidfuzz import process, utils

org_list = df['org_name']
processed_orgs = [utils.default_process(org) for org in org_list]

for (i, processed_query) in enumerate(processed_orgs):
    # None is skipped by extractOne, so we set the current element to None an
    # revert this change after the comparision
    processed_orgs[i] = None
    match = process.extractOne(processed_query, processed_orgs, processor=None, score_cutoff=93)
    processed_orgs[i] = processed_query
    if match:
        df.loc[i, 'fuzzy_match'] = org_list[match[2]]
        df.loc[i, 'fuzzy_match_score'] = match[1]

Here is a list of the most relevant improvements of RapidFuzz to make it faster than FuzzyWuzzy in this example:

  1. It is implemented fully in C++ while a big part of FuzzyWuzzy is implemented in Python

  2. When calculating the levenshtein distance it takes into account the score_cutoff to choose an optimized implementation based. E.g. when the length difference between the strings is to big it can exit in O(1).

  3. FuzzyWuzzy uses Python-Levenshtein to calculate the similarity between two strings, which uses a weightened Levenshtein distance with a weight of 2 for substitutions. This is implemented using Wagner-Fischer. RapidFuzz on the other hand uses a bitparallel implementation for this based on BitPal, which is faster

  4. fuzz.WRatio is combining the results of multiple other string matching algorithms like fuzz.ratio, fuzz.token_sort_ratio and fuzz.token_set_ratio and takes the maximum result after weighting them. So while fuzz.ratio has a weighting of 1 fuzz.token_sort_ratio and fuzz.token_set_ratio have one of 0.95. When the score_cutoff is bigger than 95 fuzz.token_sort_ratio and fuzz.token_set_ratio are not calculated anymore, since the results are guaranteed to be smaller than the score_cutoff

  5. In process.extractOne RapidFuzz avoids calls through Python whenever possible and preprocesses the query once ahead of time. E.g. the BitPal algorithm requires one of the two strings which are compared to be stored into a bitvector which takes a big part of the algorithms runtime. In process.extractOne the query is stored into this bitvector only once and the bitvector is reused afterwards making the algorithm a lot faster.

  6. since extractOne only searches for the best match it uses the ratio of the current best match as score_cutoff for the next elements. This way it can quickly discard more elements by using the improvements to the levenshtein distance calculation from 2) in many cases. When it finds a element with a similarity of 100 it exits early since there can't be a better match afterwards.

2 of 4
5

This solution leverages apply() and should demonstrate reasonable performance improvements. Feel free to play around with the scorer and change the threshold to meet your needs:

import pandas as pd, numpy as np
from fuzzywuzzy import process, fuzz

df = pd.DataFrame([['cliftonlarsonallen llp minneapolis MN'],
        ['loeb and troper llp newyork NY'],
        ["dauby o'connor and zaleski llc carmel IN"],
        ['wegner cpas llp madison WI']],
        columns=['org_name'])

org_list = df['org_name']

threshold = 40

def find_match(x):

  match = process.extract(x, org_list, limit=2, scorer=fuzz.partial_token_sort_ratio)[1]
  match = match if match[1]>threshold else np.nan
  return match

df['match found'] = df['org_name'].apply(find_match)

Returns:

                                   org_name                                     match found
0     cliftonlarsonallen llp minneapolis MN             (wegner cpas llp madison WI, 50, 3)
1            loeb and troper llp newyork NY             (wegner cpas llp madison WI, 46, 3)
2  dauby o'connor and zaleski llc carmel IN                                             NaN
3                wegner cpas llp madison WI  (cliftonlarsonallen llp minneapolis MN, 50, 0)

If you would just like to return the matching string itself, then you can modify as follows:

match = match[0] if match[1]>threshold else np.nan

I've added @user3483203's comment pertaining to a list comprehension here as an alternative option as well:

df['match found'] = [find_match(row) for row in df['org_name']]

Note that process.extract() is designed to handle a single query string and apply the passed scoring algorithm to that query and the supplied match options. For that reason, you will have to evaluate that query against all 70,000 match options (the way you currently have your code setup). So therefore, you will be evaluating len(match_options)**2 (or 4,900,000,000) string comparisons. Therefore, I think the best performance improvements could be achieved by limiting the potential match options via more extensive logic in the find_match() function, e.g. enforcing that the match options start with the same letter as the query, etc.

Find elsewhere
🌐
Understanding Data
understandingdata.com › python-for-seo › keyword-de-duplication-with-pandas-fuzzy-wuzzy
De-duplicating Keywords with Python, Pandas And Fuzzy Wuzzy - Just Understanding Data
January 27, 2023 - Performing standard data analysis operations within Pandas on a keyword report from Ahrefs (including GroupBy Objects, DataFrame Subsetting, the .drop_duplicates() method, the .apply() method and how to save your new dataframe to a .CSV file). Learn how to create groups of keywords using FuzzyWuzzy + a custom keyword grouping function.
🌐
Towards Data Science
towardsdatascience.com › home › latest › how to do fuzzy string matching in pandas dataframes
How to Do Fuzzy String Matching in Pandas Dataframes | Towards Data Science
January 20, 2025 - Along with pandas, you could use "thefuzz" to do fuzzy string matching. ... TheFuzz is an open-source Python package formally known as "FuzzyWuzzy." It uses the Levenshtein edit distance to calculate the similarity string similarity.
🌐
Cosmic Coding
cosmiccoding.com.au › tutorials › fuzzy_string_matching
Fuzzy String Matching - Samuel Hinton
August 9, 2020 - fuzzywuzzy is a great tool for matching strings based on their Levenshtein distance. The ones we touched on are: ... But there are plenty more, so go check out the library! For your convenience, here’s the code in one block: import pandas as pd import numpy as np df = pd.read_csv("fuzzy_...
🌐
PyPI
pypi.org › project › fuzzypanda
fuzzypanda · PyPI
August 25, 2019 - This fuzzy column creation approach applies a Pandas-friendly wrapper around the Symspell Symmetric Delete spelling correction algorithm to allow substantially faster fuzzy joining. Tools such as fuzzywuzzy will run in Omega(mn) to find the best-matching strings in a column of n values compared to the m values of another column, whereas this model is expected to have a runtime of Omega(m + n) due to the pre-processing of the right DataFrame columns as a spellchecker corpus that searched using the Symmetric Delete spelling correction algorithm.
      » pip install fuzzypanda
    
Published   Aug 25, 2019
Version   0.1.1
🌐
TutorialsPoint
tutorialspoint.com › how-to-do-fuzzy-matching-on-pandas-dataframe-column-using-python
How to do Fuzzy Matching on Pandas Dataframe Column Using Python?
September 9, 2021 - import pandas as pd from fuzzywuzzy import fuzz from fuzzywuzzy import process # dictionaries d1 = {'Car': ["BMW", "Audi", "Lexus", "Mercedes", "Rolls"]} d2 = {'Car': ["BM", "Audi", "Le", "MERCEDES", "Rolls Royce"]} # convert dictionaries to pandas dataframes df1 = pd.DataFrame(d1) df2 = pd.DataFrame(d2) # printing the pandas dataframes print("Dataframe 1 = ",df1) print("Dataframe 2 = ",df2) # empty lists for storing the matches later match1 = [] match2 = [] k = [] # converting dataframe column to list of elements for fuzzy matching myList1 = df1['Car'].tolist() myList2 = df2['Car'].tolist() t
🌐
Predictivehacks
predictivehacks.com › fuzzy-joins-tutorial
Fuzzy Joins Tutorial – Predictive Hacks
We will work with the FuzzyWuzzy and textdistance libraries. Assume that you are dealing with two different data frames and you want to match them on the string. Below we provide the two hypothetical data frames: import pandas as pd from fuzzywuzzy import fuzz from fuzzywuzzy import process ...
🌐
Towards Data Science
towardsdatascience.com › fuzzywuzzy-find-similar-strings-within-one-column-in-a-pandas-data-frame-99f6c2a0c212
FuzzyWuzzy: Find Similar Strings within one column in Python | Towards Data Science
March 30, 2022 - This article presents how I apply FuzzyWuzzy package to find similar ramen brand names in a ramen review dataset (full Jupyter Notebook can be found on my GitHub). You may also find functions to shorten your codes here. import pandas as pd import numpy as np from fuzzywuzzy import process, fuzzramen = pd.read_excel('The-Ramen-Rater-The-Big-List-1-3400-Current- As-Of-Jan-25-2020.xlsx') ramen.head()
🌐
Medium
medium.com › @whyamit101 › understanding-pandas-fuzzy-match-e95299709be5
Understanding pandas fuzzy match. The biggest lie in data science? That… | by why amit | Medium
April 12, 2025 - What is pandas fuzzy match? Pandas fuzzy match is a technique used to compare strings in a pandas DataFrame to identify similar entries, even if they aren’t identical, using libraries like fuzzywuzzy.
🌐
Medium
medium.com › @chrostowski.pawel › fuzzywuzzy-and-pandas-similarities-in-data-c6b24caf7b87
FuzzyWuzzy and Pandas : similarities in data. | by Paweł | Medium
September 15, 2020 - In this article I’ll present how to combine fuzzy string matching with Pandas dataframe, but before you start please read “Fuzzy string matching in python” : https://pypi.org/project/fuzzywuzzy/ if you are not familliar with this library.
🌐
Kaggle
kaggle.com › code › ruthgn › data-cleaning-integration-pandas-fuzzywuzzy
🍺 Data Cleaning & Integration - Pandas+FuzzyWuzzy | Kaggle
November 12, 2021 - Explore and run machine learning code with Kaggle Notebooks | Using data from multiple data sources
🌐
GitHub
github.com › cldeluna › fuzzy_wuzzy_examples
GitHub - cldeluna/fuzzy_wuzzy_examples: Example of Fuzzy Wuzzy Module and Pandas · GitHub
Example of Fuzzy Wuzzy Module and Pandas. Contribute to cldeluna/fuzzy_wuzzy_examples development by creating an account on GitHub.
Author   cldeluna
🌐
Reddit
reddit.com › r/learnpython › can i use fuzzy wuzzy library to match string in a column in pandas and print its corresponding row?
r/learnpython on Reddit: Can i use fuzzy wuzzy library to match string in a column in pandas and print its corresponding row?
April 29, 2021 -

Hello learning python. I'm interested in matching user input to the closest values in a column and therefore print the whole row that was matched. For example

if i have a data frame

Title Release IMDB link

A user will input a term and using fuzzy wuzzy i can reference the corresponding Title but i don't know how to print the other columns.

print(process.extractOne(user_input, df['Title']))

output is (matched_title, score_rating,  index_of_record)

I need the the whole row or preferably a way to print the IMDB link specifically.any help will be appreciated