You can try a list comp
>>> exampleSet = [{'type':'type1'},{'type':'type2'},{'type':'type2'}, {'type':'type3'}]
>>> keyValList = ['type2','type3']
>>> expectedResult = [d for d in exampleSet if d['type'] in keyValList]
>>> expectedResult
[{'type': 'type2'}, {'type': 'type2'}, {'type': 'type3'}]
Another way is by using filter
>>> list(filter(lambda d: d['type'] in keyValList, exampleSet))
[{'type': 'type2'}, {'type': 'type2'}, {'type': 'type3'}]
Answer from Bhargav Rao on Stack OverflowYou can try a list comp
>>> exampleSet = [{'type':'type1'},{'type':'type2'},{'type':'type2'}, {'type':'type3'}]
>>> keyValList = ['type2','type3']
>>> expectedResult = [d for d in exampleSet if d['type'] in keyValList]
>>> expectedResult
[{'type': 'type2'}, {'type': 'type2'}, {'type': 'type3'}]
Another way is by using filter
>>> list(filter(lambda d: d['type'] in keyValList, exampleSet))
[{'type': 'type2'}, {'type': 'type2'}, {'type': 'type3'}]
Trying a few answers from this post, I tested the performance of each answer.
As my initial guess, the list comprehension is way faster, the filter and list method is second and the pandas is third, by far.
defined variables:
import pandas as pd
exampleSet = [{'type': 'type' + str(number)} for number in range(0, 1_000_000)]
keyValList = ['type21', 'type950000']
1st - list comprehension
%%timeit
expectedResult = [d for d in exampleSet if d['type'] in keyValList]
60.7 ms ± 188 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)
2nd - filter and list
%%timeit
expectedResult = list(filter(lambda d: d['type'] in keyValList, exampleSet))
94 ms ± 328 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)
3rd - pandas
%%timeit
df = pd.DataFrame(exampleSet)
expectedResult = df[df['type'].isin(keyValList)].to_dict('records')
336 ms ± 1.84 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
On a side note, using pandas to deal with a dict is not a great idea since the pandas.DataFrame is basically a more memory consuming dict and if you are not going to use a dataframe in the end it is just inefficient.
How to go about filtering/querying a list of dictionaries?
How to filter dictionary keys by substring in Python? - Ask a Question - TestMu AI (formerly LambdaTest) Community
Filter a list of dicts
How to filter a nested dict by key?
Videos
Given a list of dictionaries like:
[{'first name': 'John', 'last name': 'Smith', 'age': 20, 'sport': 'basketball', 'level': 'college', 'team': 'Bruins', 'team city': 'Los Angeles'}, {'first name': 'Wayne', 'last name': 'Gretsky', 'age': 29, 'sport': 'ice hockey', 'level': 'professional', 'team': 'Kings', 'team city': 'Los Angeles'}, {'first name': 'Dan', 'last name': 'Marino', 'age': 31, 'sport': 'football', 'level': 'professional', 'team': 'Dolphins', 'team city': 'Miami'}...]Outside of using a SQL query via sqllite module, what would be the best approach of writing a script that returns a smaller list of dictionaries for a different combination of filters (ex. all players with last name of 'Smith', all players under the age of 25, all players with the last name 'Smith' who are under the age of 25, all players in professional teams in Las Vegas, all college baseball players under the age of 21 who are not from Los Angeles, etc.)?
I'm not so much asking for specific code, but what would be your thought process in structuring clean code for such a script?