How about a dict comprehension:
filtered_dict = {k: v for k, v in d.iteritems() if filter_string in k}
One you see it, it should be self-explanatory, as it reads like English pretty well.
This syntax requires Python 2.7 or greater.
In Python 3, there is only dict.items(), not iteritems() so you would use:
filtered_dict = {k: v for k, v in d.items() if filter_string in k}
Answer from Jonathon Reinhart on Stack OverflowHow about a dict comprehension:
filtered_dict = {k: v for k, v in d.iteritems() if filter_string in k}
One you see it, it should be self-explanatory, as it reads like English pretty well.
This syntax requires Python 2.7 or greater.
In Python 3, there is only dict.items(), not iteritems() so you would use:
filtered_dict = {k: v for k, v in d.items() if filter_string in k}
You can use the built-in filter function to filter dictionaries, lists, etc. based on specific conditions.
filtered_dict = dict(
filter(lambda item: filter_str in item[0], d.items())
)
The advantage is that you can use it for different data structures.
Constructing a new dict:
dict_you_want = {key: old_dict[key] for key in your_keys}
Uses dictionary comprehension.
If you use a version which lacks them (ie Python 2.6 and earlier), make it dict((key, old_dict[key]) for ...). It's the same, though uglier.
Note that this, unlike jnnnnn's version, has stable performance (depends only on number of your_keys) for old_dicts of any size. Both in terms of speed and memory. Since this is a generator expression, it processes one item at a time, and it doesn't looks through all items of old_dict.
Removing everything in-place:
unwanted = set(old_dict) - set(your_keys)
for unwanted_key in unwanted: del your_dict[unwanted_key]
Slightly more elegant dict comprehension:
foodict = {k: v for k, v in mydict.items() if k.startswith('foo')}
How to filter a Python dictionary by keys containing a specific substring? - Ask a Question - TestMu AI Community
How to filter dictionary keys by substring in Python? - Ask a Question - TestMu AI (formerly LambdaTest) Community
python filter list of dictionaries based on key value - Stack Overflow
Python - Filter list of dictionaries based on if key value contains all items in another list - Stack Overflow
Videos
You can try a list comp
>>> exampleSet = [{'type':'type1'},{'type':'type2'},{'type':'type2'}, {'type':'type3'}]
>>> keyValList = ['type2','type3']
>>> expectedResult = [d for d in exampleSet if d['type'] in keyValList]
>>> expectedResult
[{'type': 'type2'}, {'type': 'type2'}, {'type': 'type3'}]
Another way is by using filter
>>> list(filter(lambda d: d['type'] in keyValList, exampleSet))
[{'type': 'type2'}, {'type': 'type2'}, {'type': 'type3'}]
Trying a few answers from this post, I tested the performance of each answer.
As my initial guess, the list comprehension is way faster, the filter and list method is second and the pandas is third, by far.
defined variables:
import pandas as pd
exampleSet = [{'type': 'type' + str(number)} for number in range(0, 1_000_000)]
keyValList = ['type21', 'type950000']
1st - list comprehension
%%timeit
expectedResult = [d for d in exampleSet if d['type'] in keyValList]
60.7 ms ± 188 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)
2nd - filter and list
%%timeit
expectedResult = list(filter(lambda d: d['type'] in keyValList, exampleSet))
94 ms ± 328 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)
3rd - pandas
%%timeit
df = pd.DataFrame(exampleSet)
expectedResult = df[df['type'].isin(keyValList)].to_dict('records')
336 ms ± 1.84 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
On a side note, using pandas to deal with a dict is not a great idea since the pandas.DataFrame is basically a more memory consuming dict and if you are not going to use a dataframe in the end it is just inefficient.
First, convert your second list to a set for more efficient comparisons:
sampleFilterSet = set(sampleFilterList)
Now, compare the 'abc' keys for each list item to the aforesaid set:
[item for item in listOfDicts if not (sampleFilterSet - item['abc'].keys())]
#[{'ID': 7, 'abc': {'123': 'foo', '456': 'bar'}}]
This is the fastest solution. A more Pythonic (but somewhat slower) solution is to use filter():
list(filter(lambda item: not (sampleFilterSet - item['abc'].keys()), listOfDicts))
#[{'ID': 7, 'abc': {'123': 'foo', '456': 'bar'}}]
You need to move the in condition test before the for keyword in the list comprehension and also use get will be more safe, which returns a default value instead of throwing an error, if you are not sure if all the dictionaries in the list have the keyword abc:
listOfDicts = [{'ID': 1, 'abc': {'123': 'foo'}}, {'ID': 7, 'abc': {'123':'foo','456': 'bar'}}]
sampleFilterList = ['123', '456']
[d for d in listOfDicts if all(s in d.get('abc', {}) for s in sampleFilterList)]
# [{'ID': 7, 'abc': {'123': 'foo', '456': 'bar'}}]
Or if use a set as of @DYZ, you can use issubset:
filterSet = set(sampleFilterList)
[d for d in listOfDicts if filterSet.issubset(d.get('abc', {}))]
# [{'ID': 7, 'abc': {'123': 'foo', '456': 'bar'}}]