It is strange how much beauty varies for different people. I find the list comprehension much clearer than filter+lambda, but use whichever you find easier.
There are two things that may slow down your use of filter.
The first is the function call overhead: as soon as you use a Python function (whether created by def or lambda) it is likely that filter will be slower than the list comprehension. It almost certainly is not enough to matter, and you shouldn't think much about performance until you've timed your code and found it to be a bottleneck, but the difference will be there.
The other overhead that might apply is that the lambda is being forced to access a scoped variable (value). That is slower than accessing a local variable and in Python 2.x the list comprehension only accesses local variables. If you are using Python 3.x the list comprehension runs in a separate function so it will also be accessing value through a closure and this difference won't apply.
The other option to consider is to use a generator instead of a list comprehension:
def filterbyvalue(seq, value):
for el in seq:
if el.attribute==value: yield el
Then in your main code (which is where readability really matters) you've replaced both list comprehension and filter with a hopefully meaningful function name.
Answer from Duncan on Stack OverflowIt is strange how much beauty varies for different people. I find the list comprehension much clearer than filter+lambda, but use whichever you find easier.
There are two things that may slow down your use of filter.
The first is the function call overhead: as soon as you use a Python function (whether created by def or lambda) it is likely that filter will be slower than the list comprehension. It almost certainly is not enough to matter, and you shouldn't think much about performance until you've timed your code and found it to be a bottleneck, but the difference will be there.
The other overhead that might apply is that the lambda is being forced to access a scoped variable (value). That is slower than accessing a local variable and in Python 2.x the list comprehension only accesses local variables. If you are using Python 3.x the list comprehension runs in a separate function so it will also be accessing value through a closure and this difference won't apply.
The other option to consider is to use a generator instead of a list comprehension:
def filterbyvalue(seq, value):
for el in seq:
if el.attribute==value: yield el
Then in your main code (which is where readability really matters) you've replaced both list comprehension and filter with a hopefully meaningful function name.
This is a somewhat religious issue in Python. Even though Guido considered removing map, filter and reduce from Python 3, there was enough of a backlash that in the end only reduce was moved from built-ins to functools.reduce.
Personally I find list comprehensions easier to read. It is more explicit what is happening from the expression [i for i in list if i.attribute == value] as all the behaviour is on the surface not inside the filter function.
I would not worry too much about the performance difference between the two approaches as it is marginal. I would really only optimise this if it proved to be the bottleneck in your application which is unlikely.
Also since the BDFL wanted filter gone from the language then surely that automatically makes list comprehensions more Pythonic ;-)
map, filter vs list comprehension. Which is faster?
Currently, in the program that I am working on, lists never have more than 50 items at any given time.
The easy answer: I doubt you care at all about the question in the title.
More on reddit.comDiffenence between List Comprehension and Map/Filter?
python - When would using the filter function be used instead of a list comprehension? - Stack Overflow
python - Confusion about efficiencies after evaluating list comprehension performance - Stack Overflow
I have heard that map and filter are slightly faster than list comprehension but that only happens when the number of items in the list is extremely large (10,000 or more). Currently, in the program that I am working on, lists never have more than 50 items at any given time.
Currently, in the program that I am working on, lists never have more than 50 items at any given time.
The easy answer: I doubt you care at all about the question in the title.
"We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil. Yet we should not pass up our opportunities in that critical 3%"
-
Donald Knuth, very famous computer scientist
How much time did you spend thinking about this? How much time will it actually save you?
Is there any significant difference between List Comprehensions like in Python or JavaScript and the Higher Order Functions "Map" and "Filter" in functional languages?
It seems that both take a list and return a new list. It's just a different syntax, like this example in Python
squares = [x**2 for x in numbers] squares = list(map(lambda x: x**2, numbers))
Semantically, they are identical. They take a list of elements, apply a function to each element, and return a new list of elements.
The only difference I've noticed is that higher order functions can be faster in languages like Haskell because they can be optimized and run on multiple cores.
Edit: ChatGPT gave me a hint about the differences. Is that correct?
Semantically, List Comprehensions and the map and filter functions actually have some similarities, since they are all used to perform transformations on elements of a list and usually return a new list. These similarities can lead to confusion, as they seem similar at first glance. Let's take a closer look at the semantics:
Transformation:
List Comprehensions: you use an expression to transform each element of the source list and create a new list with the transformed values.
Map: It applies a specified function to each element of the source list and returns a list with the transformed values.
Filtering: List Comprehensions: you can insert conditions in List Comprehensions to select or filter elements based on a condition.
filter: This function is used specifically to select elements from the source list that satisfy a certain condition.
The semantic differences are therefore:
List Comprehensions can combine both transformations and filtering in a single construction, making their syntax versatile.
map is restricted to transformations and creates a new list of transformed values.
filter specializes in selecting elements based on a condition and returns a list with the selected elements.
The following are rough guidelines and educated guesses based on experience. You should timeit or profile your concrete use case to get hard numbers, and those numbers may occasionally disagree with the below.
A list comprehension is usually a tiny bit faster than the precisely equivalent for loop (that actually builds a list), most likely because it doesn't have to look up the list and its append method on every iteration. However, a list comprehension still does a bytecode-level loop:
>>> dis.dis(<the code object for `[x for x in range(10)]`>)
1 0 BUILD_LIST 0
3 LOAD_FAST 0 (.0)
>> 6 FOR_ITER 12 (to 21)
9 STORE_FAST 1 (x)
12 LOAD_FAST 1 (x)
15 LIST_APPEND 2
18 JUMP_ABSOLUTE 6
>> 21 RETURN_VALUE
Using a list comprehension in place of a loop that doesn't build a list, nonsensically accumulating a list of meaningless values and then throwing the list away, is often slower because of the overhead of creating and extending the list. List comprehensions aren't magic that is inherently faster than a good old loop.
As for functional list processing functions: While these are written in C and probably outperform equivalent functions written in Python, they are not necessarily the fastest option. Some speed up is expected if the function is written in C too. But most cases using a lambda (or other Python function), the overhead of repeatedly setting up Python stack frames etc. eats up any savings. Simply doing the same work in-line, without function calls (e.g. a list comprehension instead of map or filter) is often slightly faster.
Suppose that in a game that I'm developing I need to draw complex and huge maps using for loops. This question would be definitely relevant, for if a list-comprehension, for example, is indeed faster, it would be a much better option in order to avoid lags (Despite the visual complexity of the code).
Chances are, if code like this isn't already fast enough when written in good non-"optimized" Python, no amount of Python level micro optimization is going to make it fast enough and you should start thinking about dropping to C. While extensive micro optimizations can often speed up Python code considerably, there is a low (in absolute terms) limit to this. Moreover, even before you hit that ceiling, it becomes simply more cost efficient (15% speedup vs. 300% speed up with the same effort) to bite the bullet and write some C.
If you check the info on python.org, you can see this summary:
Version Time (seconds)
Basic loop 3.47
Eliminate dots 2.45
Local variable & no dots 1.79
Using map function 0.54
But you really should read the above article in details to understand the cause of the performance difference.
I also strongly suggest you should time your code by using timeit. At the end of the day, there can be a situation where, for example, you may need to break out of for loop when a condition is met. It could potentially be faster than finding out the result by calling map.