When I fix your code so that the list comprehension and the call to filter are actually doing the same work things change a whole lot:

import time

S=[x for x in range(1000000)]
T=[y**2 for y in range(300)]
#
#
time1 = time.time()
N=[x for x in T if x in S]
time2 = time.time()
print 'time diff [x for x in T if x in S]=', time2-time1
#print N
#
#
time1 = time.time()
N=filter(lambda x:x in S,T)
time2 = time.time()
print 'time diff filter(lambda x:x in S,T)=', time2-time1
#print N

Then the output is more like:

time diff [x for x in T if x in S]= 0.414485931396
time diff filter(lambda x:x in S,T)= 0.466315984726

So the list comprehension has a time that's generally pretty close to and usually less than the lambda expression.

The reason lambda expressions are being phased out is that many people think they are a lot less readable than list comprehensions. I sort of reluctantly agree.

This is a somewhat religious issue in Python. Even though Guido considered removing map, filter and reduce from Python 3, there was enough of a backlash that in the end only reduce was moved from built-ins to functools.reduce.

Personally I find list comprehensions easier to read. It is more explicit what is happening from the expression [i for i in list if i.attribute == value] as all the behaviour is on the surface not inside the filter function.

I would not worry too much about the performance difference between the two approaches as it is marginal. I would really only optimise this if it proved to be the bottleneck in your application which is unlikely.

Also since the BDFL wanted filter gone from the language then surely that automatically makes list comprehensions more Pythonic ;-)

This is slower than DavidE's answer (I timed it), but has the advantage of simplicity:

z = dict(foo)['b']

Of course it assumes that your keys are all hashable, but that's fine if they are strings. If you need to do multiple lookups though this is definitely the way to go (just be sure to only convert to a dict once).

TLDR: A comprehension evaluates its entire expression every time. A map evaluates its expression once, and applies the result every time.

The expression fed to map and comprehensions are treated differently. The important parts of what map and comprehensions are doing can be translated in the following way:

def map(func: 'callable', iterable):
    for item in iterable:
        yield func(item)

def comprehension(expr: 'code', iterable):
    for item in iterable:
        yield eval(expr)

The important difference is that map already receives a function, whereas comprehension receives an expression.

Now, lambda x:x is an expression that evaluates to a function.

>>> co = compile('lambda x: x', '<stackoverflow>', 'eval')
>>> co
<code object <module> at 0x105d1c810, file "<stackoverflow>", line 1>
>>> eval(co)
<function __main__.<lambda>(x)>

Notably, evaluating the expression to get the function is an action that takes time.

Since map and comprehensions expect different things, the lambda is passed to them differently. What the interpreter does can be expanded with explicit eval/compile:

>>> list(map(eval(compile('lambda x: x', '<stackoverflow>', 'eval')), range(10)))
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
>>> list(comprehension(compile('(lambda x: x)(item)', '<stackoverflow>', 'eval'), range(10))
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

The important difference should be clear now:

• For map, the expression is evaluated once before it is passed into map. For each item, map calls the resulting function.
• For comprehension, the expression is passed in unevaluated. For each item, comprehension constructs the function, then calls the resulting function.

It is important to note that map is not always faster than a comprehension. map benefits from creating/looking up a function once. However, function calls are expensive and a comprehension does not have to use a function.

In general, if the expression uses only names local to the comprehension, it is faster.

In [302]: %timeit list(map(lambda i:i,range(100000)))
     ...: %timeit [i for i in range(100000)]
8.86 ms ± 38.1 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
4.49 ms ± 35.8 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)

If the expression itself has to dynamically do an expensive operation, map does not benefit.

In [302]: %timeit list(map(lambda i: (lambda x:x)(i),range(100000)))
     ...: %timeit [(lambda x:x)(i) for i in range(100000)]
19.7 ms ± 39.7 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
15.9 ms ± 43.5 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)