Filter, Map, Reduce and Lambdas


Filter, map, and reduce are functions that are often used in functional programming. Sometimes instead of looping, you can use one of these functions to write a simple one liner. Additionally, when using this functions, we also naturally use lambdas. So first, we’ll take a look at what are lambdas.

λ

Lambda is often confusing to beginners. But it’s really not that hard. It is so called anonymous function. In essence, lambda is function that can take any number of arguments, but it can only contain one statement, and that one statement defines what will be returned. Lambda takes the form of lambda args: expression i.e.

# In pseudocode
func = lambda args: expression

# Or, you can translate it to
def func(*args):
    return expression

Raymond Hettinger, a Python core developer, referred to lambda as a “make-a-function function” in one of his speeches. Unfortunately, I watched that video like X years ago, so I cannot tell you which one exactly is it. If anyone happens to stumble upon the vid, please send me the link, so I can properly attribute the quote.

This is really useful way of looking at the lambda, because it really is a keyword that instructs interpreter to make an anonymous function. But what is a use for it? Filter, map, and reduce are only one way to use it. But, here is a simple example to go a step further:

# We define a function. Notice that when this function is called an
# anonymous function is returned. Function is an object in Python,
# and in this case, a function object is returned. You need to make
# an extra step to call this returned object
def power(n):
    return lambda x: x ** n

# Using this function, we define two new functions, one
# to get the square of a number, and the other one to get the cube.
# On the side note, these are not so anonymous anymore - we actually
# gave them names :)
square = power(2)
cube = power(3)

# Test the type of square and cube, and you'll see that these
# are indeed function objects
type(square)  # -> <class 'function'>
type(cube)  # -> <class 'function'>

# And now, it's time for that "extra step" - to call these function objects
square(4)  # -> 16
cube(2)  # -> 8

Map

Map is exactly what the mapping is in math. It takes input, and transforms it’s values with a mapping function. map() is a built-in function in Python, so you can use it out of the box.

>>> numbers = [1, 2, 3, 4, 5]
>>> list(map(lambda x: x ** 2, numbers))
[1, 4, 9, 16, 25]
>>> 
>>> names = ['amia', 'zoe', 'danika']
>>> list(map(lambda x: x.capitalize(), names))
['Amia', 'Zoe', 'Danika']

map() returns a generator, so you have to pass it to the list() method in order to evaluate it.

Filter

Name of this function is almost self explanatory. But to be more precise, filter returns only those elements of an iterable that meet certain conditions. For example:

>>> # Hey Python, give me numbers that are greater than zero
>>> numbers = range(-10, 10)
>>> list(filter(lambda x: x > 0, numbers))
[1, 2, 3, 4, 5, 6, 7, 8, 9]

In other words, this is same as:

numbers = range(-10, 10)

filtered = []
for number in numbers:
    if number > 0:
        filtered.append(number)

filter() is also a part of Python built-in functions.

Reduce

Reduce is your function if you need a sum or product of a list. But, unlike previous two functions, reduce is not part of the built-in functions (anymore). Instead, it is located in functools library, which is part of the standard library.

>>> from functools import reduce
>>> numbers = [10, 2, 4, 17, 13]
>>> reduce(lambda x, y: x * y, numbers)
17680

Outro

Ok, you may have some questions. So let’s deal with them.

Ugh, why should we use map and filter, when we have list comprehension for that!?

You may have noticed that instead of map() and filter() you could have used list comprehension. That’s right, you could have. And this is precisely one of the reasons Guido Von Rossum, the creator of Python, wanted to get rid of these functions in Python 3. However, these functions persisted as the part of built-in functions in Python 3. Here’s what Guido said:

I think dropping filter() and map() is pretty uncontroversial; filter(P, S) is almost always written clearer as [x for x in S if P(x)], and this has the huge advantage that the most common usages involve predicates that are comparisons, e.g. x==42, and defining a lambda for that just requires much more effort for the reader (plus the lambda is slower than the list comprehension). Even more so for map(F, S) which becomes [F(x) for x in S]. Of course, in many cases you’d be able to use generator expressions instead.

I don’t see a lot of purpose to reduce(), beyond sum or product, and for the sum we already have built-in sum()!?

You’re not the only one who thinks that. In the same blog Guido laid down his opinion about reduce():

So now reduce(). This is actually the one I’ve always hated most, because, apart from a few examples involving + or *, almost every time I see a reduce() call with a non-trivial function argument, I need to grab pen and paper to diagram what’s actually being fed into that function before I understand what the reduce() is supposed to do. So in my mind, the applicability of reduce() is pretty much limited to associative operators, and in all other cases it’s better to write out the accumulation loop explicitly. […] We already have sum(); I’d happily trade reduce() for product(), so that takes care of the two most common uses.

What is the difference between lambda and a regular function?

Essentially none. It’s only a different syntax for writing a function. To cite Guido:

Why drop lambda? Most Python users are unfamiliar with Lisp or Scheme, so the name is confusing; also, there is a widespread misunderstanding that lambda can do things that a nested function can’t – I still recall Laura Creighton’s Aha!-erlebnis after I showed her there was no difference! Even with a better name, I think having the two choices side-by-side just requires programmers to think about making a choice that’s irrelevant for their program; not having the choice streamlines the thought process.

So… should I use any of these?

To each their own ¯\_(ツ)_/¯. I oftentimes find that lambdas reduce the readability of a program, and some too enthusiastic developers can misuse list comprehensions also. Before you use any of these features, including list comprehension, ask yourself, do I write them for aesthetics (to show off), or because I actually need them? Readability is really important when you work in a team, or if you have an open source software.

We're not spammers, and you can opt-out at any moment. We hate spam as much as you do.

powered by TinyLetter

See also