# Map/Reduce/Filter/Apply

Great resource - http://www.u.arizona.edu/~erdmann/mse350/topics/list_comprehensions.html

Other reference (python2): http://www.python-course.eu/lambda.php

Python3 version: http://www.python-course.eu/python3_lambda.php

## Map

## Filter

lines = list(filter(lambda x: x.startswith(‘[‘), results))

Map, Filter, Lambda, and List Comprehensions in Python Author: R.G. Erdmann

map(), filter(), lambda, and list comprehensions provide compact, elegant, and efficient ways to encode a few common idioms in programming. We often encounter the following scanarios involving for-loops: Some for-loop examples to rewrite more compactly

```
Building up a list from scratch by looping over a sequence and performing some calculation on each element in the sequence.
For example, suppose we want to build a list of the squares of the integers from 0 to 9:
>>> squares = [] # start with an empty list
>>> for x in range(10): # step over every element in the list of integers from 0 to 9
... squares.append(x**2) # grow the list one element at a time
...
>>> print squares
[0, 1, 4, 9, 16, 25, 36, 49, 64, 81]
Suppose we want to build a list of the lengths of the names in a list:
>>> names = ['Anne', 'Amy', 'Bob', 'David', 'Carrie', 'Barbara', 'Zach']
>>> lengths = []
>>> for name in names:
... lengths.append(len(name))
...
>>> print lengths
[4, 3, 3, 5, 6, 7, 4]
Building up a list from scratch by looping over nested sequences.
For example, suppose we want a list of all possible pairs of drink and food from the lists ['water', 'tea', 'juice'] and ['ham', 'eggs', 'spam'], respectively:
>>> possible_choices = []
>>> for drink in ['water', 'tea', 'juice']:
... for food in ['ham', 'eggs', 'spam']:
... possible_choices.append([drink,food])
...
>>> print possible_choices
[['water', 'ham'], ['water', 'eggs'], ['water', 'spam'], ['tea', 'ham'], ['tea', 'eggs'], ['tea', 'spam'], ['juice', 'ham'], ['juice', 'eggs'], ['juice', 'spam']]
Suppose we want a list of coordinates on a rectangular grid:
>>> coords = []
>>> for x in range(5):
... for y in range(3):
... coordinate = (x, y)
... coords.append(coordinate)
...
>>> print coords
[(0, 0), (0, 1), (0, 2), (1, 0), (1, 1), (1, 2), (2, 0), (2, 1), (2, 2), (3, 0), (3, 1), (3, 2), (4, 0), (4, 1), (4, 2)]
Building a list from scratch by filtering a sequence according to some criterion or criteria.
For example, suppose we want a list of the squares of the integers from 0 to 9 where the square is greater than 5 and less than 50:
>>> special_squares = []
>>> for x in range(10):
... square = x**2
... if square > 5 and square < 50:
... special_squares.append(square)
...
>>> print special_squares
[9, 16, 25, 36, 49]
Suppose we want to take a list of names and find only those starting with the letter B:
>>> names = ['Anne', 'Amy', 'Bob', 'David', 'Carrie', 'Barbara', 'Zach']
>>> b_names = []
>>> for name in names:
... if name.startswith('B'):
... b_names.append(name)
...
>>> print b_names
['Bob', 'Barbara']
```

Map, Lambda, and Filter

One way to achieve the same goals as in the above examples is to use some of Python’s tools from functional programming: map(), filter(), and lambda(). map()

The map() function applies a function to every member of an iterable and returns the result. Typically, one would use an anonymous inline function as defined by lambda, but it is possible to use any function. The first example above can also be accomplished using map() as follows:

def square(x): … return x**2 … squares = map(square, range(10)) print squares [0, 1, 4, 9, 16, 25, 36, 49, 64, 81]

In the case of the second example, the len() function already exists, so we can use map() to apply it to every element of the list:

names = [‘Anne’, ‘Amy’, ‘Bob’, ‘David’, ‘Carrie’, ‘Barbara’, ‘Zach’] lengths = map(len, names) print lengths [4, 3, 3, 5, 6, 7, 4]

lambda

In the first map example above, we created a function, called square, so that map would have a function to apply to the sequence. This is a common occurrence, so Python provides the ability to create a simple (no statements allowed internally) anonymous inline function using a so-called lambda form. Thus, an anonymous function that returns the square of its argument can be written as lambda x: x**2. This means, “Here is an anonymous (nameless) function that takes one arguement, called x, and returns the square of x. Since the lambda form actually evaluates to a function, we can also call it inline. (This is generally a silly thing to do, but we show it here to demonstrate that lambda forms are actually inline function objects.):

print (lambda x: x**2)(5) # first parens contain a lambda form that is a squaring function, second parens represent calling that function 25

## Make a function of two arguments (x and y) that returns their product, then call the function with 3 and 4:

print (lambda x, y: x*y)(3, 4) 12 print (lambda x: x.startswith(‘B’))(‘Bob’) True print (lambda x: x.startswith(‘B’))(‘Robert’) False incrementer = lambda input: input+1

## above is same as def incrementer(input): return input+1

## now we call it

print incrementer(3) 4

Using lambda and map together

Lambda forms can be used as the required function argument to the map() function. For example, the first example above can be written as

squares = map(lambda x: x**2, range(10)) print squares [0, 1, 4, 9, 16, 25, 36, 49, 64, 81

In English, this means, “Apply a function that squares its argument (the lambda form) to every member of the list of integers from 0 to 9 (the range()), and store the result in a list called squares. filter()

The fifth and sixth examples above can also be achieved with the filter() built-in function. Filter takes a function returning True or False and applies it to a sequence, returning a list of only those members of the sequence for which the function returned True.

Lambda forms can also be used with the filter function; in fact, they can be used anywhere a function is expected in Python. In the fifth example, the list of squares is filtered according to whether the given entries are greater than 5 and less than 50. A lambda form that returns True when this condition is met is lambda x: x > 5 and x < 50. Thus, we can reproduce the fifth example as follows:

squares = map(lambda x: x**2, range(10)) special_squares = filter(lambda x: x > 5 and x < 50, squares) print special_squares [9, 16, 25, 36, 49]

In English, this means, “Find every member of the squares list for which the member is greater than 5 and less than 50 (every member for which lambda x: x > 5 and x < 50 returns True), and store that in a new variable called special_squares.

Similarly, the sixth example

names = [‘Anne’, ‘Amy’, ‘Bob’, ‘David’, ‘Carrie’, ‘Barbara’, ‘Zach’] b_names = filter(lambda s: s.startswith(‘B’), names) print b_names [‘Bob’, ‘Barbara’]

List Comprehensions

All of the six original examples can be achieved using a consistent, clean, and elegant syntax called list comprehensions. Simple list comprehensions

The simplest form of a list comprehension is

```
[ expression-involving-loop-variable for loop-variable in sequence ]
```

This will step over every element of sequence, successively setting loop-variable equal to every element one at a time, and will then build up a list by evaluating expression-involving-loop-variable for each one. This eliminates the need to use lambda forms, and thus generally produces a much more readable code than using map() and a more compact code than using a for-loop.

The first example

squares = [ x**2 for x in range(10) ] print squares [0, 1, 4, 9, 16, 25, 36, 49, 64, 81]

Some other simple examples:

```
Print the length of each word in the list of names:
>>> print [ len(name) for name in names ]
[4, 3, 3, 5, 6, 7, 4]
Print the last letter of each name in the list of names:
>>> print [ name[-1] for name in names ]
['e', 'y', 'b', 'd', 'e', 'a', 'h']
Print the reverse of each name in the list of names:
>>> print [ name[::-1] for name in names ]
['ennA', 'ymA', 'boB', 'divaD', 'eirraC', 'arabraB', 'hcaZ']
```

Note that complex expressions can be put in the slot for expression-involving-loop-variable. For example, here we build up a list of names, first letters, and lengths for each name in the list:

print [ [name, name[0], len(name)] for name in names ] [[‘Anne’, ‘A’, 4], [‘Amy’, ‘A’, 3], [‘Bob’, ‘B’, 3], [‘David’, ‘D’, 5], [‘Carrie’, ‘C’, 6], [‘Barbara’, ‘B’, 7], [‘Zach’, ‘Z’, 4]]

Where [name, name[0], len(name)] occupies the expression-involving-loop-variable slot, so that the list comprehension creates a list of [name, name[0], len(name)] for every name in the names sequence. Nested list comprehensions

List comprehensions can be nested, in which case they take on the following form:

[ expression-involving-loop-variables for outer-loop-variable in outer-sequence for inner-loop-variable in inner-sequence ]

This is equivalent to writing:

results = [] for outer_loop_variable in outer_sequence: for inner_loop_variable in inner_sequence: results.append( expression_involving_loop_variables )

Thus, the third example can be written compactly as:

possible_choices = [ [drink,food] for drink in [‘water’, ‘tea’, ‘juice’] for food in [‘ham’, ‘eggs’, ‘spam’] ] print possible_choices [[‘water’, ‘ham’], [‘water’, ‘eggs’], [‘water’, ‘spam’], [‘tea’, ‘ham’], [‘tea’, ‘eggs’], [‘tea’, ‘spam’], [‘juice’, ‘ham’], [‘juice’, ‘eggs’], [‘juice’, ‘spam’]]

And example 4. can be written as

coords = [ (x,y) for x in range(5) for y in range(3) ] # points on a rectangular grid print coords [(0, 0), (0, 1), (0, 2), (1, 0), (1, 1), (1, 2), (2, 0), (2, 1), (2, 2), (3, 0), (3, 1), (3, 2), (4, 0), (4, 1), (4, 2)]

Filtered list comprehensions

The final form of list comprehension involves creating a list and filtering it similarly to using filter(). The filtering form of list comprehension takes the following form:

```
[ expression-involving-loop-variable for loop-variable in sequence if boolean-expression-involving-loop-variable ]
```

This form is similar to the simple form of list comprehension, but it evaluates boolean-expression-involving-loop-variable for every item and keeps only those members for which the boolean expression is True. Thus we can use list comprehensions to rewrite example 5 as

special_squares = [ x

2 for x in range(10) if x2 > 5 and x**2 < 50 ] print special_squares [9, 16, 25, 36, 49]

Note that the above is inefficient, however, since it has to calculate the square of x three separate times for each element in the loop. Thus, the following is an equivalent and more efficient approach:

squares = [ x**2 for x in range(10) ] special_squares = [ s for s in squares if s > 5 and s < 50 ]

Finally, note that the foregoing can be written on a single line using a pair of list comprehensions as follows:

special_squares = [ s for s in [ x**2 for x in range(10) ] if s > 5 and s < 50 ]

Example 6 can be rewritten using a list comprehension as:

names = [‘Anne’, ‘Amy’, ‘Bob’, ‘David’, ‘Carrie’, ‘Barbara’, ‘Zach’] b_names = [ name for name in names if name.startswith(‘B’) ] print b_names [‘Bob’, ‘Barbara’]

Or, again combining the first two lines into one,

b_names = [ name for name in [‘Anne’, ‘Amy’, ‘Bob’, ‘David’, ‘Carrie’, ‘Barbara’, ‘Zach’] if name.startswith(‘B’) ]

# Map, Filter, Lambda, and List Comprehensions in Python¶

Author: | R.G. Erdmann |
---|

`map()`, `filter()`, `lambda`, and list
comprehensions provide compact, elegant, and efficient ways to encode
a few common idioms in programming. We often encounter the following
scanarios involving for-loops:

## Some for-loop examples to rewrite more compactly¶

Building up a list from scratch by looping over a sequence and performing some calculation on each element in the sequence.

For example, suppose we want to build a list of the squares of the integers from 0 to 9:

>>> squares = [] # start with an empty list >>> for x in range(10): # step over every element in the list of integers from 0 to 9 … squares.append(x

**2) # grow the list one element at a time … >>> print squares [0, 1, 4, 9, 16, 25, 36, 49, 64, 81]**

Suppose we want to build a list of the lengths of the names in a list:

>>> names = ['Anne', 'Amy', 'Bob', 'David', 'Carrie', 'Barbara', 'Zach'] >>> lengths = [] >>> for name in names: … lengths.append(len(name)) … >>> print lengths [4, 3, 3, 5, 6, 7, 4]

Building up a list from scratch by looping over nested sequences.

For example, suppose we want a list of all possible pairs of drink and food from the lists

`[‘water’, ‘tea’, ‘juice’]`and`[‘ham’, ‘eggs’, ‘spam’]`, respectively:>>> possible_choices = [] >>> for drink in ['water', 'tea', 'juice']: … for food in ['ham', 'eggs', 'spam']: … possible_choices.append([drink,food]) … >>> print possible_choices [['water', 'ham'], ['water', 'eggs'], ['water', 'spam'], ['tea', 'ham'], ['tea', 'eggs'], ['tea', 'spam'], ['juice', 'ham'], ['juice', 'eggs'], ['juice', 'spam']]

Suppose we want a list of coordinates on a rectangular grid:

>>> coords = [] >>> for x in range(5): … for y in range(3): … coordinate = (x, y) … coords.append(coordinate) … >>> print coords [(0, 0), (0, 1), (0, 2), (1, 0), (1, 1), (1, 2), (2, 0), (2, 1), (2, 2), (3, 0), (3, 1), (3, 2), (4, 0), (4, 1), (4, 2)]

Building a list from scratch by filtering a sequence according to some criterion or criteria.

For example, suppose we want a list of the squares of the integers from 0 to 9 where the square is greater than 5 and less than 50:

**>>> special_squares = [] >>> for x in range(10): … square = x**2 … if square > 5 and square < 50: … special_squares.append(square) … >>> print special_squares [9, 16, 25, 36, 49]

Suppose we want to take a list of names and find only those starting with the letter B:

>>> names = ['Anne', 'Amy', 'Bob', 'David', 'Carrie', 'Barbara', 'Zach'] >>> b_names = [] >>> for name in names: … if name.startswith('B'): … b_names.append(name) … >>> print b_names ['Bob', 'Barbara']

## Map, Lambda, and Filter¶

One way to achieve the same goals as in the above examples is to use
some of Python’s tools from functional programming: `map()`,
`filter()`, and `lambda()`.

`map()`¶

The `map()` function applies a function to every member of an
iterable and returns the result. Typically, one would use an
anonymous inline function as defined by lambda, but it is possible
to use any function. The *first example above* can also be accomplished
using `map()` as follows:

```
>>> def square(x):
… return x
```**2
…
>>> squares = map(square, range(10))
>>> print squares
[0, 1, 4, 9, 16, 25, 36, 49, 64, 81]
**

**
**

In the case of the *second example*, the `len()` function already
exists, so we can use `map()` to apply it to every element of the
list:

```
>>> names = ['Anne', 'Amy', 'Bob', 'David', 'Carrie', 'Barbara', 'Zach']
>>> lengths = map(len, names)
>>> print lengths
[4, 3, 3, 5, 6, 7, 4]
```

`lambda`¶

**In the first map example above, we created a function, called
square, so that map would have a function to apply to the
sequence. This is a common occurrence, so Python provides the ability
to create a simple (no statements allowed internally) anonymous inline
function using a so-called lambda form.
Thus, an anonymous function that returns the square of its argument
can be written as lambda x: x**

`2`. This means, “Here is an anonymous (nameless) function that takes one arguement, called x, and returns the square of x. Since the lambda form actually evaluates to a function, we can also call it inline. (This is generally a silly thing to do, but we show it here to demonstrate that lambda forms are actually inline function objects.):

`>>> print (lambda x: x`**2)(5) # first parens contain a lambda form that is a squaring function, second parens represent calling that function
25
>>> # Make a function of two arguments (x and y) that returns their product, then call the function with 3 and 4:
>>> print (lambda x, y: x*y)(3, 4)
12
>>> print (lambda x: x.startswith('B'))('Bob')
True
>>> print (lambda x: x.startswith('B'))('Robert')
False
>>> incrementer = lambda input: input+1
>>> # above is same as def incrementer(input): return input+1
>>> # now we call it
>>> print incrementer(3)
4
**

### Using lambda and map together¶

Lambda forms can be used as the required function argument to the
`map()` function. For example, the *first example above* can be written as

**>>> squares = map(lambda x: x**2, range(10))
>>> print squares
[0, 1, 4, 9, 16, 25, 36, 49, 64, 81

In English, this means, “Apply a function that squares its argument
(the lambda form) to every member of the list of integers from 0 to 9
(the `range()`), and store the result in a list called squares.

`filter()`¶

The *fifth* and *sixth* examples above can also be
achieved with the `filter()` built-in function. Filter takes a
function returning `True` or `False` and applies it to a
sequence, returning a list of only those members of the sequence for
which the function returned `True`.

Lambda forms can also be used with the filter function; in fact,
they can be used anywhere a function is expected in Python. In the
*fifth example*, the list of
squares is filtered according to whether the given entries are greater
than 5 and less than 50. A lambda form that returns `True`
when this condition is met is `lambda x: x > 5 and x < 50`. Thus,
we can reproduce the fifth example as follows:

`>>> squares = map(lambda x: x`**2, range(10))
>>> special_squares = filter(lambda x: x > 5 and x < 50, squares)
>>> print special_squares
[9, 16, 25, 36, 49]
**

**
**

In English, this means, “Find every member of the squares list for
which the member is greater than 5 and less than 50 (every member for
which `lambda x: x > 5 and x < 50` returns `True`), and store
that in a new variable called special_squares.

Similarly, the sixth example <sixth-example-list-comprehension> can be reproduced using filter as follows:

```
>>> names = ['Anne', 'Amy', 'Bob', 'David', 'Carrie', 'Barbara', 'Zach']
>>> b_names = filter(lambda s: s.startswith('B'), names)
>>> print b_names
['Bob', 'Barbara']
```

## List Comprehensions¶

All of the six original examples can be achieved using a consistent, clean, and elegant syntax called list comprehensions.

### Simple list comprehensions¶

The simplest form of a list comprehension is

[expression-involving-loop-variableforloop-variableinsequence]

This will step over every element of `sequence`, successively setting
`loop-variable` equal to every element one at a time, and will then
build up a list by evaluating `expression-involving-loop-variable`
for each one. This eliminates the need to use lambda forms, and thus
generally produces a much more readable code than using `map()`
and a more compact code than using a for-loop.

The first example <first-example-list-comprehension> can thus be written compactly as:

**>>> squares = [ x**2 for x in range(10) ]
>>> print squares
[0, 1, 4, 9, 16, 25, 36, 49, 64, 81]

Some other simple examples:

Print the length of each word in the list of

`names`:>>> print [ len(name) for name in names ] [4, 3, 3, 5, 6, 7, 4]

Print the last letter of each name in the list of names:

>>> print [ name[-1] for name in names ] ['e', 'y', 'b', 'd', 'e', 'a', 'h']

Print the reverse of each name in the list of names:

>>> print [ name[::-1] for name in names ] ['ennA', 'ymA', 'boB', 'divaD', 'eirraC', 'arabraB', 'hcaZ']

Note that complex expressions can be put in the slot for
`expression-involving-loop-variable`. For example, here we build up
a list of names, first letters, and lengths for each name in the list:

```
>>> print [ [name, name[0], len(name)] for name in names ]
[['Anne', 'A', 4], ['Amy', 'A', 3], ['Bob', 'B', 3], ['David', 'D', 5], ['Carrie', 'C', 6], ['Barbara', 'B', 7], ['Zach', 'Z', 4]]
```

Where `[name, name[0], len(name)]` occupies the
`expression-involving-loop-variable` slot, so that the list
comprehension creates a list of `[name, name[0], len(name)]` for
every `name` in the `names` sequence.

### Nested list comprehensions¶

List comprehensions can be nested, in which case they take on the following form:

**[** `expression-involving-loop-variables` **for** `outer-loop-variable` **in** `outer-sequence` **for** `inner-loop-variable` **in** `inner-sequence` **]**

This is equivalent to writing:

```
results = []
for outer_loop_variable in outer_sequence:
for inner_loop_variable in inner_sequence:
results.append( expression_involving_loop_variables )
```

Thus, the *third example* can
be written compactly as:

```
>>> possible_choices = [ [drink,food] for drink in ['water', 'tea', 'juice'] for food in ['ham', 'eggs', 'spam'] ]
>>> print possible_choices
[['water', 'ham'], ['water', 'eggs'], ['water', 'spam'], ['tea', 'ham'], ['tea', 'eggs'], ['tea', 'spam'], ['juice', 'ham'], ['juice', 'eggs'], ['juice', 'spam']]
```

And example *4.* can be
written as

```
>>> coords = [ (x,y) for x in range(5) for y in range(3) ] # points on a rectangular grid
>>> print coords
[(0, 0), (0, 1), (0, 2), (1, 0), (1, 1), (1, 2), (2, 0), (2, 1), (2, 2), (3, 0), (3, 1), (3, 2), (4, 0), (4, 1), (4, 2)]
```

### Filtered list comprehensions¶

The final form of list comprehension involves creating a list and
filtering it similarly to using `filter()`. The filtering form
of list comprehension takes the following form:

[expression-involving-loop-variableforloop-variableinsequenceifboolean-expression-involving-loop-variable]

This form is similar to the simple form of list comprehension, but it
evaluates `boolean-expression-involving-loop-variable` for every
item and keeps only those members for which the boolean expression is
`True`. Thus we can use list comprehensions to rewrite
*example 5* as

`>>> special_squares = [ x`**2 for x in range(10) if x**2 > 5 and x**2 < 50 ]
>>> print special_squares
[9, 16, 25, 36, 49]
**

Note that the above is inefficient, however, since it has to calculate
the square of `x` three separate times for each element in the
loop. Thus, the following is an equivalent and more efficient
approach:

**>>> squares = [ x**2 for x in range(10) ]
>>> special_squares = [ s for s in squares if s > 5 and s < 50 ]

Finally, note that the foregoing can be written on a single line using a pair of list comprehensions as follows:

```
>>> special_squares = [ s for s in [ x**2 for x in range(10) ] if s > 5 and s < 50 ]
```

*Example 6* can be rewritten
using a list comprehension as:

```
>>> names = ['Anne', 'Amy', 'Bob', 'David', 'Carrie', 'Barbara', 'Zach']
>>> b_names = [ name for name in names if name.startswith('B') ]
>>> print b_names
['Bob', 'Barbara']
```

Or, again combining the first two lines into one,

```
>>> b_names = [ name for name in ['Anne', 'Amy', 'Bob', 'David', 'Carrie', 'Barbara', 'Zach'] if name.startswith('B') ]
```